newspaint

Documenting Problems That Were Difficult To Find The Answer To

Category Archives: Programming

Checking SSL Certificate Expiry on Remote Server using PowerShell

Overview

There are a number of approaches to take to get the expiry time of the SSL certificate on a remote server using PowerShell. This tutorial will be conducted using PowerShell 2.0 and .NET 3.5 for maximum compatibility (as there are some organisations out there still using Microsoft Windows 2003).

The Simple Way

If you’re reasonably assured your remote server exists and you have connectivity to it then you can write a simple script to:

  • make a TCP connection to the SSL port of the host you wish to check
  • obtain a SSL stream from the TCP connection
  • SSL authenticate as a client
  • obtain the X509 certificate of the remote server from the SSL stream
  • obtain the NotAfter field from the X509 certificate

That script is as follows:

Set-StrictMode -Version 2.0

#Requires -Version 2.0

$HostName = "www.google.com"
$Port = 443

# get TCP connection
[System.Net.Sockets.TcpClient]$TcpClient = $null
$TcpClient = New-Object "System.Net.Sockets.TcpClient"
try {
    $TcpClient.Connect( [System.String]$HostName, [System.Int32]$Port )
} catch {
    Throw "TCP connection error: $_"
}

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

# authenticate SSL stream
try {
    $SslStream.AuthenticateAsClient( $HostName )
} catch {
    Throw "Failed to authenticate SSL stream: $_"
}

# get X509 certificate
[System.Security.Cryptography.X509Certificates.X509Certificate]$cert = $null
$cert = $SslStream.RemoteCertificate

# get X509 certificate with extra properties
[System.Security.Cryptography.X509Certificates.X509Certificate2]$cer2 = $null
$cer2 = New-Object "System.Security.Cryptography.X509Certificates.X509Certificate2" -ArgumentList $cert

# output expiry
$cer2.NotAfter

# close stream and connection
$SslStream.Close()
$TcpClient.Close()

Implementing Timeouts

The fact is that some operations will take a long time when things go wrong. In the code above there are two moments things can block for a long time: making a TCP connection (if the remote end is not responding or the firewall is consuming network traffic), and authenticating the SSL stream (when, for example, the connected service is not SSL and doesn’t response to the authentication process).

In PowerShell we can use the Begin/End form of operations and wait up to a specified number of milliseconds (time) before we give up. The code to do that follows:


Set-StrictMode -Version 2.0

#Requires -Version 2.0

$HostName = "www.google.com"
$Port = 443

# get TCP connection
[System.Net.Sockets.TcpClient]$TcpClient = $null
$TcpClient = New-Object "System.Net.Sockets.TcpClient"
[System.IAsyncResult]$IAsyncResult = $TcpClient.BeginConnect(
    [String]$HostName,
    [System.Int32]$Port,
    $null, # AsyncCallback
    $null # user-defined Object
)

[System.Threading.ManualResetEvent]$AsyncWaitHandle = $null
$AsyncWaitHandle = $IAsyncResult.AsyncWaitHandle

[System.Boolean]$Wait = $AsyncWaitHandle.WaitOne( 5000 ) # 5s timeout

if ( $Wait ) {
    # object was signalled, i.e. connect finished or errored
    try {
        $TcpClient.EndConnect( $IAsyncResult )
        if ( -not $TcpClient.Connected ) {
            Throw "TCP connection not connected!"
        }
    } catch {
        Throw "TCP connection error: $_"
    }
} else {
    # timeout
    $TcpClient.Close() # can't wait for EndConnect, so destroy client
    Throw "TCP connection TIMEOUT"
}

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

# authenticate SSL stream
[System.IAsyncResult]$IAsyncResult = $SslStream.BeginAuthenticateAsClient(
    [String]$HostName,
    $null, # AsyncCallback
    $null # user-defined Object
)

[System.Threading.ManualResetEvent]$AsyncWaitHandle = $null
$AsyncWaitHandle = $IAsyncResult.AsyncWaitHandle

[System.Boolean]$Wait = $AsyncWaitHandle.WaitOne( 5000 ) # 5s timeout

if ( $Wait ) {
    # object was signalled, i.e. authenticate finished or errored
    try {
        $SslStream.EndAuthenticateAsClient( $IAsyncResult )
    } catch {
        Throw "SSL authentication error: $_"
    }
} else {
    # timeout
    $SslStream.Close() # can't wait for authenticate, so destroy stream
    $TcpClient.Close() # close TCP connection
    Throw "SSL authentication TIMEOUT"
}

# get X509 certificate
[System.Security.Cryptography.X509Certificates.X509Certificate]$cert = $null
$cert = $SslStream.RemoteCertificate

# get X509 certificate with extra properties
[System.Security.Cryptography.X509Certificates.X509Certificate2]$cer2 = $null
$cer2 = New-Object "System.Security.Cryptography.X509Certificates.X509Certificate2" -ArgumentList $cert

# output expiry
$cer2.NotAfter

# close stream and connection
$SslStream.Close()
$TcpClient.Close()

Not Requiring Validation of SSL Certification

So, you want to check a SSL certificate’s expiry date, and you don’t really care what the name is on the remote server certificate. You will be getting validation errors by now, like the following:

Exception calling "AuthenticateAsClient" with "1" argument(s): "The remote certificate is invalid according to the validation procedure."

You replace the following lines of code:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

with:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = New-Object System.Net.Security.SslStream(
    $TcpClient.GetStream(),
    $True,
    [System.Net.Security.RemoteCertificateValidationCallback]{ $true }
)

This works fine on the first code example given above without timeouts.

But for the asynchronous code with timeouts this attempt to bypass certificate validation gives the error:

SSL authentication error: Exception calling "EndAuthenticateAsClient" with "1" argument(s): "There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was:  $true "

Okay things are quickly becoming rather tricky rather fast. The issue has been explained elsewhere as:

Asynchronous callback delegates are not a friend to PowerShell. They are serviced by the .NET threadpool which means that if they point to script blocks, there will be no Runspace available to execute them. Runspaces are thread-local resources in the PowerShell threadpool. The .NET threadpool, operating independently, is not too interested in coordinating callbacks with PowerShell. So what do we do?

We’re basically forced to drop into C#/.NET world whether we like it or not. So we might as well provide our own simple class that creates the appropriate callback function.

Add-Type @'
public class MyNoValidate {
  private static System.Boolean bypassvalidation(
    System.Object sender,
    System.Security.Cryptography.X509Certificates.X509Certificate certificate,
    System.Security.Cryptography.X509Certificates.X509Chain chain,
    System.Net.Security.SslPolicyErrors sslPolicyErrors
  ) {
    return true;
  }

  public static System.Net.Security.RemoteCertificateValidationCallback getcallback() {
    System.Net.Security.RemoteCertificateValidationCallback cb;

    cb = new System.Net.Security.RemoteCertificateValidationCallback(
      bypassvalidation
    );

    return cb;
  }
}
'@

and then:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
[System.Net.Security.RemoteCertificateValidationCallback]$Callback = $null
$Callback = [MyNoValidate]::getcallback()
$SslStream = New-Object System.Net.Security.SslStream(
    $TcpClient.GetStream(),
    $True,
    $Callback
)

Now you can get your SSL certificate without having to know the name on the certificate first – with timeouts, too!

Final Note

When getting the expiry time of a SSL certificate please avoid (don’t use) the System.Security.Cryptography.X509Certificates.X509Certificate2.GetExpirationDateString() method! You cannot be sure what you’re getting – whether the date is in USA format or the rest of the world format, or local or UTC time. Much, much better to use the System.Security.Cryptography.X509Certificates.X509Certificate2.NotAfter property of type System.DateTime.

Using HTML::Mason With CGI Provider

So you want to use HTML::Mason (version 1) but your web provider gives you cPanel-like access to CGI scripting only?

Download HTML::Mason from CPAN and extract the contents from the /lib directory into your account, say, into a directory called /lib/perl/mason.

Then create a file, /public_html/cgi-bin/mason_handler.cgi, which contains:

#!/usr/bin/perl

use lib $ENV{"DOCUMENT_ROOT"} . "/../lib/perl/mason";
use HTML::Mason::CGIHandler;

my $h = HTML::Mason::CGIHandler->new(
  data_dir => '/tmp/mason_data',
  allow_globals => [qw(%session $u)],
);

$h->handle_request;

Now you want to configure your Apache to use this handler for Perl Mason webpages in the /public_html/mason directory (Apache v2.2):

<Directory /public_html/mason>
  <FilesMatch "\.html$">
    Action html-mason /cgi-bin/mason_handler.cgi
    SetHandler html-mason

    # for Apache 2.2
    Order allow,deny
    Allow from all

    # for Apache 2.4 (see https://httpd.apache.org/docs/2.4/upgrading.html)
    #Require all granted
  </FilesMatch>

  <FilesMatch "^(autohandler|dhandler)$">
    Action html-mason /cgi-bin/mason_handler.cgi
    SetHandler html-mason

    # for Apache 2.2
    Order allow,deny
    Allow from all

    # for Apache 2.4 (see https://httpd.apache.org/docs/2.4/upgrading.html)
    #Require all granted
  </FilesMatch>
</Directory>

Some CGI website providers require additional Perl modules for HTML::Mason to work, these can all be downloaded and extracted from CPAN:

  • Exception/Class.pm
  • Devel/StackTrace.pm
  • Class/Container.pm
  • Class/Data/Inheritable.pm
  • Params/Validate.pm *
  • Params/ValidatePP.pm *

(the files marked with a * are those that can be downloaded from CPAN and use the command perl Makefile –pm to force native perl code generation).

Using PowerShell 2.0 With Selenium to Automate Internet Explorer, Firefox, and Chrome

PowerShell 2.0 on Windows XP/7 uses .Net 3.5 so the first thing to do is download the Selenium WebDriver.dll file from Selenium’s download page and extract the net35/ directory.

Internet Explorer

Next you want to obtain the Internet Explorer driver from this site. I recommend version 2.41 because “as of 15 April 2014, IE 6 is no longer supported”. This must reside in your current PATH so in your script you may want to modify your PATH to ensure the executable (IEDriverServer.exe) can be found there. If you’re wondering whether to get the 32-bit or the 64-bit version, start with the 32-bit even if you’ve got a 64-bit Windows.

At this point you’ll want to quickly instantiate Internet Explorer and navigate somewhere. Great. Let’s do it.

# Load the Selenium .Net library
Add-Type -Path "N:\selenium\WebDriver.dll" # or wherever your WebDriver.dll is

# Set the PATH to ensure IEDriverServer.exe can found
$env:PATH += ";N:\selenium"

# Instantiate Internet Explorer
$ie_object = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"

This outputs:

New-Object : Exception calling ".ctor" with "0" argument(s): "Request for the permission of type 'System.Net.SocketPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed."
At line:1 char:17
+ $ie = New-Object <<<<  "OpenQA.Selenium.IE.InternetExplorerDriver"
    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand

Wait, what’s this? I don’t know. I just don’t know. It will happen if the DLL is on a network drive and not marked as “trusted” (whatever that means). So copy the DLL onto a local hard drive and try again.

# Load the Selenium .Net library
Add-Type -Path "C:\selenium\WebDriver.dll" # put your DLL on a local hard drive!

# Set the PATH to ensure IEDriverServer.exe can found
$env:PATH += ";N:\selenium"

# Instantiate Internet Explorer
$ie_object = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"

Great! Now we have an Internet Explorer window appear. We can navigate to a new URL:

$ie_object.Navigate().GoToURL( "http://www.bbc.co.uk/languages" )

This worked! The call won’t return until the page download is complete.

Next let’s click on a link from the link text:

$link = $ie_object.FindElementByLinkText( "Spanish" )
$link.Click()

# display current URL
$ie_object.Url

FireFox

Let’s try it with FireFox now. We require the GeckoDriver from the Selenium downloads page. Note that there is no GeckoDriver support for Windows XP at all.

# Set the PATH to ensure geckodriver.exe can found
$env:PATH += ";N:\selenium"

$ff_object = New-Object "OpenQA.Selenium.Firefox.FirefoxDriver"

Chrome

Finally let’s try with Google Chrome. We require the ChromeDriver from the Selenium downloads page.

# Set the PATH to ensure chromedriver.exe can found
$env:PATH += ";N:\selenium"

$chrome_object = New-Object "OpenQA.Selenium.Chrome.ChromeDriver"

Serialising Arrays and Hashes in PowerShell 2.0

So you want to Invoke-Command a scriptblock on another Windows computer but are struggling to communicate results back to the caller because of a lack of serialisation routines in PowerShell 2.0? Yes, PowerShell 3.0 did introduce the ConvertFrom-Json and ConvertTo-Json cmdlets. But if you’re stuck on PowerShell 2.0 then you need another way to send hashes and lists.

Non-Recursive

Why not convert your hash-and-array data structure into a string – one that can be parsed by Invoke-Expression? This is a function that will do exactly that – and it is non-recursive for reasons that will be explained further down (and there’s a simpler recursive function provided later, too):

Function Serialise-Object {
    Param( $Root )

    Function AddAfter-ListNode {
        Param( $LinkedList, $AfterNode, $NewNode )
        if ( $AfterNode -eq $null ) {
            $LinkedList.AddLast( $NewNode )
        } else {
            $LinkedList.AddAfter( $AfterNode, $NewNode )
        }
    }

    Function Escape-SingleQuoted {
        Param( $Source )
        $Source -replace "'", "''"
    }

    # create lists
    $TodoStack = New-Object "System.Collections.Generic.Stack[Object]"
    $StringsList = New-Object "System.Collections.Generic.LinkedList[String]"

    # set up first element
    $TodoStack.Push( @( $Root, $StringsList.Last ) )

    while ( $true ) {
        try {
            $NextTodo = $TodoStack.Pop()
        } catch {
            break
        }

        ( $Item, $Node ) = @( $NextTodo )
        if ( $Item -eq $null ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "`$null"
            AddAfter-ListNode $StringsList $Node $NewStringNode
        } elseif ( $Item.getType().FullName -eq "System.Collections.Hashtable" ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "@{"
            AddAfter-ListNode $StringsList $Node $NewStringNode
            $LastStringNode = $NewStringNode

            $First = $true
            $Item.Keys |ForEach-Object {
                $keyname = ""
                if ( $First ) {
                    $First = $false
                } else {
                    $keyname += ";"
                }
                $keyname += $( "'" + (Escape-SingleQuoted $_) + "'=" )

                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $keyname
                AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
                $LastStringNode = $NewStringNode

                $TodoStack.Push( @( $Item[$_], $LastStringNode ) )
            }

            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "}"
            AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
        } elseif ( $Item.getType().FullName -eq "System.Object[]" ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "@("
            AddAfter-ListNode $StringsList $Node $NewStringNode
            $LastStringNode = $NewStringNode

            $First = $true
            $Item |ForEach-Object {
                if ( $First ) {
                    $First = $false
                } else {
                    $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" ","
                    AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
                    $LastStringNode = $NewStringNode
                }

                $TodoStack.Push( @( $_, $LastStringNode ) )
            }

            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" ")"
            AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
        } else {
            if ( $Item.GetType().FullName -eq "System.String" ) {
                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $( "'" + (Escape-SingleQuoted $Item) + "'" )
                AddAfter-ListNode $StringsList $Node $NewStringNode
            } else {
                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $( "[" + $Item.GetType().FullName + "]'" + (Escape-SingleQuoted $Item.ToString()) + "'" )
                AddAfter-ListNode $StringsList $Node $NewStringNode
            }
        }
    }

    @(
        $StringsList.GetEnumerator() |ForEach-Object { $_ }
    ) -join ""
}

Some examples of output:

> Serialise-Object $null
$null

> Serialise-Object 14.25
[System.Double]'14.25'

> Serialise-Object "Four o'clock"
'Four o''clock'

> Serialise-Object @( "First", "Second", @( "Inner1", "Inner2" ) )
@('First','Second',@('Inner1','Inner2'))

> Serialise-Object @{ "ArrayA" = @( 1, 2.5 ); "ArrayB" = @( 'e', 'f', 'g' ) }
@{'ArrayA'=@([System.Int32]'1',[System.Double]'2.5');'ArrayB'=@('e','f','g')}

> Serialise-Object @{ "OuterH" = @{ "InnerH" = @{ "key1" = [long]0xff } } }
@{'OuterH'=@{'InnerH'=@{'key1'=[System.Int64]'255'}}}

The resulting string can be fed directly into Invoke-Expression and the result is going to be very similar if not identical to the serialised object.

So, how does it work? It iterates over the object it is given. If it is a simple scalar type ($null, string, or non-list/hash) then it is converted to a string, prepended with its type if not a string, and output. If the object is an array or hash then each element or element-pair is iterated through and that object is recursively processed.

This function would have looked a lot simpler as a recursive function. So why was it implemented using lists and stacks instead of recursion? Because I wanted to send this function as a string through an Invoke-Command cmdlet and have it rebuilt as a scriptblock on the remote side; but one problem – how does one call an anonymous scriptblock recursively? Perhaps there’s a way but I don’t know how.

For example:

$remote_scriptblock = {
    Param( [String]$FnSerialiseStr )

    $FnSerialise = [scriptblock]::create( $FnSerialiseStr )

    $Start = Get-Date
    Start-Sleep -Milliseconds 1500

    & $FnSerialise @{ "time"=((Get-Date) - $Start).TotalSeconds }
}

Invoke-Command $remote_scriptblock -ArgumentList @(${function:Serialise-Object})

This outputs:

@{'time'=[System.Double]'1.5'}

Pretty neat, huh? You can send this function to the other side and run it!

Recursive

Ah.. but what if you want to send several named functions to the other side?

$remote_scriptblock = {
    Param( [String]$PreBlockStr )

    $PreBlock = [scriptblock]::create( $PreBlockStr )
    & $PreBlock

    $Start = Get-Date
    Start-Sleep -Milliseconds 2500

    Serialise-Object @{ "time"=((Get-Date) - $Start).TotalSeconds }
}
Invoke-Command $remote_scriptblock -ArgumentList @("Function Serialise-Object { ${function:Serialise-Object} }")

This outputs:

@{'time'=[System.Double]'2.5'}

Well that solves the problem of a recursive function trying to call itself.

Let’s rewrite the serialisation function as the simpler recursive form:

Function Serialise-Object {
    Param( $Root )

    Function Escape-SingleQuoted {
        Param( $Source )
        $Source -replace "'", "''"
    }

    if ( $Root -eq $null ) {
        "`$null"
    } elseif ( $Root.getType().FullName -eq "System.Collections.Hashtable" ) {
        $out = "@{"

        $First = $true
        $Root.Keys |ForEach-Object {
            if ( $First ) {
                $First = $false
            } else {
                $out += ";"
            }

            $out += $( "'" + (Escape-SingleQuoted $_) + "'=" )
            $out += Serialise-Object $Root[$_]
        }
        $out + "}"
    } elseif ( $Root.getType().FullName -eq "System.Object[]" ) {
        $out = "@("

        $First = $true
        $Root |ForEach-Object {
            if ( $First ) {
                $First = $false
            } else {
                $out += ","
            }

            $out += Serialise-Object $_
        }
        $out + ")"
    } else {
        if ( $Root.GetType().FullName -eq "System.String" ) {
            $( "'" + (Escape-SingleQuoted $Root) + "'" )
        } else {
            $( "[" + $Root.GetType().FullName + "]'" + (Escape-SingleQuoted $Root.ToString()) + "'" )
        }
    }
}

With this simpler code we can use it remotely as follows:

$remote_scriptblock = {
    Param( [String]$PreBlockStr )

    $PreBlock = [scriptblock]::create( $PreBlockStr )
    & $PreBlock

    $Start = Get-Date
    Start-Sleep -Milliseconds 3500

    Serialise-Object @{ "time"=((Get-Date) - $Start).TotalSeconds }
}
Invoke-Command $remote_scriptblock -ArgumentList @("Function Serialise-Object { ${function:Serialise-Object} }")

This outputs:

@{'time'=[System.Double]'3.5'}

… using simpler code.

Allowing Powershell Scripts to Run on Windows

So you want to run a PowerShell script on your Windows system but get the following message:

File myscript.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please
see "get-help about_signing" for more details.
At line:1 char:23
+ .\myscript.ps1 <<<<
    + CategoryInfo          : NotSpecified: (:) [], PSSecurityException
    + FullyQualifiedErrorId : RuntimeException

A fix is to change the execution policy. To see the current state of execution policies on your system:

PS C:\> Get-ExecutionPolicy -List |Format-Table -AutoSize

        Scope ExecutionPolicy
        ----- ---------------
MachinePolicy       Undefined
   UserPolicy       Undefined
      Process       Undefined
  CurrentUser       Undefined
 LocalMachine       Undefined

If you do not have administrator rights on your system the easiest way to allow scripts to be executed is to allow the current user to run scripts.

PS C:\> Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic. Do you want to change the execution
policy?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): y

Note that “RemoteSigned” means that all scripts and configuration files downloaded from the Internet must be signed by a trusted publisher, but locally authored scripts can be run without a signature.

Using Perl to Make Signed Requests to Public Transport Victoria Timetable API

So you want to create an application to access the Public Transport Victoria (PTV) Timetable API.

You’ve followed the instructions and e-mailed a request for a developer ID and an access key; and you’ve received both in an e-mail that contained text similar to the following:

Thank you for your interest in the PTV Timetable API.

Your email address has now been registered and your user Id and API key are
below.

User Id: 2912345

API Key: 4cc12345-ff11-2222-a00a-dd1297cd04aa

Now you want to create a signed request using Perl to access the API.

The following function will take a URL and return a signed URL that can be used to access that URL:

#!/usr/bin/perl -w

use Digest::HMAC;
use Digest::SHA;

use strict;

my $devid = "2912345";
my $apikey = "4cc12345-ff11-2222-a00a-dd1297cd04aa";

sub sign_url {
  my ( $url ) = @_;

  # add ?devid=$devid or &devid=$devid to URL
  $url .= ( index($url, "?") < $[ ) ? "?" : "&";
  $url .= "devid=$devid";

  # strip out base URL from part used in signing
  my $request = $url;
  $request =~ s{^(([a-z]+)://+)?[^/]+}{};

  # calculate signature using API key and URL without base
  #   e.g. sign over a string like "/v3/routes?devid=2912345"
  my $signature = Digest::HMAC::hmac_hex(
      $request,
      $apikey,
      \&Digest::SHA::sha1
  );

  $url .= "&signature=" . $signature;
  return $url;
}

print sign_url( "http://timetableapi.ptv.vic.gov.au/v3/routes" );

This should output the following:

$ perl -w signtest.pl
http://timetableapi.ptv.vic.gov.au/v3/routes?devid=2912345&signature=b7ee928f05499a0016746daef5013dba35224d8e

The example is using an invalid signature and devid, so this example URL will not actually return a page; you will have to provide the access key and devid you received in your e-mail.

Note: it is absolutely essential that you do not change the capitalisation of the access key in this script. While it looks like a hex string it is actually treated as case-sensitive text.

Rust/Cargo Outputting Dollar Angle Bracket Symbols

So you’re compiling source using Cargo/Rust and you see output similar to the following:

error[E0046]$<2>: not all trait items implemented, missing: `decode`, `encode`$<2>
  $<2>--> $<2>src/main.rs:7:1$<2>
   $<2>|$<2>
7$<2>  $<2>| $<2>impl tokio_core::io::Codec for LineCodec {$<2>
   $<2>| $<2>^$<2> missing `decode`, `encode` in implementation$<2>

error$: aborting due to previous error$

This is full of dollar, angle bracket, number, angle bracket characters.

The fix is to change your terminal, e.g.:

TERM=xterm

On doing this I had the output in full colour without the mysterious symbols.

Implementing Custom Sort For BTree In Rust

So you want to use a std::collections::btree_map::BTreeMap but you want your keys sorted in a different way to the default for the type.

In this example I show how you can provide a BTreeMap that has String keys but sorted in reverse order. But, by altering the cmp() function for the std::cmp::Ord trait for a custom type, you can make the sort in any order you desire.

It is a bit of a pain, however. As this post points out you have to define partial_cmp() and partial_eq() functions from other traits as well, as well as adding the Eq trait to your struct.

// http://rustbyexample.com/custom_types/structs.html
struct RString {
    s: String,
}

use std::cmp::Ord;
use std::cmp::Ordering;

impl PartialEq for RString {
    fn eq(&self, other:&Self) -> bool {
        if self.s == other.s {
            true
        } else {
            false
        }
    }
}

// this does not actually have any methods, it's just a flag on the type
impl Eq for RString { }

// make partial_cmp() just return result from cmp()
impl PartialOrd for RString {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        let me: &String = &(self.s);
        let them: &String = &(other.s);

        Some( me.cmp( them ) )
    }
}

impl Ord for RString {
    fn cmp(&self, other:&Self) -> Ordering {
        let me: &String = &(self.s);
        let them: &String = &(other.s);

        if me > them {
            Ordering::Less
        } else if me < them {
            Ordering::Greater
        } else {
            Ordering::Equal
        }
    }
}

use std::env;
use std::collections::btree_map::BTreeMap;
fn main() {
    // collect environment variable keys into a vector we can sort
    let mut sortedmap : BTreeMap<Box<RString>,Box<String>> = BTreeMap::new();

    for (key, value) in env::vars() {
        sortedmap.insert(
            Box::<RString>::new( RString { s: key } ),
            Box::<String>::new( value )
        );
    }

    for (key, value) in sortedmap {
        println!( "{} => {}", (*key).s, *value );
    }
}

This could also be implemented using a tuple containing a single element:

// http://rustbyexample.com/custom_types/structs.html
struct RString(String);

use std::cmp::Ord;
use std::cmp::Ordering;

impl PartialEq for RString {
    fn eq(&self, other:&Self) -> bool {
        if self.0 == other.0 {
            true
        } else {
            false
        }
    }
}

// this does not actually have any methods, it's just a flag on the type
impl Eq for RString { }

// make partial_cmp() just return result from cmp()
impl PartialOrd for RString {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        let me: &String = &(self.0);
        let them: &String = &(other.0);

        Some( me.cmp( them ) )
    }
}

impl Ord for RString {
    fn cmp(&self, other:&Self) -> Ordering {
        let me: &String = &(self.0);
        let them: &String = &(other.0);

        if me > them {
            Ordering::Less
        } else if me < them {
            Ordering::Greater
        } else {
            Ordering::Equal
        }
    }
}

use std::env;
use std::collections::btree_map::BTreeMap;
fn main() {
    // collect environment variable keys into a vector we can sort
    let mut sortedmap : BTreeMap<Box<RString>,Box<String>> = BTreeMap::new();

    for (key, value) in env::vars() {
        sortedmap.insert(
            Box::<RString>::new( RString(key) ),
            Box::<String>::new( value )
        );
    }

    for (key, value) in sortedmap {
        println!( "{} => {}", (*key).0, *value );
    }
}

Note that a tuple’s single element can be accessed through the .0 field.

Creating a Basic CGI Executable Using Rust

I created a new cargo project called “cgitest”:

~$ cargo new cgitest --bin
~$ cd cgitest
~/cgitest$

Then I added the following Rust code into src/main.rs:

// Use the following as a guide:
//   http://httpd.apache.org/docs/current/howto/cgi.html

use std::io::Write;
use std::env;
use std::collections::btree_map::BTreeMap;

fn write_stderr( msg : String ) {
    let mut stderr = std::io::stderr();
    write!(&mut stderr, "{}", msg).unwrap();
}

fn write_stderr_s( msg : &str ) {
    write_stderr( msg.to_string() );
}

fn write_stdout( msg : String ) {
    let mut stdout = std::io::stdout();
    write!(&mut stdout, "{}", msg).unwrap();
}

fn write_stdout_s( msg : &str ) {
    write_stdout( msg.to_string() );
}

fn html_escape( msg : String ) -> String {
    let mut copy : String = String::with_capacity( msg.len() );

    for thechar in msg.chars() {
        if thechar == '&' {
            copy.push_str( "&amp;" );
        } else if thechar == '<' {
            copy.push_str( "&lt;" );
        } else if thechar == '>' {
            copy.push_str( "&gt;" );
        } else if thechar == '\"' {
            copy.push_str( "&quot;" );
        } else {
            copy.push( thechar );
        }
    }

    return copy;
}

fn main() {
    write_stdout_s( "Content-type: text/html\n" );
    write_stdout_s( "\n" );
    write_stdout_s( "<html>\n" );
    write_stdout_s( "  <head>\n" );
    write_stdout_s( "    <title>Rust CGI Test</title>\n" );
    write_stdout_s( "    <style type=\"text/css\">\n" );
    write_stdout_s( "      td { border:1px solid black; }\n" );
    write_stdout_s( "      td { font-family:monospace; }\n" );
    write_stdout_s( "      table { border-collapse:collapse; }\n" );
    write_stdout_s( "    </style>\n" );
    write_stdout_s( "  </head>\n" );
    write_stdout_s( "  <body>\n" );
    write_stdout_s( "    <h1>Environment</h1>\n" );
    write_stdout_s( "    <table>\n" );
    write_stdout_s( "      <tr><th>Key</th><th>Value</th></tr>\n" );

    // copy environment into a BTreeMap which is sorted
    let mut sortedmap : BTreeMap<String,String> = BTreeMap::new();
    for (key, value) in env::vars() {
        sortedmap.insert( key, value );
    }

    // output environment into HTML table
    for (key, value) in sortedmap {
        write_stdout(
            format!(
                "      <tr><td>{}</td><td>{}</td></tr>\n",
                html_escape( key ),
                html_escape( value )
            )
        );
    }
    write_stdout_s( "    </table>\n" );
    write_stdout_s( "  </body>\n" );
    write_stdout_s( "</html>\n" );
}

Compiling it using the cargo build --release command produced a binary in target/release/cgitest which can be launched from Apache as a CGI script.

Where Is The Std Crate Located In A Rust Installation

I downloaded Rust (the programming language) from the Rust downloads page. Specifically I downloaded the 64-bit Linux .tar.gz file and then extracted it to /opt/ on my filesystem and symlinked /opt/rust to the untarred directory.

However when I went to compile a test Rust program I got the following error:

$ /opt/rust/rustc/bin/rustc test.rs
error: can't find crate for `std` [E0463]
error: aborting due to previous error

I searched for the std crate by running:

$ find /opt/rust/ -name 'libstd*'
/opt/rust/rustc/lib/libstd-d16b8f0e.so
/opt/rust/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
/opt/rust/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.rlib
/opt/rust/rust-docs/share/doc/rust/html/src/std/up/src/libstd

I tried then compiling my Rust program using the first path, “/opt/rust/rustc/lib/“:

$ /opt/rust/rustc/bin/rustc -L /opt/rust/rustc/lib test.rs
error:error: can't find crate for `core` which `std` depends on [E0463]
error:error: aborting due to previous error

So, where is libcore?

$ find /opt/rust/ -name 'libcore*'
/opt/rust/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-d16b8f0e.rlib
/opt/rust/rust-docs/share/doc/rust/html/src/core/up/src/libcore

I then tried compiling using the path to libcore.

$ /opt/rust/rustc/bin/rustc -L /opt/rust/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib test.rs
$

Success!

If you want to pass command line options for rustc to cargo you can do it as follows:

$ RUSTFLAGS="-L /opt/rust/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" cargo
$