newspaint

Documenting Problems That Were Difficult To Find The Answer To

Prioritising VoIP on OpenWRT/OpenLEDE

Following the advice at the Network Traffic Control (QOS) page on the OpenWRT wiki I installed the tc package:

opkg install tc
opkg install kmod-sched-core

I wanted to set up 4 queues:

  ---[traffic to Australian Phone Company]---
                                             \
  ---[tiny packets <64 bytes]-----------------> router ---[ADSL]--- ISP
                                             /
  ---[small packets <256 bytes]-------------/
                                           /
  ---[everything else]---------------------

I needed a root qdisc with no limitations on it, and a top-level class with a maximum bandwidth of the upload speed to my ISP (around 1,400 kilobits per second).

Beneath this I set up 4 classes with differing priorities and each with a guaranteed minimum bandwidth of 100 kilobits per second each.

#!/bin/sh

DEV=pppoe-adsl
BWMAX=1400kbit

# ensure 32-bit classifier is available
insmod cls_u32
# ensure HTB scheduler is available
insmod sch_htb

echo "Clearing existing root qdisc on $DEV"
tc qdisc del dev $DEV root

echo "Adding root qdisc on $DEV"
tc qdisc add dev $DEV root       handle 1:    htb default 99

echo "Adding top class on $DEV with rate $BWMAX"
tc class add dev $DEV parent 1:  classid 1:1  htb rate $BWMAX ceil $BWMAX burst 6k


# Classes
echo "Creating VoIP class"
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 100kbit ceil $BWMAX burst 6k prio 1

echo "Creating tiny packet class <64 bytes"
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 100kbit ceil $BWMAX burst 6k prio 1

echo "Creating small packet class <256 bytes"
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 100kbit ceil $BWMAX burst 6k prio 3

echo "Creating default class"
tc class add dev $DEV parent 1:1 classid 1:99 htb rate 100kbit ceil $BWMAX burst 6k prio 6


# Filters
echo "Creating VoIP filter to Australian Phone Company only (highest priority traffic)"
tc filter add dev $DEV parent 1: protocol ip prio 2 u32 match ip dst 103.12.10.97/32 flowid 1:10

echo "Creating tiny packet filter <64 bytes (for acknowledgements)"
echo "  allows bottom 6 bits to be anything (2^6 = 64) but all higher bits must be zero"
tc filter add dev $DEV parent 1: protocol ip prio 3 u32 match u16 0x0000 0xffc0 at 2 flowid 1:20

echo "Creating small packet filter <256 bytes (for traffic more likely to be real-time)"
echo "  allows bottom 8 bits to be anything (2^8 = 256) but all higher bits must be zero"
tc filter add dev $DEV parent 1: protocol ip prio 4 u32 match u16 0x0000 0xff00 at 2 flowid 1:30

Viewing the state of the system was straight-forward.

Seeing how many bytes and packets made it through the root qdisc:

# tc -s qdisc show dev $DEV
qdisc htb 1: root refcnt 2 r2q 10 default 99 direct_packets_stat 0 direct_qlen 3
 Sent 20016888 bytes 166952 pkt (dropped 175, overlimits 1496 requeues 0) 
 backlog 0b 0p requeues 0

I could also view the classes to see how many packets had made their way through each one:

# tc -s class show dev $DEV
class htb 1:1 root rate 1400Kbit ceil 1400Kbit burst 6Kb cburst 1599b 
 Sent 20154394 bytes 167785 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 6096 borrowed: 0 giants: 0
 tokens: 544990 ctokens: 139271

class htb 1:10 parent 1:1 prio 1 rate 100Kbit ceil 1400Kbit burst 6Kb cburst 1599b 
 Sent 218765 bytes 394 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 394 borrowed: 0 giants: 0
 tokens: 6248750 ctokens: 40610

class htb 1:20 parent 1:1 prio 1 rate 100Kbit ceil 1400Kbit burst 6Kb cburst 1599b 
 Sent 5800369 bytes 112832 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 109430 borrowed: 3402 giants: 0
 tokens: 7630000 ctokens: 139271

class htb 1:30 parent 1:1 prio 3 rate 100Kbit ceil 1400Kbit burst 6Kb cburst 1599b 
 Sent 4727569 bytes 40204 pkt (dropped 16, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 38041 borrowed: 2163 giants: 0
 tokens: 7586250 ctokens: 136146

class htb 1:99 parent 1:1 prio 6 rate 100Kbit ceil 1400Kbit burst 6Kb cburst 1599b 
 Sent 9407691 bytes 14355 pkt (dropped 159, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 13824 borrowed: 531 giants: 0
 tokens: 6747500 ctokens: 76235

Actually class 1:99 is displayed first.

Aldi WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill and Batteries

Aldi, today, had for sale the cordless hammer drill titled “WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill”. I spent some time searching the Internet to try and find out the manufacturer but I was unable. As this is the first time I know of that these products have been released to market I am writing this post with pictures of the boxes and products for others.


Battery Pack

Let’s start with the battery pack because without the battery there is no appliance.

The battery is titled “XFinity Plus 20V Battery System XFinity Li-Ion 20V 2.0AH Battery” and the battery purchased had a capacity of 2.0 amp hour (AH). The following is the view of the box from different angles:

XFinity Plus 20V Li-Ion 2.0AH Battery Box Top View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Top View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Left View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Left View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Front View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Front View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Right View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Right View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Rear View

XFinity Plus 20V Li-Ion 2.0AH Battery Box Rear View

XFinity Plus 20V Li-Ion 2.0AH Battery In Plastic Bag

XFinity Plus 20V Li-Ion 2.0AH Battery In Plastic Bag

XFinity Plus 20V Li-Ion 2.0AH Battery Manual

XFinity Plus 20V Li-Ion 2.0AH Battery Manual

XFinity Plus 20V Li-Ion 2.0AH Battery Top View

XFinity Plus 20V Li-Ion 2.0AH Battery Top View

XFinity Plus 20V Li-Ion 2.0AH Battery Left View

XFinity Plus 20V Li-Ion 2.0AH Battery Left View

XFinity Plus 20V Li-Ion 2.0AH Battery Bottom View

XFinity Plus 20V Li-Ion 2.0AH Battery Bottom View


Battery Charger

A battery isn’t much good unless the battery can be charged. So a charger was purchased. The charger is titled “XFinity Plus 20V Battery System XFinity Li-Ion 20V Quick Charger”. The following is the view of the box from different angles:

XFinity Plus 20V Li-Ion Quick Charger Box Top View

XFinity Plus 20V Li-Ion Quick Charger Box Top View

XFinity Plus 20V Li-Ion Quick Charger Box Left View

XFinity Plus 20V Li-Ion Quick Charger Box Left View

XFinity Plus 20V Li-Ion Quick Charger Box Front View

XFinity Plus 20V Li-Ion Quick Charger Box Front View

XFinity Plus 20V Li-Ion Quick Charger Box Right View

XFinity Plus 20V Li-Ion Quick Charger Box Right View

XFinity Plus 20V Li-Ion Quick Charger Box Rear View

XFinity Plus 20V Li-Ion Quick Charger Box Rear View

XFinity Plus 20V Li-Ion Quick Charger In Plastic Bag

XFinity Plus 20V Li-Ion Quick Charger In Plastic Bag

XFinity Plus 20V Li-Ion Quick Charger Manual In Plastic Bag

XFinity Plus 20V Li-Ion Quick Charger Manual In Plastic Bag

XFinity Plus 20V Li-Ion Quick Charger Top View

XFinity Plus 20V Li-Ion Quick Charger Top View

XFinity Plus 20V Li-Ion Quick Charger Bottom View

XFinity Plus 20V Li-Ion Quick Charger Bottom View

When power is applied the green LED is lit.

XFinity Plus 20V Li-Ion Quick Charger With Green LED Lit

XFinity Plus 20V Li-Ion Quick Charger With Green LED Lit

When a battery is charging the red LED flashes. When charging is complete the red LED stays lit and does not flash (one must remove the battery when the red LED is steady according to the manual).

XFinity Plus 20V Li-Ion Quick Charger With Red LED Flashing During Battery Charge

XFinity Plus 20V Li-Ion Quick Charger With Red LED Flashing During Battery Charge


Cordless Hammer Drill

The cordless hammer drill is titled “WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill”.

According to the box the technical specs are:

  • voltage: 20Vd.c.
  • no load speed: 0-380/0-1400/min
  • torque setting: 21
  • maximum torque: 28Nm
  • chuck diameter: 13mm
  • LED worklight: yes

The model is PT160103, version number 0001, product code is 56226, and the date is listed as 04/2017. After sales support line is given to be 1300 777 137 with an e-mail of service@actionspares.com.au.

Contents are listed as:

  • 1 x Hammer drill
  • 1 x Belt hook
  • 1 x Double ended bit
  • 1 x Auxiliary handle
  • 1 x Instruction manual
  • 1 x Warranty certificate

The box states “made in China”.

The following is the view of the box from different angles:

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Top View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Top View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Left View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Left View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Opened

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Box Opened

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Bottom View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill Bottom View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill With Battery Pack

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Hammer Drill With Battery Pack

According to the manual there are 3 function types:

  • (icon of drill bit) regular drilling setting for drilling wood, plastic, metal
  • (icon of hammer) hammer drilling setting for hammer drilling masonry
  • (triangular-ish icon) driving setting for tightening or loosening screws

The torque setting is from 1 to 21, 1 being the lowest torque, 21 being the highest.

The speed setting on top of the drill is 1 (slow/low) or 2 (fast/high). High speed has less torque.

The product claims to adhere to technical standards AS/NZS 60745.1, 60745.2.1, and 60745.2.2.

The drill has considerable “whine” – a high pitch tone that changes in pitch depending on how far in the trigger is pulled.

Sadly the first drill I bought failed to observe any of the torque settings (it would drive regardless of how tight it became). I took it back and replaced it with another that now correctly observes the torque setting.

Impact Driver

The impact driver is titled “WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver”.

The technical specifications on the box are:

  • input: 20Vd.c.
  • chuck size: 6.35mm Hex
  • no load speed: 0-2,300/min
  • impact rate: 0-3,200bpm
  • max torque: 140Nm

The following is the view of the box from different angles:

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Top View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Top View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Left View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Left View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Bottom View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Bottom View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Open View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Box Open View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Front View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Rear View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Right View

WorkZone Titanium+ XFinity Li-Ion 20V Cordless Impact Driver Right View

ZFS Grub Issues on Boot

I had a problem when attempting to boot into my ZFS root and landed in initramfs rescue prompt.

Using advice from this article:

Command: zpool import -N
Message: cannot import '': no such pool available
Error: 1

Manually import the root pool at the command prompt and then exit.
Hint: Try:  zpool import -f -R / -N


BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs) zpool import -f -R / -N rpool
(initramfs) exit

Begin: Setting mountpoint=/ on ZFS filesystem  ... done
Begin: Mounting ZFS filesystem  ... done
Command: mount -t zfs -o zfsutil  /root
Message: filesystem '' cannot be mounted, unable to open the dataset
mount: mounting  on /root failed: No such file or directory
Error: 1

Manually mount the root filesystem on /root and then exit.


BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) build-in shell (ash)
Enter 'help' for a list of built-in commands

(initramfs)

To fix this issue I ran:

(initramfs) zpool import -R /root rpool
(initramfs) exit

Unfortunately there is a known bug in Ubuntu 16.04.1 grub-probe command which states “error: unknown filesystem” when running update-grub.

The work-around is to update /etc/default/grub and add:

GRUB_CMDLINE_LINUX_DEFAULT="boot=zfs root=ZFS=rpool/ROOT"

This results in the grub menu specifying the root parameter twice on the kernel boot line in /etc/grub/grub.cfg but the second one takes precedence.

Monitor No Signal on Xubuntu 16.04.1

My Dell server seemed to stop outputting to the monitor on the VGA cable. No signal, the monitor said. It was blank, it was black, it was powered off. I tried unplugging the cable and plugging it back in, no joy.

I tried pressing ctrl-alt-1 to switch to the text console, and the screen came alive, but all I could see was a flashing underline of a cursor in the upper left-hand corner, no login prompt. Same thing for ctrl-alt-2. Tried ctrl-alt-7 to get back to graphics mode and the monitor turned off again.

The following repaired the issue for me without having to reboot, but it did kill my GUI session and all open windows:

sudo /etc/init.d/lightdm restart

My monitor came back alive and I found myself at the GUI XFCE login prompt.

Checking SSL Certificate Expiry on Remote Server using PowerShell

Overview

There are a number of approaches to take to get the expiry time of the SSL certificate on a remote server using PowerShell. This tutorial will be conducted using PowerShell 2.0 and .NET 3.5 for maximum compatibility (as there are some organisations out there still using Microsoft Windows 2003).

The Simple Way

If you’re reasonably assured your remote server exists and you have connectivity to it then you can write a simple script to:

  • make a TCP connection to the SSL port of the host you wish to check
  • obtain a SSL stream from the TCP connection
  • SSL authenticate as a client
  • obtain the X509 certificate of the remote server from the SSL stream
  • obtain the NotAfter field from the X509 certificate

That script is as follows:

Set-StrictMode -Version 2.0

#Requires -Version 2.0

$HostName = "www.google.com"
$Port = 443

# get TCP connection
[System.Net.Sockets.TcpClient]$TcpClient = $null
$TcpClient = New-Object "System.Net.Sockets.TcpClient"
try {
    $TcpClient.Connect( [System.String]$HostName, [System.Int32]$Port )
} catch {
    Throw "TCP connection error: $_"
}

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

# authenticate SSL stream
try {
    $SslStream.AuthenticateAsClient( $HostName )
} catch {
    Throw "Failed to authenticate SSL stream: $_"
}

# get X509 certificate
[System.Security.Cryptography.X509Certificates.X509Certificate]$cert = $null
$cert = $SslStream.RemoteCertificate

# get X509 certificate with extra properties
[System.Security.Cryptography.X509Certificates.X509Certificate2]$cer2 = $null
$cer2 = New-Object "System.Security.Cryptography.X509Certificates.X509Certificate2" -ArgumentList $cert

# output expiry
$cer2.NotAfter

# close stream and connection
$SslStream.Close()
$TcpClient.Close()

Implementing Timeouts

The fact is that some operations will take a long time when things go wrong. In the code above there are two moments things can block for a long time: making a TCP connection (if the remote end is not responding or the firewall is consuming network traffic), and authenticating the SSL stream (when, for example, the connected service is not SSL and doesn’t response to the authentication process).

In PowerShell we can use the Begin/End form of operations and wait up to a specified number of milliseconds (time) before we give up. The code to do that follows:


Set-StrictMode -Version 2.0

#Requires -Version 2.0

$HostName = "www.google.com"
$Port = 443

# get TCP connection
[System.Net.Sockets.TcpClient]$TcpClient = $null
$TcpClient = New-Object "System.Net.Sockets.TcpClient"
[System.IAsyncResult]$IAsyncResult = $TcpClient.BeginConnect(
    [String]$HostName,
    [System.Int32]$Port,
    $null, # AsyncCallback
    $null # user-defined Object
)

[System.Threading.ManualResetEvent]$AsyncWaitHandle = $null
$AsyncWaitHandle = $IAsyncResult.AsyncWaitHandle

[System.Boolean]$Wait = $AsyncWaitHandle.WaitOne( 5000 ) # 5s timeout

if ( $Wait ) {
    # object was signalled, i.e. connect finished or errored
    try {
        $TcpClient.EndConnect( $IAsyncResult )
        if ( -not $TcpClient.Connected ) {
            Throw "TCP connection not connected!"
        }
    } catch {
        Throw "TCP connection error: $_"
    }
} else {
    # timeout
    $TcpClient.Close() # can't wait for EndConnect, so destroy client
    Throw "TCP connection TIMEOUT"
}

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

# authenticate SSL stream
[System.IAsyncResult]$IAsyncResult = $SslStream.BeginAuthenticateAsClient(
    [String]$HostName,
    $null, # AsyncCallback
    $null # user-defined Object
)

[System.Threading.ManualResetEvent]$AsyncWaitHandle = $null
$AsyncWaitHandle = $IAsyncResult.AsyncWaitHandle

[System.Boolean]$Wait = $AsyncWaitHandle.WaitOne( 5000 ) # 5s timeout

if ( $Wait ) {
    # object was signalled, i.e. authenticate finished or errored
    try {
        $SslStream.EndAuthenticateAsClient( $IAsyncResult )
    } catch {
        Throw "SSL authentication error: $_"
    }
} else {
    # timeout
    $SslStream.Close() # can't wait for authenticate, so destroy stream
    $TcpClient.Close() # close TCP connection
    Throw "SSL authentication TIMEOUT"
}

# get X509 certificate
[System.Security.Cryptography.X509Certificates.X509Certificate]$cert = $null
$cert = $SslStream.RemoteCertificate

# get X509 certificate with extra properties
[System.Security.Cryptography.X509Certificates.X509Certificate2]$cer2 = $null
$cer2 = New-Object "System.Security.Cryptography.X509Certificates.X509Certificate2" -ArgumentList $cert

# output expiry
$cer2.NotAfter

# close stream and connection
$SslStream.Close()
$TcpClient.Close()

Not Requiring Validation of SSL Certification

So, you want to check a SSL certificate’s expiry date, and you don’t really care what the name is on the remote server certificate. You will be getting validation errors by now, like the following:

Exception calling "AuthenticateAsClient" with "1" argument(s): "The remote certificate is invalid according to the validation procedure."

You replace the following lines of code:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = $TcpClient.GetStream()

with:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
$SslStream = New-Object System.Net.Security.SslStream(
    $TcpClient.GetStream(),
    $True,
    [System.Net.Security.RemoteCertificateValidationCallback]{ $true }
)

This works fine on the first code example given above without timeouts.

But for the asynchronous code with timeouts this attempt to bypass certificate validation gives the error:

SSL authentication error: Exception calling "EndAuthenticateAsClient" with "1" argument(s): "There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was:  $true "

Okay things are quickly becoming rather tricky rather fast. The issue has been explained elsewhere as:

Asynchronous callback delegates are not a friend to PowerShell. They are serviced by the .NET threadpool which means that if they point to script blocks, there will be no Runspace available to execute them. Runspaces are thread-local resources in the PowerShell threadpool. The .NET threadpool, operating independently, is not too interested in coordinating callbacks with PowerShell. So what do we do?

We’re basically forced to drop into C#/.NET world whether we like it or not. So we might as well provide our own simple class that creates the appropriate callback function.

Add-Type @'
public class MyNoValidate {
  private static System.Boolean bypassvalidation(
    System.Object sender,
    System.Security.Cryptography.X509Certificates.X509Certificate certificate,
    System.Security.Cryptography.X509Certificates.X509Chain chain,
    System.Net.Security.SslPolicyErrors sslPolicyErrors
  ) {
    return true;
  }

  public static System.Net.Security.RemoteCertificateValidationCallback getcallback() {
    System.Net.Security.RemoteCertificateValidationCallback cb;

    cb = new System.Net.Security.RemoteCertificateValidationCallback(
      bypassvalidation
    );

    return cb;
  }
}
'@

and then:

# get SSL stream from TCP connection
[System.Net.Security.SslStream]$SslStream = $null
[System.Net.Security.RemoteCertificateValidationCallback]$Callback = $null
$Callback = [MyNoValidate]::getcallback()
$SslStream = New-Object System.Net.Security.SslStream(
    $TcpClient.GetStream(),
    $True,
    $Callback
)

Now you can get your SSL certificate without having to know the name on the certificate first – with timeouts, too!

Final Note

When getting the expiry time of a SSL certificate please avoid (don’t use) the System.Security.Cryptography.X509Certificates.X509Certificate2.GetExpirationDateString() method! You cannot be sure what you’re getting – whether the date is in USA format or the rest of the world format, or local or UTC time. Much, much better to use the System.Security.Cryptography.X509Certificates.X509Certificate2.NotAfter property of type System.DateTime.

Using HTML::Mason With CGI Provider

So you want to use HTML::Mason (version 1) but your web provider gives you cPanel-like access to CGI scripting only?

Download HTML::Mason from CPAN and extract the contents from the /lib directory into your account, say, into a directory called /lib/perl/mason.

Then create a file, /public_html/cgi-bin/mason_handler.cgi, which contains:

#!/usr/bin/perl

use lib $ENV{"DOCUMENT_ROOT"} . "/../lib/perl/mason";
use HTML::Mason::CGIHandler;

my $h = HTML::Mason::CGIHandler->new(
  data_dir => '/tmp/mason_data',
  allow_globals => [qw(%session $u)],
);

$h->handle_request;

Now you want to configure your Apache to use this handler for Perl Mason webpages in the /public_html/mason directory (Apache v2.2):

<Directory /public_html/mason>
  <FilesMatch "\.html$">
    Action html-mason /cgi-bin/mason_handler.cgi
    SetHandler html-mason

    # for Apache 2.2
    Order allow,deny
    Allow from all

    # for Apache 2.4 (see https://httpd.apache.org/docs/2.4/upgrading.html)
    #Require all granted
  </FilesMatch>

  <FilesMatch "^(autohandler|dhandler)$">
    Action html-mason /cgi-bin/mason_handler.cgi
    SetHandler html-mason

    # for Apache 2.2
    Order allow,deny
    Allow from all

    # for Apache 2.4 (see https://httpd.apache.org/docs/2.4/upgrading.html)
    #Require all granted
  </FilesMatch>
</Directory>

Some CGI website providers require additional Perl modules for HTML::Mason to work, these can all be downloaded and extracted from CPAN:

  • Exception/Class.pm
  • Devel/StackTrace.pm
  • Class/Container.pm
  • Class/Data/Inheritable.pm
  • Params/Validate.pm *
  • Params/ValidatePP.pm *

(the files marked with a * are those that can be downloaded from CPAN and use the command perl Makefile –pm to force native perl code generation).

Using PowerShell 2.0 With Selenium to Automate Internet Explorer, Firefox, and Chrome

PowerShell 2.0 on Windows XP/7 uses .Net 3.5 so the first thing to do is download the Selenium WebDriver.dll file from Selenium’s download page and extract the net35/ directory.

Internet Explorer

Next you want to obtain the Internet Explorer driver from this site. I recommend version 2.41 because “as of 15 April 2014, IE 6 is no longer supported”. This must reside in your current PATH so in your script you may want to modify your PATH to ensure the executable (IEDriverServer.exe) can be found there. If you’re wondering whether to get the 32-bit or the 64-bit version, start with the 32-bit even if you’ve got a 64-bit Windows.

At this point you’ll want to quickly instantiate Internet Explorer and navigate somewhere. Great. Let’s do it.

# Load the Selenium .Net library
Add-Type -Path "N:\selenium\WebDriver.dll" # or wherever your WebDriver.dll is

# Set the PATH to ensure IEDriverServer.exe can found
$env:PATH += ";N:\selenium"

# Instantiate Internet Explorer
$ie_object = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"

This outputs:

New-Object : Exception calling ".ctor" with "0" argument(s): "Request for the permission of type 'System.Net.SocketPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed."
At line:1 char:17
+ $ie = New-Object <<<<  "OpenQA.Selenium.IE.InternetExplorerDriver"
    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand

Wait, what’s this? I don’t know. I just don’t know. It will happen if the DLL is on a network drive and not marked as “trusted” (whatever that means). So copy the DLL onto a local hard drive and try again.

# Load the Selenium .Net library
Add-Type -Path "C:\selenium\WebDriver.dll" # put your DLL on a local hard drive!

# Set the PATH to ensure IEDriverServer.exe can found
$env:PATH += ";N:\selenium"

# Instantiate Internet Explorer
$ie_object = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"

Great! Now we have an Internet Explorer window appear. We can navigate to a new URL:

$ie_object.Navigate().GoToURL( "http://www.bbc.co.uk/languages" )

This worked! The call won’t return until the page download is complete.

Next let’s click on a link from the link text:

$link = $ie_object.FindElementByLinkText( "Spanish" )
$link.Click()

# display current URL
$ie_object.Url

FireFox

Let’s try it with FireFox now. We require the GeckoDriver from the Selenium downloads page. Note that there is no GeckoDriver support for Windows XP at all.

# Set the PATH to ensure geckodriver.exe can found
$env:PATH += ";N:\selenium"

$ff_object = New-Object "OpenQA.Selenium.Firefox.FirefoxDriver"

Chrome

Finally let’s try with Google Chrome. We require the ChromeDriver from the Selenium downloads page.

# Set the PATH to ensure chromedriver.exe can found
$env:PATH += ";N:\selenium"

$chrome_object = New-Object "OpenQA.Selenium.Chrome.ChromeDriver"

How to Redirect Stderr From Executable in PowerShell 2.0 Without Line Breaks at Console Width

So you think PowerShell is flexible and useful and you go to redirect output from an external command to a file. You’ll parse this file later. But when you do you are horrified to find line breaks inserted in the middle of lines in the file… at the width of the console in which your PowerShell session was running.

The problem is that PowerShell is so smart that it unhelpfully inserts line breaks in all output at the current console width.

There are a few suggested answers to this including adding a -Width parameter to Out-File, but this doesn’t solve the issue for stderr redirects.

The solution was found in this post. Simply put, run your external command through cmd.exe and redirect from there!

To ensure the redirect operator is passed to cmd.exe escape the angle bracket with a backtick.

cmd.exe /C wget.exe -O - http://www.google.com/ 2`>stderr.log

Using wget to Automate Logging Into Websites

The open-source wget tool is useful for automating website access/scraping. In particular because it can store/retrieve cookies from a file.

# create a name for the cookie jar/file
COOKIE_JAR=/tmp/cookies.$$.txt

# save cookies from homepage access
wget --spider --save-cookies $COOKIE_JAR --keep-session-cookies http://www.smrt.com.sg/

# now submit request using saved cookies
wget -O - \
  --load-cookies $COOKIE_JAR \
  --save-cookies $COOKIE_JAR \
  --keep-session-cookies \
  --header "Referer: http://journey.smrt.com.sg/" \
  --post-data='startlat=1.357348601&startlng=103.9884093&endlat=1.276243657&endlng=103.8545958&routeopt=fastest&start_type=mrt&end_type=mrt&mode=TRANSIT&use_lrt=yes' \
  https://connect.smrt.wwprojects.com/smrt/api/journey/

Note that –spider performs a HEAD request and does not download the response. Options useful for debugging and seeing what is sent/received are -d and -S. For cookies the –keep-session-cookies option is essential to save session cookies (with no expiry time set) to the cookie file.

Serialising Arrays and Hashes in PowerShell 2.0

So you want to Invoke-Command a scriptblock on another Windows computer but are struggling to communicate results back to the caller because of a lack of serialisation routines in PowerShell 2.0? Yes, PowerShell 3.0 did introduce the ConvertFrom-Json and ConvertTo-Json cmdlets. But if you’re stuck on PowerShell 2.0 then you need another way to send hashes and lists.

Non-Recursive

Why not convert your hash-and-array data structure into a string – one that can be parsed by Invoke-Expression? This is a function that will do exactly that – and it is non-recursive for reasons that will be explained further down (and there’s a simpler recursive function provided later, too):

Function Serialise-Object {
    Param( $Root )

    Function AddAfter-ListNode {
        Param( $LinkedList, $AfterNode, $NewNode )
        if ( $AfterNode -eq $null ) {
            $LinkedList.AddLast( $NewNode )
        } else {
            $LinkedList.AddAfter( $AfterNode, $NewNode )
        }
    }

    Function Escape-SingleQuoted {
        Param( $Source )
        $Source -replace "'", "''"
    }

    # create lists
    $TodoStack = New-Object "System.Collections.Generic.Stack[Object]"
    $StringsList = New-Object "System.Collections.Generic.LinkedList[String]"

    # set up first element
    $TodoStack.Push( @( $Root, $StringsList.Last ) )

    while ( $true ) {
        try {
            $NextTodo = $TodoStack.Pop()
        } catch {
            break
        }

        ( $Item, $Node ) = @( $NextTodo )
        if ( $Item -eq $null ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "`$null"
            AddAfter-ListNode $StringsList $Node $NewStringNode
        } elseif ( $Item.getType().FullName -eq "System.Collections.Hashtable" ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "@{"
            AddAfter-ListNode $StringsList $Node $NewStringNode
            $LastStringNode = $NewStringNode

            $First = $true
            $Item.Keys |ForEach-Object {
                $keyname = ""
                if ( $First ) {
                    $First = $false
                } else {
                    $keyname += ";"
                }
                $keyname += $( "'" + (Escape-SingleQuoted $_) + "'=" )

                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $keyname
                AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
                $LastStringNode = $NewStringNode

                $TodoStack.Push( @( $Item[$_], $LastStringNode ) )
            }

            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "}"
            AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
        } elseif ( $Item.getType().FullName -eq "System.Object[]" ) {
            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" "@("
            AddAfter-ListNode $StringsList $Node $NewStringNode
            $LastStringNode = $NewStringNode

            $First = $true
            $Item |ForEach-Object {
                if ( $First ) {
                    $First = $false
                } else {
                    $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" ","
                    AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
                    $LastStringNode = $NewStringNode
                }

                $TodoStack.Push( @( $_, $LastStringNode ) )
            }

            $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" ")"
            AddAfter-ListNode $StringsList $LastStringNode $NewStringNode
        } else {
            if ( $Item.GetType().FullName -eq "System.String" ) {
                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $( "'" + (Escape-SingleQuoted $Item) + "'" )
                AddAfter-ListNode $StringsList $Node $NewStringNode
            } else {
                $NewStringNode = New-Object "System.Collections.Generic.LinkedListNode[String]" $( "[" + $Item.GetType().FullName + "]'" + (Escape-SingleQuoted $Item.ToString()) + "'" )
                AddAfter-ListNode $StringsList $Node $NewStringNode
            }
        }
    }

    @(
        $StringsList.GetEnumerator() |ForEach-Object { $_ }
    ) -join ""
}

Some examples of output:

> Serialise-Object $null
$null

> Serialise-Object 14.25
[System.Double]'14.25'

> Serialise-Object "Four o'clock"
'Four o''clock'

> Serialise-Object @( "First", "Second", @( "Inner1", "Inner2" ) )
@('First','Second',@('Inner1','Inner2'))

> Serialise-Object @{ "ArrayA" = @( 1, 2.5 ); "ArrayB" = @( 'e', 'f', 'g' ) }
@{'ArrayA'=@([System.Int32]'1',[System.Double]'2.5');'ArrayB'=@('e','f','g')}

> Serialise-Object @{ "OuterH" = @{ "InnerH" = @{ "key1" = [long]0xff } } }
@{'OuterH'=@{'InnerH'=@{'key1'=[System.Int64]'255'}}}

The resulting string can be fed directly into Invoke-Expression and the result is going to be very similar if not identical to the serialised object.

So, how does it work? It iterates over the object it is given. If it is a simple scalar type ($null, string, or non-list/hash) then it is converted to a string, prepended with its type if not a string, and output. If the object is an array or hash then each element or element-pair is iterated through and that object is recursively processed.

This function would have looked a lot simpler as a recursive function. So why was it implemented using lists and stacks instead of recursion? Because I wanted to send this function as a string through an Invoke-Command cmdlet and have it rebuilt as a scriptblock on the remote side; but one problem – how does one call an anonymous scriptblock recursively? Perhaps there’s a way but I don’t know how.

For example:

$remote_scriptblock = {
    Param( [String]$FnSerialiseStr )

    $FnSerialise = [scriptblock]::create( $FnSerialiseStr )

    $Start = Get-Date
    Start-Sleep -Milliseconds 1500

    & $FnSerialise @{ "time"=((Get-Date) - $Start).TotalSeconds }
}

Invoke-Command $remote_scriptblock -ArgumentList @(${function:Serialise-Object})

This outputs:

@{'time'=[System.Double]'1.5'}

Pretty neat, huh? You can send this function to the other side and run it!

Recursive

Ah.. but what if you want to send several named functions to the other side?

$remote_scriptblock = {
    Param( [String]$PreBlockStr )

    $PreBlock = [scriptblock]::create( $PreBlockStr )
    & $PreBlock

    $Start = Get-Date
    Start-Sleep -Milliseconds 2500

    Serialise-Object @{ "time"=((Get-Date) - $Start).TotalSeconds }
}
Invoke-Command $remote_scriptblock -ArgumentList @("Function Serialise-Object { ${function:Serialise-Object} }")

This outputs:

@{'time'=[System.Double]'2.5'}

Well that solves the problem of a recursive function trying to call itself.

Let’s rewrite the serialisation function as the simpler recursive form:

Function Serialise-Object {
    Param( $Root )

    Function Escape-SingleQuoted {
        Param( $Source )
        $Source -replace "'", "''"
    }

    if ( $Root -eq $null ) {
        "`$null"
    } elseif ( $Root.getType().FullName -eq "System.Collections.Hashtable" ) {
        $out = "@{"

        $First = $true
        $Root.Keys |ForEach-Object {
            if ( $First ) {
                $First = $false
            } else {
                $out += ";"
            }

            $out += $( "'" + (Escape-SingleQuoted $_) + "'=" )
            $out += Serialise-Object $Root[$_]
        }
        $out + "}"
    } elseif ( $Root.getType().FullName -eq "System.Object[]" ) {
        $out = "@("

        $First = $true
        $Root |ForEach-Object {
            if ( $First ) {
                $First = $false
            } else {
                $out += ","
            }

            $out += Serialise-Object $_
        }
        $out + ")"
    } else {
        if ( $Root.GetType().FullName -eq "System.String" ) {
            $( "'" + (Escape-SingleQuoted $Root) + "'" )
        } else {
            $( "[" + $Root.GetType().FullName + "]'" + (Escape-SingleQuoted $Root.ToString()) + "'" )
        }
    }
}

With this simpler code we can use it remotely as follows:

$remote_scriptblock = {
    Param( [String]$PreBlockStr )

    $PreBlock = [scriptblock]::create( $PreBlockStr )
    & $PreBlock

    $Start = Get-Date
    Start-Sleep -Milliseconds 3500

    Serialise-Object @{ "time"=((Get-Date) - $Start).TotalSeconds }
}
Invoke-Command $remote_scriptblock -ArgumentList @("Function Serialise-Object { ${function:Serialise-Object} }")

This outputs:

@{'time'=[System.Double]'3.5'}

… using simpler code.