newspaint

Documenting Problems That Were Difficult To Find The Answer To

Category Archives: Technology

Exim: Helo Command Rejected Need Fully-Qualified Hostname

I’ve been running my Exim configuration for years when received a mail delivery failure message with the error:

504 5.5.2 <myhost>: Helo command rejected: need fully-qualified hostname

I’d not come across this issue before and my Exim set-up had been happily introducing itself to other mail servers as myhost which was the unqualified hostname of the virtual machine Exim was running on.

So – what to do?

The helo_data command, when added to a transport configuration, could be set to whatever string you want sent by Exim with the HELO message during the SMTP handshake phase of a connection.

But rather than just simply give a fixed string – I wanted my configuration to be a little more clever – and determine the MX host from the sent e-mail’s from domain.

To do this the ${lookup dnsdb} function was used.

The configuration setting I use is as follows, which I’ll break down and explain afterwards:

helo_data = ${extract{2}{,}{${extract{1}{:}{${sort {${lookup dnsdb{>:,,mx=$sender_address_domain}}} {<} {${extract{1}{,}{$item}}}}}}}{$value}{unknown}}

If I expand this out and add comments it may aid in understanding, I’ll start from the centre and work my way out.

First the sender’s domain lookup:

# Do a DNS lookup of the MX (mail exchange) records of the sender's domain
#
# EXAMPLE OUTPUT (for "gmail.com"):
#   40,alt3.aspmx.l.google.com:20,alt1.aspmx.l.google.com:30,alt2.aspmx.l.google.com:50,alt4.aspmx.l.google.com:10,aspmx.l.google.com

${lookup dnsdb{>:,,mx=$sender_address_domain}}

# NOTES
#   records are not given in any particular order, and are colon-separated

Obviously having a list of MX hosts is too much, we just want one, and we want the highest priority host (with the smallest number). So we have to sort the list using the ${sort} function:

# Sort the list of MX records in order of priority from smallest to largest
#
# EXAMPLE OUTPUT (for "gmail.com"):
#   10,aspmx.l.google.com:20,alt1.aspmx.l.google.com:30,alt2.aspmx.l.google.com:40,alt3.aspmx.l.google.com:50,alt4.aspmx.l.google.com

${sort
  { ${lookup dnsdb {>:,,mx=$domain} } }  # text to sort
  { < }                                  # comparison operator
  { ${extract {1} {,} {$item} } }        # fn to run on each item
}

# NOTES
#   each item to be sorted must have the number extracted

Okay now we have a sorted list. We just want the first entry in the list and to do this we use the ${extract} function.

# Sort the list of MX records in order of priority from smallest to largest
# and select the first record in the list
#
# EXAMPLE OUTPUT (for "gmail.com"):
#   10,aspmx.l.google.com

${extract
  {1}                                       # select first field
  {:}                                       # separator is colon
  {                                         # text to extract from
    ${sort
      { ${lookup dnsdb {>:,,mx=$domain} } }
      {<}
      { ${extract {1} {,} {$item} } }
    }
  }
}

Finally we just want the hostname which is the second field in a comma-separated list of that string.

We can provide a default value in the event nothing is extracted. Which leads us to our final formula:

# Sort the list of MX records in order of priority from smallest to largest
# and select the first record in the list, then select the second column
# which is the hostname, otherwise return "unknown"
#
# EXAMPLE OUTPUT (for "gmail.com"):
#   aspmx.l.google.com

${extract
  {2}                                           # select 2nd field
  {,}                                           # separator is comma
  {                                             # text to extract from
    ${extract
      {1}
      {:}
      {
        ${sort
          { ${lookup dnsdb {>:,,mx=$domain} } }
          {<}
          { ${extract {1} {,} {$item} } }
        }
      }
    }
  }
  {$value}                                      # result on success
  {unknown}                                     # result on failure
}

Selecting a column in tab-delimited text in Splunk

So you have a raw field that looks something like this:

field1	field2	field3	field4	field5

… where the gaps between fields are tab characters (“\t” or ASCII character 9).

You might think the way to select the 3rd field would be as follows:

index=myindex mysearchterm
  |eval fields=split(_raw, "\t" )
  |eval desiredfield=mvindex(fields,3)

But no. The Splunk split command does not recognise “\t” as the tab character.

A work around is to replace all the tabs with a unique string and split on this instead. So the above could be re-written as:

index=myindex mysearchterm
  |eval myraw=_raw
  |rex mode=sed field=myraw "s/\t/MYUNIQUESEPARATOR/g"
  |eval fields=split(myraw, "MYUNIQUESEPARATOR" )
  |eval desiredfield=mvindex(fields,3)

Xubuntu 16.04 and XFCE Not Allowing Monitors to Sleep or Power Off

The answer is that Google Chrome disables (“inhibits”) sleeping when doing WebRTC communications. Unbelievable, right? Anyway. Here’s how to discover the cause.

First kill any instance of xfce4-power-manager running.

Then in a command window run:

me@myhost:~$ xfce4-power-manager --debug --no-daemon

...
TRACE[xfpm-dpms.c:75] xfpm_dpms_set_timeouts(): Settings dpms: standby=1200 suspend=0 off=3600
...

Now, try opening a Chrome browser, say, version 60.0.3112.90 (Official Build) (64-bit). Next open up a WebRTC webpage such as https://webrtc.github.io/samples/src/content/datachannel/basic/ and click Start on the demo page.

You will probably see the following appear in your command window:

TRACE[xfpm-inhibit.c:343] xfpm_inhibit_inhibit(): Inhibit send application name=/usr/bin/google-chrome-stable reason=WebRTC has active PeerConnections sender=:1.2387
TRACE[xfpm-inhibit.c:101] xfpm_inhibit_has_inhibit_changed(): Inhibit added

You can confirm that Chrome is using WebRTC for this page by visiting chrome://webrtc-internals/ and seeing the webpage(s) using WebRTC.

And if you close the offending tab you will see the following in your command window:

TRACE[xfpm-inhibit.c:357] xfpm_inhibit_un_inhibit(): UnHibit message received
TRACE[xfpm-inhibit.c:95] xfpm_inhibit_has_inhibit_changed(): Inhibit removed

At this point you have two choices:

  • stop using Chrome altogether
  • stop visiting that particular page that uses WebRTC in Chrome

The issue was discussed on this forum page and at this page.

BASH /usr/bin/adb No Such File or Directory

Using Xubuntu (Ubuntu) 16.04.3 I encountered a strange problem. I’d removed the Android debugging tools using the command:

my@myhost:~$ sudo apt-get remove android-tools-adb

I did this because I installed the latest platform tools manually from Google directly.

So next time I went to run adb I didn’t expect to see the following:

my@myhost:~$ adb
bash: /usr/bin/adb: No such file or directory

The answer was found in this forum post. I did not realise this but BASH actually caches executables found in the $PATH to avoid having to perform future searches.

The type command can reveal whether an executable has been stored in the cache (or “hashed”):

my@myhost:~$ type adb
adb is hashed (/usr/bin/adb)

To clear the entire cache the following command can be run:

my@myhost:~$ hash -r

.. but it might be preferable to only clear that command that has been removed from the cache:

my@myhost:~$ hash -d adb

How to Migrate WhatsApp From One Rooted Android Phone to Another Rooted Android Phone

If you are using CyanogenMod or LineageOS or similar and want to copy WhatsApp from one phone to the other here are the steps that may do what you want.

Firstly decide where you’re going to back your WhatsApp data to, in my case I wanted to write to the /storage/sdcard1/ folder on my old phone in which was inserted a MicroSD card.

Start off by backing up the app, data, and media files from the existing (old) phone:

me@server:~$ adb root
me@server:~$ adb shell
root@oldphone:/ # tar -cvpJf /storage/sdcard1/data-whatsapp.tar.xz /data/data/com.whatsapp
root@oldphone:/ # tar -cvpJf /storage/sdcard1/app-whatsapp.tar.xz /data/app/com.whatsapp-3
root@oldphone:/ # tar -cvpJf /storage/sdcard1/media-whatsapp.tar.xz /data/media/0/WhatsApp
root@oldphone:/ # exit

me@server:~$ adb pull /storage/sdcard1/data-whatsapp.tar.xz
me@server:~$ adb pull /storage/sdcard1/app-whatsapp.tar.xz
me@server:~$ adb pull /storage/sdcard1/media-whatsapp.tar.xz

An alternate method to avoid storing on the device’s flash is the following:

me@server:~$ adb root
me@server:~$ adb exec-out "tar -cvpJf - /data/data/com.whatsapp 2>/dev/null" >/tmp/data-whatsapp.tar.xz
me@server:~$ adb exec-out "tar -cvpJf - /data/app/com.whatsapp-3 2>/dev/null" >/tmp/app-whatsapp.tar.xz
me@server:~$ adb exec-out "tar -cvpJf - /data/media/0/WhatsApp 2>/dev/null" >/tmp/media-whatsapp.tar.xz

Now unplug the existing (old) phone. Plug in the new phone. Choose where you want to upload the backups to for extraction later (I chose /storage/0000-0000 as that was my MicroSD card on the new phone).

me@server:~$ adb root
me@server:~$ adb push data-whatsapp.tar.xz /storage/0000-0000/
me@server:~$ adb push app-whatsapp.tar.xz /storage/0000-0000/
me@server:~$ adb push media-whatsapp.tar.xz /storage/0000-0000/

me@server:~$ adb shell
root@newphone:/ # cd /
root@newphone:/ # tar -xvpJf /storage/0000-0000/data-whatsapp.tar.xz
root@newphone:/ # tar -xvpJf /storage/0000-0000/app-whatsapp.tar.xz
root@newphone:/ # tar -xvpJf /storage/0000-0000/media-whatsapp.tar.xz

At this point everything is where it needs to be. However there’s a big problem. Every app has a unique userid/groupid (uid/gid) – but uid=gid for each application. The application we just copied won’t be able to access its own data and user files because the files it needs will have a different uid/gid on the other phone.

First, soft restart the phone so the newly copied application can be found by Android.

If you fail to change the uid/gid of the data files for the application you will see messages like the following in logcat:

me@server:~$ adb root
me@server:~$ adb shell
root@newphone:/ # logcat |grep -i whatsapp
03-14 15:38:26.368 22689 22689 W pool-6-thread-1: type=1400 audit(0.0:1111): avc: denied { getattr } for path="/data/data/com.whatsapp/databases/_jobqueue-WhatsAppJobManager" dev="dm-0" ino=134223 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:system_data_file:s0 tclass=file permissive=0

03-14 15:38:26.371 22689 22689 W pool-6-thread-1: type=1400 audit(0.0:1112): avc: denied { read } for name="_jobqueue-WhatsAppJobManager" dev="dm-0" ino=134223 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:system_data_file:s0 tclass=file permissive=0

This article illustrates how to find the uid/gid of an app.

me@server:~$ adb root
me@server:~$ adb shell
root@newphone:/ # cat /data/system/packages.xml |grep '<package' |grep whatsapp
    <package name="com.whatsapp" ... userId="10053" ...

Armed with knowing what the new uid/gid should be we can go change the data from the existing uid/gid.

me@server:~$ adb root
me@server:~$ adb shell
root@newphone:/ # ls -alnd /data/data/com.whatsapp/
drwxr-x--x 10 10065 10065 4096 2018-03-14 16:11 /data/data/com.whatsapp/

root@newphone:/ # find /data/data/com.whatsapp -user 10065 -exec chown -h 10053 {} \;
root@newphone:/ # find /data/data/com.whatsapp -group 10065 -exec chgrp -h 10053 {} \;

If you’re using LineageOS 14.1 (“Nougat”) then you will also have to explicitly give your app permission to read storage (for the media that has been stored). Otherwise you might get messages like the following in logcat if you try and open a picture:

03-18 15:10:09.137 xxxxx xxxxx E BitmapFactory: Unable to decode stream: java.io.FileNotFoundException: /storage/emulated/0/WhatsApp/Media/WhatsApp Images/Sent/IMG-20180122-WA0000.jpg (Permission denied)

To do this on your phone:

  • open Settings
  • find and open Apps
  • navigate to your app (e.g. “WhatsApp”) and open
  • open Permissions
  • enable Storage permission
  • (you may want to enable other permissions, too, depending on what you want your app to be able to do)

This should largely do it. Maybe you’ll need to soft reboot your phone. If WhatsApp works after this then you may want to consider going to Google Play and attempting to install WhatsApp – it should recognise that the app is already installed and offer to upgrade – so upgrade.

Certbot Not Renewing Certificate for Apache in Ubuntu 16.04

Recently while upgrading Ubuntu it appeared to switch to Python 3. Something must have broken because my Let’s Encrypt (letsencrypt) certificate did not automatically renew as per the configured /etc/cron.d/certbot cron job.

I inspected the /var/log/letsencrypt/letsencrypt.log and found the following:

2018-03-13 02:48:02,521:WARNING:certbot.renewal:Attempting to renew cert (www.mydomain) from /etc/letsencrypt/renewal/www.mydomain.conf produced an unexpected error: The requested apache plugin does not appear to be installed. Skipping.
2018-03-13 02:48:02,523:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 425, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1058, in renew_cert
    installer, auth = plug_sel.choose_configurator_plugins(config, plugins, "certonly")
  File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 201, in choose_configurator_plugins
    diagnose_configurator_problem("authenticator", req_auth, plugins)
  File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 297, in diagnose_configurator_problem
    raise errors.PluginSelectionError(msg)
certbot.errors.PluginSelectionError: The requested apache plugin does not appear to be installed

To fix I ran:

~ $ sudo apt-get install python3-certbot-apache

Wireshark Showing Wrong Timezone/Times for Packets

If you find yourself wondering why Wireshark seems to be displaying the wrong time for packets – it could be something as simple as having configured the display timezone incorrectly.

If the timestamps appear to be for the wrong timezone...

If the timestamps appear to be for the wrong timezone…

 

Right click the column header “Time” and choose Column Preferences…

Right click on the Time column header and choose Column Preferences...

Right click on the Time column header and choose Column Preferences…

 

Next double click (using left mouse button) on the “Type” field to bring up a drop down list of options.

Double click on the Type field for the Time row

Double click on the Type field for the Time row

 

From the drop down list choose “Time (format as specified)”. If this was set to something different (e.g. “UTC time”) then this could explain why you saw a different date/time in the packet capture display.

Choose "Time (format as specified)" from the drop down list

Choose “Time (format as specified)” from the drop down list

Introduction to Packet Capture on Microsoft Windows for Later Analysis with Wireshark

Windows versions 7 and above come with the netsh trace command which can be used to take packet captures – however these captures are incompatible with Wireshark. Such captures can be converted into a basic format that can be read with Wireshark using the Microsoft Message Analyzer that Microsoft makes freely available (a 68MB download).

The process is typically:

  • netsh trace show – get information useful to use when specifying the trace
  • netsh trace start – begin a trace specifying filters to use
  • netsh trace stop – end the trace and create the capture file

You can get information for the netsh trace show command:

C:\> netsh trace show /?

The following commands are available:

Commands in this context:
show CaptureFilterHelp - List supported capture filters and usage.
show globalKeywordsAndLevels - List global keywords and levels.
show helperclass - Show helper class information.
show interfaces - List available interfaces.
show provider  - Shows provider information.
show providers - Shows available providers.
show scenario  - Shows scenario information.
show scenarios - Shows available scenarios.
show status    - Shows tracing configuration.

The most useful of these is netsh trace show CaptureFilterHelp as follows:

  Capture Filters:
        Capture filters are only supported when capture is explicitly
        enabled with capture=yes. Supported capture filters are:

        CaptureInterface=<interface name or GUID>
         Enables packet capture for the specified interface name or GUID. Use
         'netsh trace show interfaces' to list available interfaces.
        e.g. CaptureInterface={716A7812-4AEE-4545-9D00-C10EFD223551}
        e.g. CaptureInterface=!{716A7812-4AEE-4545-9D00-C10EFD223551}
        e.g. CaptureInterface="Local Area Connection"

        Ethernet.Address=<MAC address>
         Matches the specified filter against both source and destination
         MAC addresses.
        e.g. Ethernet.Address=00-0D-56-1F-73-64

        Ethernet.SourceAddress=<MAC address>
         Matches the specified filter against source MAC addresses.
        e.g. Ethernet.SourceAddress=00-0D-56-1F-73-64

        Ethernet.DestinationAddress=<MAC address>
         Matches the specified filter against destination MAC addresses.
        e.g. Ethernet.DestinationAddress=00-0D-56-1F-73-64

        Ethernet.Type=<ethertype>
         Matches the specified filter against the MAC ethertype.
        e.g. Ethernet.Type=IPv4
        e.g. Ethernet.Type=NOT(0x86DD)
        e.g. Ethernet.Type=(IPv4,IPv6)

        Wifi.Type=<Management|Data>
         Matches the specified filter against the Wifi type. Allowed values
         are 'Management' and 'Data'. If not specified, the Wifi.Type filter
         is not applied.
         Note: This capture filter does not support ranges, lists or negation.
        e.g. Wifi.Type=Management

        Protocol=<protocol>
         Matches the specified filter against the IP protocol.
        e.g. Protocol=6
        e.g. Protocol=!(TCP,UDP)
        e.g. Protocol=(4-10)

        IPv4.Address=<IPv4 address>
         Matches the specified filter against both source and destination
         IPv4 addresses.
        e.g. IPv4.Address=157.59.136.1
        e.g. IPv4.Address=!(157.59.136.1)
        e.g. IPv4.Address=(157.59.136.1,157.59.136.11)

        IPv4.SourceAddress=<IPv4 address>
         Matches the specified filter against source IPv4 addresses.
        e.g. IPv4.SourceAddress=157.59.136.1

        IPv4.DestinationAddress=<IPv4 address>
         Matches the specified filter against destination IPv4 addresses.
        e.g. IPv4.DestinationAddress=157.59.136.1

        IPv6.Address=<IPv6 address>
         Matches the specified filter against both source and destination
         IPv6 addresses.
        e.g. IPv6.Address=fe80::5038:3c4:35de:f4c3\%8
        e.g. IPv6.Address=!(fe80::5038:3c4:35de:f4c3\%8)

        IPv6.SourceAddress=<IPv6 address>
         Matches the specified filter against source IPv6 addresses.
        e.g. IPv6.SourceAddress=fe80::5038:3c4:35de:f4c3\%8

        IPv6.DestinationAddress=<IPv6 address>
         Matches the specified filter against destination IPv6 addresses.
        e.g. IPv6.DestinationAddress=fe80::5038:3c4:35de:f4c3\%8

        CustomMac=<type(offset,value)>
         Matches the specified filter against the value at the specified
         offset starting with the MAC header.
         Note: This capture filter does not support ranges, lists or negation.
        e.g. CustomMac=UINT8(0x1,0x23)
        e.g. CustomMac=ASCIISTRING(3,test)
        e.g. CustomMac=UNICODESTRING(2,test)

        CustomIp=<type(offset,value)>
         Matches the specified filter against the value at the specified
         offset starting with the IP header.
         Note: This capture filter does not support ranges, lists or negation.
        e.g. CustomIp=UINT16(4,0x3201)
        e.g. CustomIp=UINT32(0x2,18932)

        CaptureMultiLayer=<yes|no>
         Enables multi-layer packet capture.
         Note: This capture filter does not support ranges, lists or negation.

        PacketTruncateBytes=<value>
         Captures only the the specified number of bytes of each packet.
         Note: This capture filter does not support ranges, lists or negation.
        e.g. PacketTruncateBytes=40

Note:
        Multiple filters may be used together. However the same filter may
        not be repeated.
        e.g. 'netsh trace start capture=yes Ethernet.Type=IPv4
              IPv4.Address=157.59.136.1'

        Filters need to be explicitly stated when required. If a filter is
        not specified, it is treated as "don't-care".
         e.g. 'netsh trace start capture=yes IPv4.SourceAddress=157.59.136.1'
              This will capture IPv4 packets only from 157.59.136.1, and it
              will also capture packets with non-IPv4 Ethernet Types, since
              the Ethernet.Type filter is not explicitly specified.
         e.g. 'netsh trace start capture=yes IPv4.SourceAddress=157.59.136.1
               Ethernet.Type=IPv4'
              This will capture IPv4 packets only from 157.59.136.1. Packets
              with other Ethernet Types will be discarded since an explicit
              filter has been specified.

        Capture filters support ranges, lists and negation (unless stated
        otherwise).
         e.g. Range: 'netsh trace start capture=yes Ethernet.Type=IPv4
                      Protocol=(4-10)'
              This will capture IPv4 packets with protocols between 4 and 10
              inclusive.
         e.g. List: 'netsh trace start capture=yes Ethernet.Type=(IPv4,IPv6)'
              This will capture only IPv4 and IPv6 packets.
         e.g. Negation: 'netsh trace start capture=yes Ethernet.Type=!IPv4'
              This will capture all non-IPv4 packets.

        Negation may be combined with lists in some cases.
         e.g. 'netsh trace start capture=yes Ethernet.Type=!(IPv4,IPv6)'
               This will capture all non-IPv4 and non-IPv6 packets.

        'NOT' can be used instead of '!' to indicate negation. This requires
        parentheses to be present around the values to be negated.
         e.g. 'netsh trace start capture=yes Ethernet.Type=NOT(IPv4)'

You can get information about the netsh trace start command:

C:\> netsh trace start /?

start
  Starts tracing.

  Usage: trace start [[scenario=]]
        [[globalKeywords=]keywords] [[globalLevel=]level]
        [[capture=]yes|no] [[report=]yes|no]
        [[persistent=]yes|no] [[traceFile=]path\filename]
        [[maxSize=]filemaxsize] [[fileMode=]single|circular|append]
        [[overwrite=]yes|no] [[correlation=]yes|no|disabled] [capturefilters]
        [[provider=]providerIdOrName] [[keywords=]keywordMaskOrSet]
        [[level=]level] [[provider=]provider2IdOrName]
        [[keywords=]keyword2MaskOrSet] [[level=]level2] ...

Defaults:
        capture=no (specifies whether packet capture is enabled
                in addition to trace events)
        report=no (specifies whether a complementing report will be generated
                along with the trace file)
        persistent=no (specifies whether the tracing session continues
                across reboots, and is on until netsh trace stop is issued)
        maxSize=250 MB (specifies the maximum trace file size, 0=no maximum)
        fileMode=circular
        overwrite=yes (specifies whether an existing trace output file will
                be overwritten)
        correlation=yes (specifies whether related events will be correlated
                and grouped together)
        traceFile=%LOCALAPPDATA%\Temp\NetTraces\NetTrace.etl
                (specifies location of the output file)

Provider keywords default to all and level to 255 unless otherwise specified.

For example:

netsh trace start scenario=InternetClient capture=yes

        Starts tracing for the InternetClient scenario and dependent providers
                with packet capture enabled.
        Tracing will stop when the "netsh trace stop" command is issued
                or when the system reboots.
        Default location and name will be used for the output file. If an old
                file exists, it will be overwritten.

netsh trace start provider=microsoft-windows-wlan-autoconfig
        keywords=state,ut:authentication

        Starts tracing for the microsoft-windows-wlan-autoconfig provider
        Tracing will stop when the "netsh trace stop" command is issued
                or when the system reboots.
        Default location and name will be used for the output file. If an old
                file exists, it will be overwritten.
        Only events with keyword 'state' or 'ut:authentication' will be logged.

        netsh trace show provider command can be used to display
                supported keywords and levels.

Capture Filters:
        Capture filters are only supported when capture is explicitly
        enabled with capture=yes. Use 'netsh trace show CaptureFilterHelp'
        to display a list of supported capture filters and their usage.

All captures should, unless you have a more specific goal in mind, have the Capture=yes parameter at the beginning of the trace statement, and provider=Microsoft-Windows-NDIS-PacketCapture level=5 keywords=ut:ReceivePath,ut:SendPath provider=Microsoft-Windows-Networking-Correlation level=1 keywords=ut:Packet at the end of the trace statement (after the capture filters have been specified) to ensure just the packet data is captured and not other verbose unrelated messages.

Optionally choose an interface:

C:\> netsh trace show interface
Ethernet adapter Local Area Connection 4:
    Description:     Intel(R) PRO/1000 MT Desktop Adapter
    Interface GUID:  {7C101C48-7750-4876-9746-C55584C59818}
    Interface Index: 21
    Interface Luid:  0x6000011000000

Ethernet adapter VirtualBox Host-Only Network:
    Description:     VirtualBox Host-Only Ethernet Adapter
    Interface GUID:  {2557E211-2DEF-47E6-B6B6-B56CD20092D5}
    Interface Index: 17
    Interface Luid:  0x600000F000000

Some examples:

Command Description
netsh trace start Capture=yes Correlation=no Report=no Scenario=InternetClient Tracefile=c:\temp\capture.etl Ethernet.Type=IPv4 IPv4.Address=(10.0.0.1,10.1.0.1,10.2.0.1,10.3.0.1) Protocol=UDP PacketTruncateBytes=256 CaptureInterface={7C101C48-7750-4876-9746-C55584C59818} limit packet captures to first 256 bytes of each packet, UDP only, IPv4 only, on the network interface specified only, to the specified etl file, make no report, do not correlate, and to/from any of the 4 IP addresses specified
netsh trace start Capture=yes Correlation=no Report=no Tracefile=c:\temp\capture.etl Ethernet.Type=IPv4 IPv4.Address=(10.0.0.1,10.1.0.1,10.2.0.1,10.3.0.1) Protocol=UDP PacketTruncateBytes=256 CaptureInterface={7C101C48-7750-4876-9746-C55584C59818} provider=Microsoft-Windows-NDIS-PacketCapture level=5 keywords=ut:ReceivePath,ut:SendPath provider=Microsoft-Windows-Networking-Correlation level=1 keywords=ut:Packet largely prevent “Microsoft_Windows_Networking_Correlation” module noise in packet capture by limiting verbosity level to 1, specify that packets coming in and going out should be captured with debugging level (5) verbosity for the “Microsoft-Windows-NDIS-PacketCapture” provider, everything else as above (produces far smaller capture files, note that the provider specifications should come after the filters)

Converting to Wireshark .cap Format: you can either use the Microsoft Message Analyzer GUI to open the .etl file, wait until it has processed the entire file (takes tens of minutes), and then save the file as a .cap export.

Or you can do the following in Powershell:

import-module PEF
$s = New-PefTraceSession -Path "C:\temp\wireshark-dump.cap" -SaveOnStop
$s | Add-PefMessageProvider -Provider "C:\temp\capture.etl"
$s | Start-PefTraceSession

So what can’t netsh do?

  • capture a specific application protocol, such as DNS, SNMP, or the like, because it does not permit matching on ports, and even if you tried using CustomIp=UINT16(22,53) you have to hope that the IPv4 header is its minimum 20 bytes and no longer and you can only capture on source OR destination ports but not both which means you will not see request/response pairs which is so essential to any tracing session
  • export in a Wireshark-friendly format

The provided “netsh trace” command that comes with Microsoft products is conclusive proof that Windows is never a good solution in an enterprise environment. While Linux servers have the “tcpdump” tool that allows specific filtering on so many different parameters the Windows “netsh trace” command flails around unable to be of much use to anybody trying to debug a networking or application related issue.

Any time a “Solution Architect” recommends a Windows or other Microsoft related product you will know instantly that they have never had to maintain or troubleshoot a real enterprise system in the wild.

Splunk Regular Expression Flags

Very little is openly documented about Splunk regular expressions – which is surprising given how important pattern matching is when trawling through large amounts of data.

Most documentation about Splunk regular expressions is found at this linked page.

In addition the two most commonly used functions that utilise regular expressions during searches are:

  • regex – filters based on regular expression, i.e. matches or doesn’t match (if field!=regex syntax is used)
  • rex – allows extraction of named capturing groups from text into parameters, does not filter

Flags

The hint about supported flags comes from the documentation of the regex command which states:

The regular expression must be a Perl Compatible Regular Expression supported by the PCRE library.

Possible flags are described in PHP’s PCRE pattern modifiers page however not all apply to Splunk.

Here are the flags that appear supported in Splunk:

Modifier Behaviour Example Matches When Used
(?i) case insensitive (?i)moon Moon
(?m) ^ and $ can match beginning and end of line as well as string (?m)^The Who art thou?
The butler.
(?s) dot matches newlines (?s)P.G It is P
G.
(?x) extended, white space in expression is ignored (?x)ID \d+ ID4132
(?U) ungreedy, by default make expressions ungreedy, and greedy if question mark used (?U)(M.+t) Meet the president

Citrix Keyboard Shortcuts for Linux Receiver

From this page:

Combination Description
SHIFT+F1 Displays the Windows Task List
SHIFT+F2 Displays or Hides the Windows Title Bar for the Citrix Receiver Client window
SHIFT+F3 Disconnects the ICA session and exits the Citrix Receiver
CTRL+F1 Opens the Windows Security dialog box (similar to CTRL+ALT+DEL)
CTRL+F2 Displays the remote computer’s Start Menu (similar to CTRL+ESC)
CTRL+F3 Displays the Windows Task Manager (similar to CTRL+SHIFT+ESC)
ALT+F2 Cycles through running programs in the Citrix session (similar to ALT+ESC)
ALT+PLUS Switches between programs from left to right (similar to ALT+TAB)
ALT+MINUS Switches between programs from right to left (similar to ALT+SHIFT+TAB)