Perl WWW::Mechanize or LWP::UserAgent and AWS SSL Handshake Failure

TL;DR (too long, didn’t read) summary:

Force protocol and cipher:

my $ua = WWW::Mechanize->new();
$ua->ssl_opts(
  'SSL_version' => 'TLSv1_2',
  'SSL_cipher_list' => 'ECDHE-RSA-AES256-GCM-SHA384',
);

---

I had a script using WWW::Mechanize (which uses LWP::UserAgent under the hood) which used to work with a website. But then the website was moved and hosted on Amazon Web Services.

Suddenly my script started reporting:

SSL connect attempt failed error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure at /usr/share/perl5/LWP/Protocol/http.pm line 47.

I tried debugging using a technique I found elsewhere and that was providing a command line switch to enable SSL debugging:

$ perl -MIO::Socket::SSL=debug4 myscript.pl
DEBUG: .../IO/Socket/SSL.pm:2700: new ctx 53819024
DEBUG: .../IO/Socket/SSL.pm:612: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:614: socket connected
DEBUG: .../IO/Socket/SSL.pm:636: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:669: using SNI with hostname www.thetarget.domain
DEBUG: .../IO/Socket/SSL.pm:704: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:723: set socket to non-blocking to enforce timeout=15
DEBUG: .../IO/Socket/SSL.pm:736: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:739: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:749: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:759: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:779: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:736: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:739: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:742: SSL connect attempt failed

DEBUG: .../IO/Socket/SSL.pm:742: local error: SSL connect attempt failed error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
DEBUG: .../IO/Socket/SSL.pm:745: fatal SSL error: SSL connect attempt failed error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
DEBUG: ...erl5/Net/HTTPS.pm:69: ignoring less severe local error 'IO::Socket::IP configuration failed', keep 'SSL connect attempt failed error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'
DEBUG: .../IO/Socket/SSL.pm:2733: free ctx 53819024 open=53819024
DEBUG: .../IO/Socket/SSL.pm:2738: free ctx 53819024 callback
DEBUG: .../IO/Socket/SSL.pm:2745: OK free ctx 53819024

I tried capturing the session with tcpdump and viewing the session in Wireshark to make sense of what was happening:

time=0.291136 client>server protocol=TLSv1.2 message=Client Hello
time=0.581841 server>client protocol=TLSv1.2 message=Alert (Level: Fatal, Description: Handshake Failure)

So basically AWS was outright rejecting the “Client Hello” packet without any negotiation at all. Neither was it citing a reason for the handshake failure.

At this point it became guesswork. Initially I considered forcing the protocol using the ssl_opts() function:

my $ua = WWW::Mechanize->new();
$ua->ssl_opts(
  'SSL_version' => 'TLSv1_2',
);

But this gave exactly the same result.

Then I thought about forcing the cipher. But to which cipher? I looked up Amazon’s list of accepted ciphers but that didn’t help much. You can try running the following but it gives too many options:

$ openssl ciphers
ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:SRP-DSS-AES-256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:...

So then I tried doing a connect using openssl to see what cipher it chose:

$ # note you can also add the -servername <domain> option for SNI
$ openssl s_client -connect www.thetarget.domain:443
...
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
...

So I then tried my Perl script setting the cipher to that reported by openssl:

my $ua = WWW::Mechanize->new();
$ua->ssl_opts(
  'SSL_version' => 'TLSv1_2',
  'SSL_cipher_list' => 'ECDHE-RSA-AES256-GCM-SHA384',
);

This time my script worked!

$ perl -MIO::Socket::SSL=debug4 myscript.pl
DEBUG: .../IO/Socket/SSL.pm:2700: new ctx 38327392
DEBUG: .../IO/Socket/SSL.pm:612: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:614: socket connected
DEBUG: .../IO/Socket/SSL.pm:636: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:669: using SNI with hostname www.thetarget.domain
DEBUG: .../IO/Socket/SSL.pm:704: request OCSP stapling
DEBUG: .../IO/Socket/SSL.pm:723: set socket to non-blocking to enforce timeout=15
DEBUG: .../IO/Socket/SSL.pm:736: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:739: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:749: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:759: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:779: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:736: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:2601: did not get stapled OCSP response
DEBUG: .../IO/Socket/SSL.pm:2554: ok=1 [2] /O=Digital Signature Trust Co./CN=DST Root CA X3/O=Digital Signature Trust Co./CN=DST Root CA X3
DEBUG: .../IO/Socket/SSL.pm:2554: ok=1 [1] /O=Digital Signature Trust Co./CN=DST Root CA X3/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
DEBUG: .../IO/Socket/SSL.pm:2554: ok=1 [0] /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3/CN=www.thetarget.domain
DEBUG: .../IO/Socket/SSL.pm:1664: scheme=www cert=40017248
DEBUG: .../IO/Socket/SSL.pm:1674: identity=www.thetarget.domain cn=www.thetarget.domain alt=2 www.thetarget.domain
DEBUG: .../IO/Socket/SSL.pm:739: done Net::SSLeay::connect -> -1
DEBUG: .../IO/Socket/SSL.pm:749: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:759: waiting for fd to become ready: SSL wants a read first
DEBUG: .../IO/Socket/SSL.pm:779: socket ready, retrying connect
DEBUG: .../IO/Socket/SSL.pm:736: call Net::SSLeay::connect
DEBUG: .../IO/Socket/SSL.pm:739: done Net::SSLeay::connect -> 1
DEBUG: .../IO/Socket/SSL.pm:794: ssl handshake done