Wednesday, March 8, 2017

Never Give Up, Never Surrender: How to connect to modern SSL websites from EZproxy 5.x using stunnel

Prior to OCLC's acquisition of EZproxy, there were permanent licensing options available to institutions that many sites chose to exercise.  Unfortunately, due to the continued practice of EZproxy not embracing the platform and linking dynamically to the operating system's installation of OpenSSL, sites still running version 5.x of the software are now starting to run into SSL issues due to PCI-DSS mandated SSL changes that are being adopted by various hosting providers and CDN platforms.

In a nutshell, EZproxy is no longer tall enough to ride the modern encrypted Internet.  

Or for you meme lovers, EZproxy no can haz interwebz.

Fear not, there is a solution that will enable sites with permanent licenses to continue using the software for a while longer:

We need to give EZproxy some platform shoes.




The root of the problem is that PCI-DSS compliance now requires SSL web servers to use increasingly sophisticated encryption settings, some of which were simply not available in the version of OpenSSL that EZproxy 5.x uses.  The work-around is to add a helper tool that will still talk the older version of SSL to EZproxy while talking the newer versions of SSL to remote web servers.

Now this is not quite as easy as configuring a proxy server and using ProxySSL to solve the issue.  Why?  Because while a SSL proxy uses the HTTP CONNECT verb to establish an on-demand tunnel to the remote endpoint, it does not actively participate in the conversation beyond the initial "CONNECT http://search.example.com HTTP/1.1" request, which initiates the tunnel connection.  After that, the client and the remote system interact directly, which means that the SSL protocol negotiation occurs directly between EZproxy and the remote system, and that is no better than connecting directly to the remote server.

What needs to happen is that EZproxy needs to talk to something using the encryption that it supports, and that something needs to talk to the remote system using encryption that the remote system support.

Enter stunnel.

The work-around is actually pretty simple once you understand the problem.  What needs to be achieved looks like this:

EZproxy => stunnel (server) => stunnel (client) => remote server

Let's tackle this one piece at a time, using https://search.example.com/ for the demonstration destination website.

First we need to intercept all calls to the remote server.  Normally when EZproxy contacts a remote server, it performs a DNS lookup to find the IP address of the remote system and connects.  This needs to be short-circuited so that we can intercept the HTTP traffic and route it through stunnel instead.  

The easiest way to do this is to leverage your system's hosts file (/etc/hosts on *NIX systems, c:\Windows\System32\Drivers\etc\hosts on MS Windows systems) and add an override for the hostname:
127.0.0.2 search.example.com
Now your EZproxy server will use the locally defined address instead of the DNS entry:

Let's try to ping it:
# ping search.example.com
PING server.example.com (127.0.0.2) 56(84) bytes of data.
From 127.0.0.2 icmp_seq=1 Destination Net Unreachable
Well, the name override worked, but the IP address is not answering.  That's easy to fix, actually, by using interface aliases on the loopback interface:
ifconfig lo:0 127.0.0.2
Now the ping works:
# ping server.example.com
PING server.example.com (127.0.0.2) 56(84) bytes of data.
64 bytes from 127.0.0.2: icmp_seq=1 ttl=64 time=0.040 ms
So far so good.  The server now thinks that server.example.com is 127.0.0.2, and that IP address is also answering to basic networking functions.

Next we setup stunnel itself (in this example, all of the following will go into /etc/stunnel/ezproxy.conf).

First some global settings:
#foreground = yes
#debug = 7
syslog = yes
The foreground option and debug level are very useful in debugging, but be sure to comment them out when you're done (as seen above).  Recording access in syslog is handy for troubleshooting, but not strictly necessary.
Next we setup the local stunnel server.  This is the part that is going to accept SSL connections from EZproxy itself.
[server.example.com-server]
client = no
accept = 127.0.0.2:443
connect = 127.0.0.2:80
protocolHost = server.example.com:443
cert = /opt/ezproxy/ssl/00000001.crt
key = /opt/ezproxy/ssl/00000001.key
There are a few things to note here:

  1. "client = no", so this defines a service that is listening for connections.
  2. This server is setup to accept HTTPS connections on port 443 and turn back around and connect to the same IP address on a HTTP connection via port 80.
  3. This configuration is re-using the EZproxy SSL certificate and key pair, so the filenames used may be different on your system.

Next, add the stunnel client:
[server.example.com-client]
client = yes
accept = 127.0.0.2:80
connect = 192.0.2.1:443
protocolHost = 192.0.2.1:443
sni = server.example.com
sslVersion = TLSv1.2
Again a few notes to walk you through this:

  1. "client = yes", so this is the part that is going to be talking to the remote server.
  2. This accepts the un-encrypted traffic from the server block above.
  3. Stunnel supports SNI, which is being more widely adopted now that XP is no longer a concern.  If you have to enact this work-around, you should anticipate needing to use SNI as well.
  4. The last line is what allows EZproxy 5.x to use modern SSL with PCI-DSS compliant platforms.
Now that your hosts entry is in place, your IP alias is answering, and you have configured stunnel, simply point stunnel at the configuration file, and fire it up:
# stunnel /etc/stunnel/ezproxy.conf
If you enabled the debug and foreground options in the configuration file, you should see something like this emitted:
Service [search.example.com-server] accepted connection from 127.0.0.2:60181
connect_blocking: connected 127.0.0.2:80
Service [search.example.com-client] accepted connection from 127.0.0.2:51642
connect_blocking: connected 192.0.2.1:443
At this point, unless you are using a caching Squid proxy (and you ARE, aren't you?), you should be able to access https://search.example.com via your EZproxy server and load the content.  If you're running Squid, you will need to reload the cache to pick up on the new /etc/hosts entries before this will work.

So there you have it, another tool for your EZproxy toolbox.

4 comments:

  1. Upgrading to a EZproxy v6 is cheap. While your workaround probably works, i don't see this beeing worth it in the long run. Because you have to configure two more things for every site that has SSL problems. Lots of work to just save a couple hundred coins per year.

    ReplyDelete
  2. In the pre-OCLC days, with the licensing terms that Chris Zagar used, I would agree with you.

    These days, things are not as simple, or affordable: add a some zeros to "a couple hundred coins per year", and the economics of managing workarounds for EZproxy 5.x becomes quite different.

    Using a DevOps approach where much of this is maintained by systems management software and not hand-copied to every installation, and the work required to handle this is not overly burdensome.

    ReplyDelete
  3. Thanks for taking the time to create this article Andrew. We are experiencing SSL issues for a legacy proxy we host and this may help to fix the problem. Much appreciated!

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete