Thumbs on tunnels – terminate tunnel and check content – DLP

The past was easy peasy, not the case anymore these days. HTTPS was rarely used and only where it really needed to be. Anyone on the wire could see what’s going on. And yeah, it meant literally everyone.

Was it good? No, not at all. Everyone could track anyone, collect information, behavior, etc. Result? Everyone started to move towards HTTPS (ssl then tls). This basically means couple of good and bad outcomes. Squid cache ration went down, and I mean very, very low these days. Most connections handled by Squid these days are tunnels (CONNECT) and anything can be sent through such tunnel, not only private data, credit card numbers, etc. but also ads and virus (sic!).

This created specific need to be able to check the content of the stream, especially in enterprise environment. All the DLP systems can only work on streams if they can check the payload, decrypted traffic which means they need to be so well known man-in-the-middle. This means to be tunnel end-point from client side and initiation of new tunnel to server. Only this allows to inspect the traffic.

At the same time one could ask, hey how about certificates? The long winded answer can be found elsewhere, but short answer is that in enterprise environment there is at least one Root Certificate Authority (CA) and Intermediate Certificate Authority can be created. The root or intermediate CA (called CA afterwards) together with key file is uploaded to the DLP system allowing it to generate new certificates on the fly for terminated tunnels.

The CA is already trusted in enterprise environment as Root CA certificate is added to Trusted CA key ring on client host as part of operating system deployment package or domain connection.

Since client trusts Root CA it automatically trusts certificates signed by the given CA – this is how Public Key Infrastructure works. Additionally the DLP system usually tries to mimic all original certificate parameters and only CA related details are different. But people rarely check details of certificate if everything is in green and no popup/error is raised.

private DLP

With all above being said one can have their own DLP system based on Squid. This “little piece of software” is great in handling huge loads, caching data and calling others for help. This is what we’re going to do today.

Squid to terminate SSL connection uses ssl_bump functionality. The Ubunty 16.04 LTS default package does not allow to use this great functionality hence we need to start with little configuration.

Let’s get sources and all needed libraries (for all below any sudo call is skipped as it just makes the output longer and you, dear reader certainly know how to use sudo).

then we need to apply patch to enable ssl as needed

To apply patch, use your standard methodology patch -p<level> < diff-file.patch

Afterward squid package and any missing packages need to be installed

Certificate generation

First of all we need to have folder where we will store it all

 

The certificates mea:

Depending on client system certificate import will look differently. The good idea might be to place it on some easily accessible server, i.e. local wpad system.

Once certificate is downloaded it should be installed. On windows this can be done with:

Some apps do not respect Operating System level certificates, but most will. Some apps might need to be restarted it shouldn’t be required to restart full system, but who knows what type of ancient system one can be using?

All the new certificates generated on the fly need to be stored somewhere. The folder needs to be created:

And update squid.conf  (not all SSL cert ERROR related flags should be as below, tune it up to your needs).

Create set of files under /etc/squid/ssl_bump:

  1. sslBumpnet – subnets/hosts which we will bump, can be selective if needed
  2. sslnoBumpnet – these subnets/hosts we won’t bump (see logical construction above and tune it to your needs
  3. sslnoBumpnetdst – these domains/servers we won’t bump
  4. sslnoBumpSites

Example sslnoBumpnetdst as it might be tricky to set it up at the beginning. Some apps have built-in certificates and verify connection against them, i.e. Google Play store and couple of others.

Interesting was to find out what banks are doing with data and identify as in one case at least that for stats reasons were sending out GET request with bank account details, including saldo and transaction details to on-line stats agency. This was against any bank standards and bank should be prosecuted due to this. This was only possible after terminating tunnels as otherwise GET wasn’t visible. This can only be acceptable for tests and for your own private use at home in the isolated lab (as always).

Below list was fine tuned based on tests and was longer earlier.

The last point is to restart squid

To test it, select port 3127 as proxy. Once all tested transparent interception (don’t forget to have CA trusted on client side).

 

For non-Android based systems proxy.pac/wpad.dat file could be created

To get that working the wpad.yourdomain and wpad host needs to be resolved to your web server and thw wpad.dat/proxy.pac file needs to be in root folder.

Android platform is prone to not process this file (at least as for April 2017 and manual proxy settings in WiFi section need to be set.

This should all work… but hey, problems are expected.

  1. The usual problem is that CA is not trusted.
  2. Client/app has it’s own CA list and does not trust operating system level list. This required adding CA to this trusted app ring.
  3. Perl/Python based apps might have their local ssl and root cert trusted ring, see: Kodi and ssl_bump Squid.
  4. To troubleshoot full logging is often useful, see Log all request details on Squid

 

Links and reads

  1. Diladele non-free:
    https://docs.diladele.com/administrator_guide_4_0/installation_and_removal/install_on_ubuntu.html
    https://www.diladele.com/solution.html
  2. ClamAV & SquidClamAV
    http://terminal28.com/how-to-install-and-configure-squid-proxy-server-clamav-squidclamav-c-icap-server-debian-linux/
    http://squidclamav.darold.net/install.html
  3. http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit
  4. http://wiki.squid-cache.org/Features/SslPeekAndSplice
  5. http://www.squid-cache.org/Doc/config/ssl_bump/
  6. http://wiki.squid-cache.org/Features/DynamicSslCert
  7. http://ubuntuserverguide.com/2013/12/how-to-filter-https-traffic-with-squid-3-on-ubuntu-server-13-10.htmlhttps://forums.kali.org/showthread.php?23036-SSL-Interception-with-Squid3-(MITM)
  8. http://marek.helion.pl/install/squid.html
  9. http://thejimmahknows.com/squid-3-1-caching-proxy-with-ssl/
  10. http://www.squid-cache.org/Doc/config/acl/https://docs.diladele.com/administrator_guide_4_0/system_configuration/https_filtering/recompile_squid.htmlhttps://smoothnet.org/squid-v3-5-proxy-with-ssl-bump/

Leave a Reply

Your email address will not be published. Required fields are marked *