An exercise in frustration: Setting up Web Filtering on Sophos UTM

Overall, Sophos is a great platform. And part of what adds to that greatness is the web filtering options. By default, it gives a bunch of options that can be filtered out by default. However, most of these are suited for a business setting, but it definitely works well of the home environment. In fact, it features an inline download scanner virus scanner.

However, because of it’s complexity, and that some programs may not like what it is doing, it is one feature that may require the most amount of tweaking to get everything working just right.

First thing first here: If you’re not willing to spend hours reading logs and testing out rules just to get everything working just right, and not willing to listen to your wife or significant other to bitch because they can’t access Facebook/YouTube/Hulu or whatever iOS/Android game of the week for a few hours…..

Then set the HTTPS service to “URL Filtering” and not “Decrypt and scan”, or just turn off the Web Filter feature altogether.


To start off with, this is going to be a LONG post. There is a heck of a lot of features to go over, and some more technical stuff to do. I could break this up, but I’d rather do this all in one go. Just make sure you read this entire article first and then go through and mess with your router. Also, make sure you create a backup first!


 

By default, Web Filtering will be enabled. During the initial setup wizard, it will have asked you if you wanted to scan downloads, and if you wanted to block different categories of websites. These settings set up the defaults for the web filtering.

While there are a LOT of options here that could be configured, I am just going to go over the settings that allow you to simply filter all the web traffic, do so transparently and how to allow exceptions (so stuff like Netflix will work right).

Another important part here, is that I have Windows Server Essentials. The reason that this is important, is that it includes a CA Certificate. Why is this important? Because the full decryption and scanning of HTTPS traffic needs to be resigned with a trusted CA Certificate, otherwise you will get certificate errors all over the place.

 

Now to start off, you’ll need to log into the WebAdmin site. Once you’ve done so, you can click on the “Web Filtering” section in the “Current system configuration” on the dashboard or click on the “Web Protection” in the sidebar on the right and then click on the “Web Filtering” option underneath that.

Dashboard

From here, you can choose turn on and off filtering, by using the toggle switch in the top, right corner (though there is a better way to do this if you’re testing stuff).

Additionally, you can specify the networks that are to be filtered (if you’re segmenting the network and don’t want a specific segment filtered, you can remove it from the the “Allowed networks” list here). Additionally, you can change the operation mode (don’t, leave it on “Transparent Mode” here, so that you won’t have to do anything, but the traffic will be intercepted and filtered by default).

So basically, leave all the settings here at their defaults. This will get you the best experience for home use.

Web Filtering Main

The next section is the real fun one! See the “HTTPS” tab? Click on that and open it. See that notification about the “HTTPS Certificate Authority”? That’s what I was talking about above. This is a warning about it. We’ll cover that in a bit.

On this tab, you control HOW the HTTPS scanning occurs. The default option is the “URL filtering only”. This will only look at the URL and determine if it needs scanning.  For most people, this is what you want. This will not require doing anything else in most cases.

However, we will want to enable the “Decrypt and scan” option. Why is this better, you ask? Well, ever downloaded an EXE file and the URL included a bunch of random characters but not the actual file name? Well, the URL Filtering won’t catch the file if you’re downloading it via HTTPS. But the Decrypt and scan will. This is especially useful if you have clever teenagers in your home, trying to bypass your download restrictions.

You can also completely disable HTTPS scanning by enabling the “Do not proxy HTTPS traffic in Transparent Mode”, if you want. But that defeats the point, especially as more websites are moving over to HTTPS (and Google increasing page ranks for HTTPS only sites).

So select the “Decrypt and scan” here.

Web Filtering Main-HTTPS

 

Now for the real fun here. We want to set up the “Base Policy” to … well be a bit more lenient in some regards and more restrictive in others. So open the “Policies” tab and click on the “Default content filter action” option at the bottom of the screen. If you want to get super trick, you could create multiple profiles and use the authentication stuff to finely tune the settings (so some users can bypass certain parts, but other users can’t).

However, I’m just covering the basics here. If you want in depth, leave a comment or email me, and I’ll look into this (this is much more complex, and more likely to break my network, so I’ve avoided it).

Web Filtering Main-Policies

 

The Filter action information here is the heart and sole of what the Web Filter does. And there is a LOT of options here.

On the “Categories” tab, leave it as “Allow all content”, make sure the “Block Spyware infection and communication” option is enabled, and choose which categories that you want to block (there is a warn and quota option, but chances are, “block” is the best option). A couple of good categories to add to the block list are “Criminal Activities”, “Drugs”, and “Extremist Sites”. “Suspicious” and “Weapons” are already allowed. Feel to change those, as needed (especially, if you’re an active member of the NRA).

At the bottom, there is a section for “Uncategorized websites”. I would recommend leaving that as “Allow”, or you may block a LOT of websites unintentionally.

At the very bottom is a check-box to “Block websites with a reputation below a threshold of” option. I would recommend enabling this, as this may block sites not on the list above but are not trustworthy. Setting the option to “Neutral” should block a vast majority of those sites, without impacting normal browsing.

Web Filtering Main-Policies-Categories

If you want to block specific websites out of the door, the “Websites” tab here is a good option to use. This would be a great place to add Ad websites, as they will be outright blocked. However, I would recommend not adding them here, unless you’re absolutely certain. There is a section elsewhere that is much more conducive to “testing”.

Web Filtering Main-Policies-websites

Now, the “Downloads” tab is where the real fun begins!

On this tab, you can specify which extensions and which mime types are blocked. Here is why the “decrypt” option is really important! You can add all sorts of entries here.

However, by default, it’s very restrictive. It won’t even let you download EXE’s, MSI’s or other installer types. This is fine for a lot of  these extensions, but we want to be able to download the files.

Remove “exe” and “msi” from the “Blocked file extensions” list, by clicking on the trash can icon next to them. Then add these to the “Warned file extensions” list, by clicking on the green “+” and typing the extension. You may want to add extensions like “zip”, “rar”, “7z”, “gz” or the like here, so it will scan the archives when it downloads them, as well.

Additionally, you can add MIME types here, such as “application/binary” or “application/octet-stream” to identify these files better. “Fileext.com” is a great resource for identifying common MIME Types used based on the file extensions.

This will cause the Sophos UTM Web Filter to warn you about these file types before downloading them. This is great for making sure that you actually want to download them, before the browser or an app just downloads and installs them! However, if you want it to just scan the files, remove them from both lists.

You’ll notice that a lot of common attack vectors are blocked in this list by default, such as “scr”, “hta”, and “vbs” extensions. So you may not need to do a lot of configuring here.

However, you can block large downloads. Just keep in mind, that depending on the size, you may block ISOs and other large downloads from occurring by enabling this option.

Web Filtering Main-Policies-Downloads

The “Antivirus” tab is my personal favorite section here! This is why I have become very lax about antivirus on my personal systems.

Make sure the “Use Antivirus scanning” option is enabled. You have the option of a single scan or a dual scan. I highly recommend the dual scan, as is uses both Sophos Endpoint and Avira antivirus scanning engines, which are both good scanners.  But scanning with both will take more time and use more resources, but in most cases, it’s absolutely worth it.

There is the option to “Block potentially unwanted applications”, check this as well. This will block a lot of ad-ware and other potentially unwanted files from being downloaded as well.

As for the “Do not scan files larger than X Megabytes” option, that’s up to you. However, any file larger than this will not be scanned. By default, this is 30 MBs, but that’s too small in my opinion. However, increasing this will use more resources and take longer to download. But that may be worth it for you. I would recommend 300MBs for the size here.

Also, on this page is the option to strip out <SCRIPT> and <OBJECT> tags in all the websites filtered, by using the “Disable JavaScript” and “Remove embedded objects” options respectively. I would recommend against this, as it may significantly impact your browsing experience.

Web Filtering Main-Policies-Antivirus

The final page has some neat settings as well. Here you can forcible enable the “SafeSearch” features of Google, Bing and/or Yahoo. This is great if you have teenagers in your home, as you can at least reduce their ability to find porn or other adult content while browsing the web!

There are some other options here, such as YouTube for Schools, but that requires a School ID, which I’m sure is harder to get a hold of.

You also can add a “parent proxy”, so if you’re using a web service proxy, you can specify that here and EVERYTHING on the network will be routed through that proxy as well.

Web Filtering Main-Policies-Misc

Now you can save the settings, and you’re filtering your web traffic!

 

Up Next! …..

However, we’re not quite done yet! But because of the length of this post, and the fact that were not even HALF done, I’ve split up these posts into multiple parts.

Up next is how to set up the certificates and everything to get HTTPS working properly!

And by now, you may be seeing why this is an exercise in frustration. It’s not quick, nor is it exactly simple.

Next, see how set up Web Filter Certificates in Sophos UTM

 

Author: Drashna Jael're

Drashna Jael're

7 thoughts on “An exercise in frustration: Setting up Web Filtering on Sophos UTM”

  1. Great stuff. Very informative. So I follow your instructions and setup “Decrypt and Scan”. Now to install the CA’s. But I can’t get back to your site to follow the second post, Sophos won’t let me in, no CA. So back to Sophos and turn off web filtering so I can load the CA’s on to my machines. Meanwhile #2 son is trying watch some YouTube and can’t cause I haven’t got to his machine yet. While I am loading the CA’s on his my wife can’t watch her movies. All hell has broken loose but I am safe behind my Sophos. Haven’t tried the netflix on the Xbox one yet. Will it need some CA attention?

    1. Sorry, Part 3 covers all of that, actually.

      It’s even larger than the other two posts… so it’s coming up.
      In the meanwhile…
      Add these as exceptions (in the filter options, “matching these urls”:
      ^https?://([A-Za-z0-9.-]*\.)?ne?t?fli?x(img|ext|video)?\.(com|net)/
      ^https?://[\d+(\.\d+){3}/]*/[0-9]{8}\.ism
      ^https?://([A-Za-z0-9.-]*\.)?netflix-*\.vo\.llnwd\.net
      ^https?://[\d+(\.\d+){3}/]*/[0-9]{9}\.ism
      ^https?://[\d+(\.\d+){3}/]*/[0-9]{10}\.ism

      for netflix.

      ^https?://[A-Za-z0-9.-]*\.ytimg\.com/
      ^https?://([A-Za-z0-9.-]*\.)youtube(-nocookie)?\.com/
      ^https?://[A-Za-z0-9.-]*\.ggpht.com/
      ^https://www\.googleapis\.com/

      For youtube.

      As for my site. I’m not sure about that. It should.
      Try adding “drashna.net” to the “website” section (in filter options) and add it as a “trusted” site under “IT” (or such) category.

  2. If you’re just testing things and don’t want to disrupt other people in your household, you could find out their IPs and create an exception to all filtering for traffic coming from those hosts. That way you get to tinker with things and they don’t have to deal with the Internet going up and down.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.