{"id":254,"date":"2012-09-06T19:24:47","date_gmt":"2012-09-06T23:24:47","guid":{"rendered":"https:\/\/infotechguy.net\/?p=254"},"modified":"2025-02-22T13:00:18","modified_gmt":"2025-02-22T18:00:18","slug":"network-adblocking-using-squid-squidguard-and-iptables","status":"publish","type":"post","link":"https:\/\/infotechguy.net\/?p=254","title":{"rendered":"Linux — SquidProxy Network Adblocking using Squid1.4"},"content":{"rendered":"

I originally discovered Adblock Plus<\/a> when I first downloaded Firefox many years ago. Since then I’ve installed the Adblock plugin right after Firefox, etc. It’s become so standard that I almost think Firefox should just bundle them together. Including it in it’s default install exe.<\/p>\n

Adblock Plus works as if it were a local content policy,\u00a0 filtering each request you make with Firefox. Each URL, each domain, each link you navigate to is check based on a static blacklist of expressions and URLs. If a match is found, Adblock Plus simply discards the content from rendering. The discarding and allowing content to load is managed by the Content Policy engine within Firefox. Adblock Plus simply utilizes this in order to block the unwanted contents. Or at least this is my comprehension of how it works. :-p<\/p>\n

Setting up your own Network wide Adblocker<\/h3>\n

The purpose of this guide and tutorial is to instruct you on how to set up your own network based adblocker. Expections after completion is every client browser on the network will benefit from adblocking. I will include as much as possible, and feel free to ping me with questions or comment down below.<\/p>\n

You will need:<\/p>\n

    \n
  1. Computer that will be running the Web Proxy. (For this article, see specs below)<\/li>\n
  2. OS that will host the Proxy Software. (For this article, Ubuntu<\/a> 12.04 32-bit Server)<\/li>\n
  3. Proxy software<\/a> that allows rewrite engines\/programs. (squidGuard<\/a>)<\/li>\n
  4. Content-Control-Software<\/a> or URL Redirect Application(This will consume your blacklists)<\/li>\n
  5. URL and RegExp Blacklists<\/a> consumable by your Content-Control-Software (Here<\/a> are some free ones)<\/li>\n
  6. Optional: ipTables for transparent proxy redirection<\/li>\n
  7. Patients and enthusiasm :-p<\/li>\n<\/ol>\n

    <\/p>\n

    Step 1: Installling OS and software<\/strong><\/h4>\n
      \n
    1. Spec’ing out your hardware. Keep in mind your sizing and scope of clients you will be serving
      \nFor this tutorial I used the following:
      \n–Dual Core -Intel(R) Xeon(TM) CPU 2.66GHz 32-bit
      \n–4GB DDR RAM
      \n–320GB SATA HD
      \n–Dual 10\/100\/1000 NIC<\/em><\/li>\n
    2. Download a ISO copy of Ubuntu or Debian.<\/li>\n
    3. Install Ubuntu, for tutorial see here<\/a>.<\/li>\n
    4. Make sure you are a sudo users or have root access, needed for the next steps.<\/li>\n
    5. After Install update your system to the latest via\n
      sudo apt-get update && sudo apt-get upgrade -y<\/pre>\n<\/li>\n
    6. Next install squidProxy\n
      sudo apt-get install squid3 -y<\/pre>\n<\/li>\n
    7. Install squidGuard from the repositories\n
      sudo apt-get install squidGuard -y<\/pre>\n<\/li>\n
    8. Setting static IP address on the server. Do a ifconfig eth0<\/code> to figure out what your current IP is. Mine was 192.168.0.113<\/strong>. We will assume this is a \/24 network, so pick a lower number for the last octet. I chose  192.168.0.5<\/strong>. Reason for this is out side the scope of this article. Perform the static ip address config.
      \nAdd the following in<\/p>\n
      sudo vi \/etc\/network\/interfaces \r\n\r\nauto eth1 \r\niface eth1 inet static \r\n      address 192.168.0.5 \r\n      netmask 255.255.255.0<\/pre>\n<\/li>\n
    9. Restart network interfaces. Make sure you are local, you will lose remote access!!.\n
      sudo \/etc\/init.d\/networking restart<\/pre>\n<\/li>\n
    10. Check ifconfig eth0<\/code> you should have 192.168.0.5<\/strong>(or the IP you picked).<\/li>\n<\/ol>\n
      ***The base installation is now complete and we are now ready to configure our service applications!<\/p>\n
      Step 2: Configuring squidProxy<\/strong><\/h4>\n
        \n
      1. Make a copy of your default config file.\n
        sudo cp \/etc\/squid3\/squid.conf \/etc\/squid3\/squid.conf.bak<\/pre>\n<\/li>\n
      2. Null the squid.conf file.\n
        sudo su - cat \/dev\/null > \/etc\/squid3\/squid.conf<\/pre>\n<\/li>\n
      3. Edit the squid.conf vith vi and paste the following lines.\n
        sudo vi \/etc\/squid3\/squid.conf\r\n#acl lists\r\nacl manager proto cache_object\r\nacl localhost src 127.0.0.1\/32 ::1\r\nacl to_localhost dst 127.0.0.0\/8 0.0.0.0\/32 ::1\r\nacl localnet src 192.168.0.0\/24 # RFC1918 possible internal network\r\n\r\n#port connections\r\nacl SSL_ports port 443\r\nacl SSL method CONNECT\r\nacl Safe_ports port 80          # http\r\nacl Safe_ports port 21          # ftp\r\nacl Safe_ports port 443         # https\r\nacl Safe_ports port 70          # gopher\r\nacl Safe_ports port 210         # wais\r\nacl Safe_ports port 1025-65535  # unregistered ports\r\nacl Safe_ports port 280         # http-mgmt\r\nacl Safe_ports port 488         # gss-http\r\nacl Safe_ports port 591         # filemaker\r\nacl Safe_ports port 777         # multiling http\r\nacl CONNECT method CONNECT\r\n\r\n#allow\/deny\r\nhttp_access allow manager localhost\r\nhttp_access deny manager\r\nhttp_access allow localnet\r\n\r\n# Deny requests to certain unsafe ports\r\nhttp_access deny !Safe_ports\r\n\r\n# Deny CONNECT to other than secure SSL ports\r\nhttp_access deny CONNECT !SSL_ports\r\n\r\n# Example rule allowing access from your local networks.\r\nhttp_access allow localnet\r\nhttp_access allow localhost\r\n\r\n# And finally deny all other access to this proxy\r\nhttp_access deny all\r\n\r\n#bind address default port is 3128\r\nhttp_port 192.168.0.5:8080\r\n\r\n#cache directory\r\ncache_dir ufs \/home\/serveruser\/squidcache\/ 512 16 128\r\ncache_mem 2048MB\r\n#coredump_dir \/home\/serveruser\/squidcache\/\r\n\r\n#log\r\ncache_store_log \/var\/log\/squid3\/store.log\r\n<\/pre>\n<\/li>\n
      4. Next start the service up.\n
        sudo service squid3 restart<\/pre>\n<\/li>\n
      5. If you encounter any issues or error message, check the syslog\n
        less \/var\/log\/syslog<\/pre>\n<\/li>\n
      6. TEST with Firefox. Go into Firefox settings –> Connection Settings. Enter in the IP address and Port of your proxy server. Similar to this \"\"<\/a> Try to browse to some sites like Google, MSN, Facebook, etc. Ensure you can get to them. Next, to make this a thorough test and ensure you are connecting through the proxy. Go back to your Proxy server’s terminal. Enter the following to turn off the proxy service.\n
        sudo service squid3 stop<\/pre>\n
        After that completes try to navigate and browse on the client machines. You should get an error.<\/li>\n<\/ol>\n
        ***Congratulations you have successfully set up your first Proxy Server with squidProxy!!!<\/p>\n
        Step 3: Configuring squidGuard and Blacklists<\/strong><\/h4>\n
          \n
        1. Make a backup of the squidGuard config file.\n
          sudo cp \/etc\/squid3\/squidGuard.conf \/etc\/squid3\/squidGuard.conf.bak<\/pre>\n<\/li>\n
        2. Grab a blacklist from here<\/a>.\n
          cd ~ && wget http:\/\/squidguard.mesd.k12.or.us\/blacklists.tgz<\/pre>\n<\/li>\n
        3. Uncompress.\n
          tar xzvf blacklists.tgz<\/pre>\n<\/li>\n
        4. This Uncompresses the folder structure for all the blacklists categories, i.e. Ads, Porn, Gambling, etc. We are only concerned with Ads, so grab it’s path.\n
          cd blacklists\/ads\/ \r\npwd<\/pre>\n
          COPY this path, need it for later.<\/li>\n
        5. We have to make these blacklists readable by the user that squid runs as.\n
          cd ~ && sudo chown -R proxy.proxy blacklists \r\nsudo chmod -R 750 blacklists<\/pre>\n
          I had to use the permission bits of 750, it was the only permissions that would allow squid to read the blacklists. I put a 0 at the end because I don’t want the list readable by any other users on the system.<\/li>\n
        6. Now time to edit the squidGuard.conf file.\n
          sudo vi \/etc\/squid3\/squidGuard.conf\r\ndbhome \/home\/serveruser\/squidGuard\r\nlogdir \/var\/log\/squid3\r\n#create ads category\r\ndest ads {\r\n        #location of blacklists, domains, urls, expressions. \r\n        domainlist blacklists\/ads\/domains\r\n        urllist blacklists\/ads\/urls\r\n        expressionlist blacklists\/ads\/expressions\r\n}\r\nacl {\r\n        default {\r\n                #allow except 'ads'\r\n                pass !ads all\r\n                #redirect to transparent gif\r\n                redirect http:\/\/localhost\/blank.gif\r\n\r\n        }\r\n}<\/pre>\n<\/li>\n
        7. Now add the squidGuard specific’s to the squid.conf file, so the main squid process is aware of squidGuard. Add these lines to the end of your squid.conf file.\n
          sudo vi \/etc\/squid3\/squid.conf \r\n\r\n#rewrite program squidGuard \r\nurl_rewrite_program \/usr\/bin\/squidGuard -c \/etc\/squid3\/squidGuard.conf\r\nurl_rewrite_children 5 #threads \r\nurl_rewrite_concurrency 0 #jobs per threads<\/pre>\n<\/li>\n
        8. Initialize the squidGuard database files, to consume the blacklists you just downloaded. Needs to be done everytime you update the list.\n
          sudo squidGuard -C all<\/pre>\n
          Should be fairly quick, if it hangs, squidGuard probably cannot read the blacklist directory. Check your syslog.<\/li>\n
        9. For the next step I recommend having two terminal windows open to your proxy server. This will make it easier to tail the logs when you try to start it for the first time.<\/li>\n
        10. Restart squid3 to pick up the new configuration items(i.e. squidGuard) Make sure you tail your syslog with the other window.\n
          sudo service squid3 restart<\/pre>\n
          In your squidGuard log you should see the following lines:<\/p>\n
          less \/var\/log\/squid3\/squidGuard \r\nsquidGuard 1.4 started \r\nsquidGuard ready for requests<\/pre>\n
          You will also see in the log how it loaded the dbfiles you defined in your squidGuard.conf file.<\/li>\n
        11. Testing. Ensure your client’s firefox still has Connection Settings pointing at your proxy server. Now browse to a website that has a lot of ads. I suggest p2p sites, usually utilize a crap load of ads.<\/li>\n<\/ol>\n
          ***Congratulations you have protected your network from ADs!!!<\/p>\n
          Optional Step 4: Transparent Proxy Redirection via iptables (optional)<\/strong><\/h4>\n
            \n
          1. \n
              \n
            1. This next section assumes you have a working IPTables setup, with a Router\/Firewall at 192.168.0.1<\/strong> and a Proxy Server at 192.168.0.5<\/strong><\/li>\n
            2. Need to add two statements.\n
              sudo iptables -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.0.5:8080<\/pre>\n<\/li>\n
            3. ***NOTICE We are only specifying port 80, not 443!<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
              Port 443 bypasses the proxy as we are not doing SSL Interception.<\/p>\n