nguvu

pfSense WPAD/PAC proxy configuration guide

Last revised 11 December 2017.

I don’t use a proxy on my network for ad filtering any more as I moved over to pfBlockerNG which provides greater control and flexibility however I’ve received a number of requests for an updated guide so here it is. I initially set my proxy to work transparently but this method didn’t work fully with all the recent changes to my network. To enable my filtering system to continue working I had to configure each device separately to use port 3128 which was an inconvenience. A system called WPAD/PAC (Web Proxy Auto-discovery Protocol & Proxy Auto Configure) promised an easy workaround and the ability for devices to auto configure themselves when connecting to to my network. This wasn’t as easy as I expected partly due to different devices having different ways of accessing the configuration files.

Create proxy discovery files

First create the proxy configuration files. Three of them are required due to different browsers using different filenames to obtain proxy configuration information.

We’ll create the initial proxy.pac file, and then create two ‘linked’ files to wpad.dat and wpad.da files which will make keeping them all in sync easier.

ee /usr/local/www/nginx/proxy.pac

Paste the following into the file you just created. Change my proxy address (192.168.1.1) to yours obviously.

function FindProxyForURL(url,host)
{
  // If the requested website is hosted within the internal networks, send direct.
  if (isPlainHostName(host) ||
    shExpMatch(host, "*.local.lan") ||
    isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
    isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
    isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
    isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
    return "DIRECT";

  //Default rule: all other traffic use below proxy
  return "PROXY 192.168.1.1:3128";
}

This PAC file checks to see if the host which is being looked up is a plain host name, i.e “intranet”, or is in the local networks, if it is, the access bypasses the proxy (“DIRECT”). If neither of the above conditions are met then the access is made via the proxy and port 3128.

Its possible to extend this PAC files functionality to include features such as black/white lists etc,

Create the other two file variations by generating links. This is preferable as it keeps all versions synced if you make any changes in future.

ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.dat

ln -s /usr/local/www/nginx/proxy.pac /usr/local/www/nginx/wpad.da

Verify your folder looks something like this when done,

$ ls -l
-rw-r--r--  1 root  wheel  537 Jul 19 12:07 50x.html
-rw-r--r--  1 root  wheel  612 Jul 19 12:07 index.html
-rw-r--r--  1 root  wheel  212 Aug  6 11:43 proxy.pac
lrwxr-xr-x  1 root  wheel    9 Aug  6 11:45 wpad.da -> proxy.pac
lrwxr-xr-x  1 root  wheel    9 Aug  6 11:45 wpad.dat -> proxy.pac

Setup a second nginx web server

Best practice dictates we should only allow access to the pfSense webConfigurator via secure HTTP and on a different port to 443. This leaves us with a slight problem. The default and only system file server is dedicated to serving the webConfigurator pages on this secure port and can’t be used simultaneously to serve WPAD data on port 80 any more. To be able to serve up the WAD config files, we need to create a second file server which will handle the WPAD requests for us.

Lets start by duplicating the systems default nginx configuration file. We can use this as the basis for our second nginx instance. Don’t try and create this file in /var/etc folder as it will be deleted on reboot. We’ll use the same folder as where our config files are stored.

ee /usr/local/etc/nginx/nginx-wpad.conf

Paste the contents below into this file.

worker_processes  1;
pid               /var/run/nginx-wpad.pid;

events {
    worker_connections  1024;
}

http {
    include       wpad-mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    server {
        listen       80;
        server_name  wpad;

        location / {
            root   /usr/local/www/nginx;
            index  index.html index.htm;
        }
    }
}

Add additional filetypes

Lets create a specific wpad-mime.types for this instance of nginx by entering ee /usr/local/etc/nginx/wpad-mime.types. Paste the following lines into this file.

types {
    text/html                             html htm shtml;
    text/css                              css;
    image/jpeg                            jpeg jpg;
    text/plain                            txt;

    application/x-ns-proxy-autoconfig     pac;
    application/x-ns-proxy-autoconfig     dat;
    application/x-ns-proxy-autoconfig     da;
}

Verify second nginx is working

Test our second nginx instance works by entering the following command and verifying you don’t see any errors.

/usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf

You can verify both Nginx servers are running by entering

ps ax | grep -i "nginx: master"

the result should include both nginx instances are running with their respective configuration files, for example.

$ ps ax | grep -i "nginx: master"
ps ax | grep -i "nginx: master"
52936  -  Is       0:00.00 nginx: master process /usr/local/sbin/nginx -c /var/etc/nginx-webConfigurator.conf (nginx)
75222  -  Is       0:00.00 nginx: master process /usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf (nginx)
84471  2  S+       0:00.00 grep -i nginx: master

NOTE: If you have difficulties starting a second NGINX instance and see an error reporting the port is already in use, please verify that the WebConfigurator redirect is disabled. Navgate to System > Advanced > Admin Access and verify WebGUI redirect, Disable webConfigurator redirect is selected.

Set nginx-wpad to autorun

We’ll configure a cron job to start the second nginx instance on boot, or restart it should it crash for any reason. Create a file to check if our nginx module is already running, and if not, start it by entering

ee /root/check_proxy_wpad.sh

Paste the following into this file you just created.

#!/bin/sh
pgrep -F /var/run/nginx-wpad.pid
if [ $? == 0 ];then
    /bin/echo running
else
    /usr/local/sbin/nginx -c /usr/local/etc/nginx/nginx-wpad.conf
fi

Set this script as executable

chmod +x /root/check_proxy_wpad.sh

Create a cron job to autorun

You could do this via the command line but the Cron tool available under System > Packages makes life a lot easier. Install it if you haven’t already and navigate to Services > Cron

Create a job as

Your job should look like this

Cron edit

Click Save

Its worth rebooting and verifying the second nginx instance starts automatically using the command shell and the command ps aux | grep -i "nginx: master" command

Serve the WPAD files

There are two methods of serving WPAD files, DNS and DHCP. You can use either one or both together, in my personal testing I’ve found DNS to be the more reliable though I have heard other users who found DHCP to be more reliable.

It is possible to run both systems concurrently, if an address isn’t received via DHCP the system will fall back to DNS next. As of when I wrote the original guide in 2014, Firefox didn’t support WPAD via DHCP which was a known bug for quite a number of years. I’ll detail both methods but suggest you try both and see which one works best for you and the clients you want to serve.

Configure DNS

I’ve upgraded from the DNS Forwarder to DNS Resolver which this guide will focus on.

Navigate to Service > DNS Resolver

We will now create a wpad host override for ‘wpad’.

Edit host

Test the WPAD host override works with nslookup. This should report the host override IP address back to you. (192.168.20.1 is my DNS server)

$ nslookup wpad.local.lan
Server:   192.168.20.1
Address:  192.168.20.1#53

Name: wpad.local.lan
Address: 192.168.1.1

Its also worth testing to make sure you can download the wpad.dat file (and other two) from within your browser. Open your browser and navigate to http://wpad.local.lan/wpad.dat. You can use curl too if you prefer, here curl is returning my wpad.da file correctly.

$ curl wpad.local.lan/wpad.da
function FindProxyForURL(url,host)
{
  // If the requested website is hosted within the internal networks, send direct.
  if (isPlainHostName(host) ||
    shExpMatch(host, "*.local.lan") ||
    isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
    isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
    isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
    isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
    return "DIRECT";

  //Default rule: all other traffic use below proxy
  return "PROXY 192.168.1.1:3128";
}

Configure DHCP

In your DHCP server, add the following three options in the ‘Additional BOOTP/DHCP section’.

For the wpad.dat file

number: 252
type: string
value: "http://wpad.local.lan/wpad.dat"

For the wpad.da file

number: 252
type: string
value: "http://wpad.local.lan/wpad.da"

and finally for the proxy.pac file

number: 252
type: string
value: "http://wpad.local.lan/proxy.pac"

DHCP Server

Verification

I ran Wireshark at this stage to make sure everything was working as expected. Here’s a screen grab showing

You can see top to bottom where I’ve highlighted key pieces of info, i.e

Wireshark verification

Client and browser configuration

Assuming you can download the WPAD configuration files without issue in your browser, this part should be easy. I’ll provide a few examples of my configurations for my Macbook and Firefox but other clients should be similar enough.

Mac Network Settings

Mac Settings

Firefox Settings

Note its possible to tell Firefox to use system settings too but this is an alternative.

Firefox config

Windows / Internet Explorer

Internet Explorer

Iphone & Ipad

Auto discovery works in iOS too. Head into settings, click the info symbol besides your network connection, scroll down and select ‘Auto’. Don’t fill in any details, just leave them blank.

iPhone proxy

iPad proxy

Firewall

Ensure you have a firewall rule that will permit port 80 traffic to allow the delivery of the wpad file from the server.

Possible next steps

Block surfing on port 80

Once you have the WPAD redirection working and all traffic is flowing via port 3128, you could create a firewall rule to prevent browsing via the usual HTTP port 80 ensuring all browsing is only done via your proxy.

NAT backup

To catch any PCs which aren’t configured with ‘auto configure’ in their settings, its possible to implement a port forward to forward any traffic for port 80 back to 3128.

References

FindProxyforURL
Nginx docs

Changelog

11 December 2017
Improved text formatting, fixed typos

17 November 2017
Added details re Disabling webConfigurator redirect.

2 September 2017
pgrep -F /var/run/nginx-wpad.pid use
updated proxy.pac file