Raspberry Pi as "ADBLOCK" box for the entire network

These days most of us have multiple devices connected to the internet. AD campaigns are so evil and boring. Instead of installing some AD block extension on each device, I'm using dnsmasq service (lightweight DHCP and DNS server) which is installed on a Raspberry Pi. Dnsmasq blocks ADs based on DNS queries for the entire network. Distribution on RaspberryPi is Raspbian.

FYI there is a adblock package for openwrt which will do the same on the router itself. NOTICE: you should have at least 128MB of RAM on your Openwrt compatible router - in my case I have a TPLink 1043ND with only 32MB of RAM, so RaspberryPi suits me better + I had one laying around.

So let's start with Raspberry Pi configuration

Install and configure DNS service.

apt-get install dnsmasq

Create dnsmasq config file.

vim /etc/dnsmasq.d/adb.conf

interface=eth0
no-poll
bogus-priv
neg-ttl=3600
cache-size=10000
dns-forward-max=1000
domain-needed
resolv-file=/etc/resolv.personal
addn-hosts=/etc/hosts.adblock
log-queries # we will use it for debug purpose, it should be turned off later ...

Create two files. The first /etc/resolv.personal file will contain a list of public DNS servers where queried DNS requests will be forwarded, before they have been cached locally. The second /etc/hosts.adblock file will hold the list of malicious sites that will be blocked.

touch /etc/resolv.personal
touch /etc/hosts.adblock

I'm using Google public DNS servers.

vim /etc/resolv.personal

nameserver 8.8.8.8
nameserver 8.8.4.4

Edit configuration file for DNS resolvers

vim /etc/resolv.conf

nameserver 127.0.0.1

I decided to "lock down" file for further modifications.

chattr +i /etc/resolv.conf

Restart dnsmasq service

/etc/init.d/dnsmasq restart

Check if service is listening on defined interface and port.

netstat -ulnp | grep dnsmasq

udp        0      0 0.0.0.0:53              0.0.0.0:*                           31880/dnsmasq   
udp6       0      0 :::53                   :::*  

Create script which will download lists of "malicious" domains from sources.

vim /root/updateadb.sh

#!/bin/bash

destinationIP="0.0.0.0"
outlist='/etc/hosts.adblock'
tempoutlist="$outlist.tmp"
unsortedoutlist='$tempoutlist.tmp'

echo "Getting - ADAWAY - focus on mobile ads, infrequent updates, approx. 400 entries"
curl -s https://adaway.org/hosts.txt | grep -v "#" | grep -v "0.0.0.0" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - DISCONNECT - mozilla driven content blocklist, numerous updates on the same day, approx. 6.500 entries"
curl -s https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt | grep -v "#" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - DSHIELD - broad blocklist for suspicious domains, daily updates, approx. 4.500 entries"
curl -s http://www.dshield.org/feeds/suspiciousdomains_Low.txt | grep -v "#" | grep -v "Site" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - MALWARE - broad blocklist for malware domains, daily updates, approx. 16.000 entries"
curl -s https://mirror.cedia.org.ec/malwaredomains/justdomains | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - MALWARELIST - generic blocklist for malware domains, daily updates, approx. 1.500 entries"
curl -s http://www.malwaredomainlist.com/hostslist/hosts.txt | grep -v "#" | sed '/^$/d' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - OPENPHISH - focus on phishing domains, numerous updates on the same day, approx. 1.800 entries"
curl -s https://openphish.com/feed.txt | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - PALEVO - focus on palevo worm domains, daily updates, approx. 15 entries"
curl -s https://palevotracker.abuse.ch/blocklists.php?download=domainblocklist | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - RANSOMWARE - focus on ransomware domains, numerous updates on the same day, approx. 130 entries"
curl -s https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - SPAM404 - generic blocklist for suspicious domains, infrequent updates, approx. 5.000 entries"
curl -s https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - SYSCTL - generic blocklist for ad related domains, weekly updates, approx. 21.000 entries"
curl -s http://sysctl.org/cameleon/hosts | grep -v "#" | sed '/^$/d' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - WHOCARES - broad blocklist for suspicious domains, weekly updates, approx. 12.000 entries"
curl -s http://someonewhocares.org/hosts/hosts | grep -v "#" | sed '/^$/d' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - WINSPY - focus on windows spy & telemetry domains, infrequent updates, approx. 120 entries"
curl -s https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/win10/spy.txt | grep -v "#" | sed '/^$/d' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - WINHELP - broad blocklist for suspicious domains, infrequent updates, approx. 15.000 entries"
curl -s http://winhelp2002.mvps.org/hosts.txt | grep -v "#" | grep -v "localhost" | sed '/^$/d' | awk '{print $2}' | sort >> $tempoutlist

echo "Getting - YOYO - focus on ad related domains, weekly updates, approx. 2.500 entries"
curl -s 'https://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext' | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

echo "Getting - ZEUS - focus on zeus botnet domains, daily updates, approx. 440 entries"
curl -s 'https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist' | grep -v "#" | sed '/^$/d' | awk '{print $1}' | sort >> $tempoutlist

# Removing duplicates and formating list
cat $tempoutlist | sed $'s/\r$//' | sed '/thisisiafakedomain123\.com/d;/www\.anotherfakedomain123\.com/d' | sort -u | sed '/^$/d' | awk -v "IP=$destinationIP" '{sub(/\r$/,""); print IP" "$0}' > $unsortedoutlist

rm $tempoutlist

sort $unsortedoutlist | uniq -u > $outlist

rm $unsortedoutlist
​
echo "Reloading dnsmasq service"
/etc/init.d/dnsmasq restart

# Print statistics
numberOfAdsBlocked=$(cat $outlist | wc -l | sed 's/^[ \t]*//')
echo "$numberOfAdsBlocked ad domains blocked."

Make script executable

chmod +x /root/updateadb.sh

Run the script

sh /root/updateadb.sh

You should get output

Getting - ADAWAY - focus on mobile ads, infrequent updates, approx. 400 entries
Getting - DISCONNECT - mozilla driven content blocklist, numerous updates on the same day, approx. 6.500 entries
Getting - DSHIELD - broad blocklist for suspicious domains, daily updates, approx. 4.500 entries
Getting - MALWARE - broad blocklist for malware domains, daily updates, approx. 16.000 entries
Getting - MALWARELIST - generic blocklist for malware domains, daily updates, approx. 1.500 entries
Getting - OPENPHISH - focus on phishing domains, numerous updates on the same day, approx. 1.800 entries
Getting - PALEVO - focus on palevo worm domains, daily updates, approx. 15 entries
Getting - RANSOMWARE - focus on ransomware domains, numerous updates on the same day, approx. 130 entries
Getting - SPAM404 - generic blocklist for suspicious domains, infrequent updates, approx. 5.000 entries
Getting - SYSCTL - generic blocklist for ad related domains, weekly updates, approx. 21.000 entries
Getting - WHOCARES - broad blocklist for suspicious domains, weekly updates, approx. 12.000 entries
Getting - WINSPY - focus on windows spy & telemetry domains, infrequent updates, approx. 120 entries
Getting - WINHELP - broad blocklist for suspicious domains, infrequent updates, approx. 15.000 entries
Getting - YOYO - focus on ad related domains, weekly updates, approx. 2.500 entries
Getting - ZEUS - focus on zeus botnet domains, daily updates, approx. 440 entries
Reloading dnsmasq service
[ ok ] Restarting dnsmasq (via systemctl): dnsmasq.service.
57908 ad domains blocked.

Let's make a DNS query for my personal site vsefer.com

host -v -t A vsefer.com

Trying "vsefer.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58530
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;vsefer.com.			IN	A

;; ANSWER SECTION:
vsefer.com.		1256	IN	A	31.187.70.56

Received 44 bytes from 127.0.0.1#53 in 8 ms

Now let's try to make a DNS query for some malicious site (I chose 03.sharedsource.org from the list generated with script)

host -v -t A 03.sharedsource.org

Trying "03.sharedsource.org"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37749
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;03.sharedsource.org.           IN      ANY

;; ANSWER SECTION:
03.sharedsource.org.    0       IN      A       0.0.0.0

Received 53 bytes from 127.0.0.1#53 in 23 ms

Check log. 

tail -fn 100 /var/log/syslog | grep dnsmasq

...
Jul 30 22:30:49 osmc dnsmasq[1056]: dnsmasq: syntax check OK.
Jul 30 22:30:49 osmc dnsmasq[1066]: started, version 2.72 cachesize 10000
Jul 30 22:30:49 osmc dnsmasq[1066]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect
Jul 30 22:30:49 osmc dnsmasq[1066]: read /etc/hosts - 8 addresses
Jul 30 22:31:03 osmc dnsmasq[1066]: read /etc/hosts.adblock - 69683 addresses
Jul 30 22:31:03 osmc dnsmasq[1066]: using nameserver 8.8.8.8#53
Jul 30 22:31:03 osmc dnsmasq[1066]: using nameserver 8.8.4.4#53
...

...
Jul 30 22:38:34 osmc dnsmasq[1118]: query[A] vsefer.com from 127.0.0.1
Jul 30 22:38:34 osmc dnsmasq[1118]: forwarded vsefer.com to 8.8.8.8
Jul 30 22:38:34 osmc dnsmasq[1118]: forwarded vsefer.com to 8.8.4.4
Jul 30 22:38:34 osmc dnsmasq[1118]: reply vsefer.com is 31.187.70.56
...

...
Jul 30 22:38:42 osmc dnsmasq[1118]: query[A] 03.sharedsource.org from 127.0.0.1
Jul 30 22:38:42 osmc dnsmasq[1118]: /etc/hosts.adblock 03.sharedsource.org is 0.0.0.0
...

As you can see from the log, domain 03.sharedsource.org has received IP address 0.0.0.0. That's exactly what I wanted.

Don't forget to disable query logs (comment out the last line). 

vim /etc/dnsmasq.d/adb.conf

interface=eth0
no-poll
bogus-priv
neg-ttl=3600
cache-size=10000
dns-forward-max=1000
domain-needed
resolv-file=/etc/resolv.personal
addn-hosts=/etc/hosts.adblock
#log-queries

Restart dnsmasq service

/etc/init.d/dnsmasq restart

Configure DNS settings on your client device to use a RaspberryPI address. In my case I configured DNS settings directly on my router to use RaspberryPi IP address 10.10.10.172.

Let's make final test. Open http://ads-blocker.com/testing/ page on client device and there should not be ads.

 Screenshots from the mobile phone. First picture is with internet connection over GSM network (AD displayed), the second one is connected to a wireless connection secured with RaspberryPi (AD blocked).