Discussion:
[Dnsmasq-discuss] Slow DNSMasq with > 100, 000 entries in additional addresses file
TheWerthFam
2016-12-27 00:39:22 UTC
Permalink
Using DNSMasq 2.76 on the Openwrt platform with a raspberry pi 3 like device for home networking 1GB RAM, dual core 1 GHZ processor). Additionally I'm using the the adblock set of scripts found @ (https://github.com/openwrt/packages/tree/master/net/adblock/files) to block malware and porn sites. The porn sites list is about 800,000 entries, about 10x the number of sites adblock normally uses. With the full list of malware and porn domains loaded,
dnsmasq takes 115M of memory and normally sits around 50% CPU usage with moderate browsing usage. CPU and RAM usage isn't really a problem other
than lookups are slow now. Platform is cc 15.05.1 r49389.

The adblock script takes downloads different lists, creates a file for each list in the format:
local=/40def14.codns.com/ local=/944413269.3322.org
local=/domainnottogoto.com/ ... With one entry per line. The goal is to return NXDOMAIN when the local clients use the dnsmasq as the its primary dns server. Lists are sorted and with unique entries, so duplicates are limited.

In an effort to address the performance issues I've tried increasing the dnsmasq cachesize to 10,000 but that made no change in performance. Also tried neg-ttl=3600 with default negative caching enabled with no change - thinking that if dnsmasq would cache the no response queries performance would go up. This didn't have any noticeable improvement in performance.

Are there dnsmasq setting that will improve the performance? or should it
be configured differently to achieve this goal?
Perhaps unbound would be better suited?

Cheers
Derek
Yousong Zhou
2016-12-27 02:17:34 UTC
Permalink
Post by TheWerthFam
Using DNSMasq 2.76 on the Openwrt platform with a raspberry pi 3 like device
for home networking 1GB RAM, dual core 1 GHZ processor). Additionally I'm
(https://github.com/openwrt/packages/tree/master/net/adblock/files) to block
malware and porn sites. The porn sites list is about 800,000 entries, about
10x the number of sites adblock normally uses. With the full list of
malware and porn domains loaded,
dnsmasq takes 115M of memory and normally sits around 50% CPU usage with
moderate browsing usage. CPU and RAM usage isn't really a problem other
than lookups are slow now. Platform is cc 15.05.1 r49389.
The adblock script takes downloads different lists, creates a file for each
local=/40def14.codns.com/
local=/944413269.3322.org
local=/domainnottogoto.com/
...
With one entry per line. The goal is to return NXDOMAIN when the local
clients use the dnsmasq as the its primary dns server. Lists are sorted and
with unique entries, so duplicates are limited.
In an effort to address the performance issues I've tried increasing the
dnsmasq cachesize to 10,000 but that made no change in performance. Also
tried neg-ttl=3600 with default negative caching enabled with no change -
thinking that if dnsmasq would cache the no response queries performance
would go up. This didn't have any noticeable improvement in performance.
Are there dnsmasq setting that will improve the performance? or should it
be configured differently to achieve this goal?
Perhaps unbound would be better suited?
Cheers
Derek
There was a patch posted in 2015 trying to optimise this. It's worth a try.

[1] [PATCH v2] Optimize ipset/server/local/address/rebind-domain-ok
performance, http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2015q1/009257.html

yousong
Post by TheWerthFam
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
TheWerthFam
2016-12-27 02:51:23 UTC
Permalink
Yousong,
Do you know what version this was patched against?
Thanks
Post by Yousong Zhou
Post by TheWerthFam
Using DNSMasq 2.76 on the Openwrt platform with a raspberry pi 3 like device
for home networking 1GB RAM, dual core 1 GHZ processor). Additionally I'm
(https://github.com/openwrt/packages/tree/master/net/adblock/files) to block
malware and porn sites. The porn sites list is about 800,000 entries, about
10x the number of sites adblock normally uses. With the full list of
malware and porn domains loaded,
dnsmasq takes 115M of memory and normally sits around 50% CPU usage with
moderate browsing usage. CPU and RAM usage isn't really a problem other
than lookups are slow now. Platform is cc 15.05.1 r49389.
The adblock script takes downloads different lists, creates a file for each
local=/40def14.codns.com/
local=/944413269.3322.org
local=/domainnottogoto.com/
...
With one entry per line. The goal is to return NXDOMAIN when the local
clients use the dnsmasq as the its primary dns server. Lists are sorted and
with unique entries, so duplicates are limited.
In an effort to address the performance issues I've tried increasing the
dnsmasq cachesize to 10,000 but that made no change in performance. Also
tried neg-ttl=3600 with default negative caching enabled with no change -
thinking that if dnsmasq would cache the no response queries performance
would go up. This didn't have any noticeable improvement in performance.
Are there dnsmasq setting that will improve the performance? or should it
be configured differently to achieve this goal?
Perhaps unbound would be better suited?
Cheers
Derek
There was a patch posted in 2015 trying to optimise this. It's worth a try.
[1] [PATCH v2] Optimize ipset/server/local/address/rebind-domain-ok
performance, http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2015q1/009257.html
yousong
Post by TheWerthFam
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Chen Wei
2017-03-02 09:02:24 UTC
Permalink
Post by TheWerthFam
Yousong,
Do you know what version this was patched against?
I have updated the patch for 2.77test4.

https://github.com/infinet/dnsmasq/tree/fastlookup-v2.77test4
Post by TheWerthFam
Post by Yousong Zhou
Post by TheWerthFam
Using DNSMasq 2.76 on the Openwrt platform with a raspberry pi 3 like device
for home networking 1GB RAM, dual core 1 GHZ processor). Additionally I'm
(https://github.com/openwrt/packages/tree/master/net/adblock/files) to block
malware and porn sites. The porn sites list is about 800,000 entries, about
10x the number of sites adblock normally uses. With the full list of
malware and porn domains loaded,
dnsmasq takes 115M of memory and normally sits around 50% CPU usage with
moderate browsing usage. CPU and RAM usage isn't really a problem other
The adblock script takes downloads different lists, creates a file for each
local=/40def14.codns.com/
local=/944413269.3322.org
local=/domainnottogoto.com/
Are there dnsmasq setting that will improve the performance? or should it
be configured differently to achieve this goal?
Perhaps unbound would be better suited?
Cheers
Derek
There was a patch posted in 2015 trying to optimise this. It's worth a try.
[1] [PATCH v2] Optimize ipset/server/local/address/rebind-domain-ok
performance, http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2015q1/009257.html
yousong
--
Chen Wei
Loading...