Discussion:
[Dnsmasq-discuss] dnsmasq process in S, D state causing high load average
green krypton
2015-11-18 12:23:11 UTC
Permalink
I have around 500 dnsmasq process which are configured to give ipv6,ipv4
addresses out of this some fluctuate between D and S state while others are
in continuous S state.Probably because of too many processes in D state
with "rtnetlink_rcv" in wchan its showing very high load average.


functions in stack when in D state:

[<ffffffff816360a9>] rtnetlink_rcv+0x19/0x30
[<ffffffff81653e25>] netlink_unicast+0xd5/0x1b0
[<ffffffff8165420e>] netlink_sendmsg+0x30e/0x680
[<ffffffff8160e32b>] sock_sendmsg+0x8b/0xc0
[<ffffffff8160e871>] SYSC_sendto+0x121/0x1c0
[<ffffffff8160f25e>] SyS_sendto+0xe/0x10
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff

but in couple of seconds it changes to S state with this in stack,all idle
process have this in stack.

[<ffffffff811d20d9>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d2ac6>] do_select+0x5b6/0x780
[<ffffffff811d2e5c>] core_sys_select+0x1cc/0x2e0
[<ffffffff811d301b>] SyS_select+0xab/0x100
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff


interface in which dnsmasq is listening has " tentative dadfailed" in "scope
global" error for ipv6 .

Not able to understand why dnsmasq that are connected to interfaces that
have "tentative dadfailed " continuously changes between S and D state
while others are idle.

attached is the strace while dnsmasq is in D state.

Any idea about this issue?
Simon Kelley
2015-11-20 21:35:42 UTC
Permalink
Post by green krypton
I have around 500 dnsmasq process which are configured to give ipv6,ipv4
addresses out of this some fluctuate between D and S state while others are
in continuous S state.Probably because of too many processes in D state
with "rtnetlink_rcv" in wchan its showing very high load average.
[<ffffffff816360a9>] rtnetlink_rcv+0x19/0x30
[<ffffffff81653e25>] netlink_unicast+0xd5/0x1b0
[<ffffffff8165420e>] netlink_sendmsg+0x30e/0x680
[<ffffffff8160e32b>] sock_sendmsg+0x8b/0xc0
[<ffffffff8160e871>] SYSC_sendto+0x121/0x1c0
[<ffffffff8160f25e>] SyS_sendto+0xe/0x10
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
but in couple of seconds it changes to S state with this in stack,all idle
process have this in stack.
[<ffffffff811d20d9>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d2ac6>] do_select+0x5b6/0x780
[<ffffffff811d2e5c>] core_sys_select+0x1cc/0x2e0
[<ffffffff811d301b>] SyS_select+0xab/0x100
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
interface in which dnsmasq is listening has " tentative dadfailed" in "scope
global" error for ipv6 .
Not able to understand why dnsmasq that are connected to interfaces that
have "tentative dadfailed " continuously changes between S and D state
while others are idle.
attached is the strace while dnsmasq is in D state.
Any idea about this issue?
Dnsmasq polls every second whilst one or more interfaces it's listening
on are still doing DAD. Once all interfaces have completed DAD,it stops
polling.


Are you really running 500 separate instances on dnsmasq? That's an
installation which is way out of the normal way of doing things.


Cheers,

Simon.
Post by green krypton
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
green krypton
2016-02-17 12:53:14 UTC
Permalink
500 instances in 500 vlans
Post by green krypton
Post by green krypton
I have around 500 dnsmasq process which are configured to give ipv6,ipv4
addresses out of this some fluctuate between D and S state while others
are
Post by green krypton
in continuous S state.Probably because of too many processes in D state
with "rtnetlink_rcv" in wchan its showing very high load average.
[<ffffffff816360a9>] rtnetlink_rcv+0x19/0x30
[<ffffffff81653e25>] netlink_unicast+0xd5/0x1b0
[<ffffffff8165420e>] netlink_sendmsg+0x30e/0x680
[<ffffffff8160e32b>] sock_sendmsg+0x8b/0xc0
[<ffffffff8160e871>] SYSC_sendto+0x121/0x1c0
[<ffffffff8160f25e>] SyS_sendto+0xe/0x10
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
but in couple of seconds it changes to S state with this in stack,all
idle
Post by green krypton
process have this in stack.
[<ffffffff811d20d9>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d2ac6>] do_select+0x5b6/0x780
[<ffffffff811d2e5c>] core_sys_select+0x1cc/0x2e0
[<ffffffff811d301b>] SyS_select+0xab/0x100
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
interface in which dnsmasq is listening has " tentative dadfailed" in
"scope
Post by green krypton
global" error for ipv6 .
Not able to understand why dnsmasq that are connected to interfaces that
have "tentative dadfailed " continuously changes between S and D state
while others are idle.
attached is the strace while dnsmasq is in D state.
Any idea about this issue?
Dnsmasq polls every second whilst one or more interfaces it's listening
on are still doing DAD. Once all interfaces have completed DAD,it stops
polling.
Are you really running 500 separate instances on dnsmasq? That's an
installation which is way out of the normal way of doing things.
Cheers,
Simon.
Post by green krypton
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Simon Kelley
2016-02-17 16:25:48 UTC
Permalink
Best I can suggest is to fix the cause of some interfaces failing the
DAD test. Failing that, patch dnsmasq to avoid the polling looking for
DAD to succeed.

Cheers,

Simon
Post by green krypton
500 instances in 500 vlans
Post by green krypton
Post by green krypton
I have around 500 dnsmasq process which are configured to give ipv6,ipv4
addresses out of this some fluctuate between D and S state while others
are
Post by green krypton
in continuous S state.Probably because of too many processes in D state
with "rtnetlink_rcv" in wchan its showing very high load average.
[<ffffffff816360a9>] rtnetlink_rcv+0x19/0x30
[<ffffffff81653e25>] netlink_unicast+0xd5/0x1b0
[<ffffffff8165420e>] netlink_sendmsg+0x30e/0x680
[<ffffffff8160e32b>] sock_sendmsg+0x8b/0xc0
[<ffffffff8160e871>] SYSC_sendto+0x121/0x1c0
[<ffffffff8160f25e>] SyS_sendto+0xe/0x10
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
but in couple of seconds it changes to S state with this in stack,all
idle
Post by green krypton
process have this in stack.
[<ffffffff811d20d9>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d2ac6>] do_select+0x5b6/0x780
[<ffffffff811d2e5c>] core_sys_select+0x1cc/0x2e0
[<ffffffff811d301b>] SyS_select+0xab/0x100
[<ffffffff817342dd>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
interface in which dnsmasq is listening has " tentative dadfailed" in
"scope
Post by green krypton
global" error for ipv6 .
Not able to understand why dnsmasq that are connected to interfaces that
have "tentative dadfailed " continuously changes between S and D state
while others are idle.
attached is the strace while dnsmasq is in D state.
Any idea about this issue?
Dnsmasq polls every second whilst one or more interfaces it's listening
on are still doing DAD. Once all interfaces have completed DAD,it stops
polling.
Are you really running 500 separate instances on dnsmasq? That's an
installation which is way out of the normal way of doing things.
Cheers,
Simon.
Post by green krypton
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Loading...