Discussion:
[Dnsmasq-discuss] CPU spin in master
Kevin Darbyshire-Bryant
2016-01-01 11:28:38 UTC
Permalink
Hi Simon,

First off, Happy New Year!

I compiled master ec0628c4b2a06e1fc21216091bb040d61a43b271 on OpenWrt
(mips Archer C7 v2 platform Linux 4.1) a few hours ago and have
experienced dnsmasq going into a tight cpu loop. Running strace showed
no syscalls, so is spinning in dnsmasq somewhere. Kill -9 seemed to be
the only way out, and the behaviour would return an indeterminate time
after restart. Unfortunately I didn't have gdb installed on the router,
dnsmasq compiled with debug, nor any experience with gdb for that
matter, so it's a very limited amount of info I can offer.

I tried turning off dnssec usage in case that avoided the problem, which
it didn't. I'm not really sure what provokes the behaviour. However
I'd been running efef497b890231ba9232d02e7bfaf8273f044622 for a week
without incident, and have now backed out to
d3a8b39c7df2f0debf3b5f274a1c37a9e261f94e as of a few hours ago (avoiding
the arp caching) so far also without incident.

So this is a pretty vague report of something lurking in very recent code.

Cheers,

Kevin
Simon Kelley
2016-01-01 20:27:23 UTC
Permalink
Post by Kevin Darbyshire-Bryant
Hi Simon,
First off, Happy New Year!
.... and to you, and all our readers!
Post by Kevin Darbyshire-Bryant
I compiled master ec0628c4b2a06e1fc21216091bb040d61a43b271 on OpenWrt
(mips Archer C7 v2 platform Linux 4.1) a few hours ago and have
experienced dnsmasq going into a tight cpu loop. Running strace showed
no syscalls, so is spinning in dnsmasq somewhere. Kill -9 seemed to be
the only way out, and the behaviour would return an indeterminate time
after restart. Unfortunately I didn't have gdb installed on the router,
dnsmasq compiled with debug, nor any experience with gdb for that
matter, so it's a very limited amount of info I can offer.
I tried turning off dnssec usage in case that avoided the problem, which
it didn't. I'm not really sure what provokes the behaviour. However
I'd been running efef497b890231ba9232d02e7bfaf8273f044622 for a week
without incident, and have now backed out to
d3a8b39c7df2f0debf3b5f274a1c37a9e261f94e as of a few hours ago (avoiding
the arp caching) so far also without incident.
So this is a pretty vague report of something lurking in very recent code.#
It's pretty good really. I stared at the ARP-caching code and found a
fault in the linked list code that could introduce a cycle and create
exactly the symptoms you're seeing.


Git HEAD or 2.76test2 should do it. Please could you try it?


And many thanks for testing my new code!



Cheers,

Simon
Post by Kevin Darbyshire-Bryant
Cheers,
Kevin
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Kevin Darbyshire-Bryant
2016-01-02 16:42:36 UTC
Permalink
Post by Simon Kelley
Post by Kevin Darbyshire-Bryant
Hi Simon,
So this is a pretty vague report of something lurking in very recent code.#
It's pretty good really. I stared at the ARP-caching code and found a
fault in the linked list code that could introduce a cycle and create
exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the bleeding edge stuff
nothing would get spotted & fixed would it :-) Someone has to try and
I'd hardly regard my home router as life critical (although my niece
would have a different opinion on that if she were visiting)

Thanks,

Kevin
Kevin Darbyshire-Bryant
2016-01-03 10:42:10 UTC
Permalink
Router survived the night. No obvious problems noted :-)

--
Cheers,

Kevin
Sent from my phone, apologies for brevity, spelling & top posting
Post by Kevin Darbyshire-Bryant
Post by Simon Kelley
Post by Kevin Darbyshire-Bryant
Hi Simon,
So this is a pretty vague report of something lurking in very recent code.#
It's pretty good really. I stared at the ARP-caching code and found a
fault in the linked list code that could introduce a cycle and create
exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the bleeding edge stuff
nothing would get spotted & fixed would it :-) Someone has to try and
I'd hardly regard my home router as life critical (although my niece
would have a different opinion on that if she were visiting)
Thanks,
Kevin
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Simon Kelley
2016-01-04 16:14:48 UTC
Permalink
Think that one's fixed then :) Many thanks.


Simon.
Post by Kevin Darbyshire-Bryant
Router survived the night. No obvious problems noted :-)
-- Cheers,
Kevin Sent from my phone, apologies for brevity, spelling & top
posting
On 2 Jan 2016, at 17:20, Kevin Darbyshire-Bryant
Post by Simon Kelley
On 01/01/16 11:28, Kevin Darbyshire-Bryant wrote: Hi Simon,
So this is a pretty vague report of something lurking in very recent code.#
It's pretty good really. I stared at the ARP-caching code and
found a fault in the linked list code that could introduce a
cycle and create exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the bleeding edge
stuff nothing would get spotted & fixed would it :-) Someone has
to try and I'd hardly regard my home router as life critical
(although my niece would have a different opinion on that if she
were visiting)
Thanks,
Kevin
_______________________________________________ Dnsmasq-discuss
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________ Dnsmasq-discuss
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Simon Kelley
2016-01-04 17:19:24 UTC
Permalink
Please use test4, which fixes the problem (again!)

Cheers,

Simon.
Hi,
I'll try. Unfortunately I have to provoke the spinning somehow. I
just installed the test version, was happy, and a few minutes back
it was no longer responding. TOP showed 99% CPU.
By the way, box is a VIA C7 standard x86 box (32 bits), not MIPS
like Kevin's.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Simon Kelley
[Dnsmasq-discuss] CPU spin in master
Yes, the fix is in test3. Can you build with debug symbols
make CFLAGS=-g
and run under gdb, to find where it's spinning?
Cheers,
Simon
Hi,
ALARM: I compiled "2.76test3" and now it is spinning with
100% CPU on my box, box responds slow or not at all on DNS
query. Was the fix included in "test3"? I updated from 2.75
to 2.76test3 because of the previously mentioned wildcard
dnssec issue.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Dnsmasq-discuss
On Behalf Of Simon Kelley Sent: Monday, January 04, 2016
Subject: Re: [Dnsmasq-discuss] CPU spin in master
Think that one's fixed then :) Many thanks.
Simon.
Post by Kevin Darbyshire-Bryant
Router survived the night. No obvious problems noted
:-)
-- Cheers,
Kevin Sent from my phone, apologies for brevity,
spelling & top posting
On 2 Jan 2016, at 17:20, Kevin Darbyshire-Bryant
Post by Simon Kelley
Post by Kevin Darbyshire-Bryant
Hi Simon,
So this is a pretty vague report of something
lurking in very recent code.#
It's pretty good really. I stared at the
ARP-caching code and found a fault in the linked
list code that could introduce a cycle and create
exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could
you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the
bleeding edge stuff nothing would get spotted & fixed
would it :-) Someone has to try and I'd hardly regard
my home router as life critical (although my niece
would have a different opinion on that if she were
visiting)
Thanks,
Kevin
_______________________________________________
Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________ Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Uwe Schindler
2016-01-04 17:29:17 UTC
Permalink
Hi,

Grabbed, compiled, and installed it. I'll report back. It is now in use on my router, so I'll see if anything like this happens again.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
-----Original Message-----
Sent: Monday, January 04, 2016 6:19 PM
Subject: Re: [Dnsmasq-discuss] CPU spin in master
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Please use test4, which fixes the problem (again!)
Cheers,
Simon.
Hi,
I'll try. Unfortunately I have to provoke the spinning somehow. I
just installed the test version, was happy, and a few minutes back
it was no longer responding. TOP showed 99% CPU.
By the way, box is a VIA C7 standard x86 box (32 bits), not MIPS
like Kevin's.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Simon Kelley
[Dnsmasq-discuss] CPU spin in master
Yes, the fix is in test3. Can you build with debug symbols
make CFLAGS=-g
and run under gdb, to find where it's spinning?
Cheers,
Simon
Hi,
ALARM: I compiled "2.76test3" and now it is spinning with
100% CPU on my box, box responds slow or not at all on DNS
query. Was the fix included in "test3"? I updated from 2.75
to 2.76test3 because of the previously mentioned wildcard
dnssec issue.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Dnsmasq-discuss
On Behalf Of Simon Kelley Sent: Monday, January 04, 2016
Subject: Re: [Dnsmasq-discuss] CPU spin in master
Think that one's fixed then :) Many thanks.
Simon.
Post by Kevin Darbyshire-Bryant
Router survived the night. No obvious problems noted
:-)
-- Cheers,
Kevin Sent from my phone, apologies for brevity,
spelling & top posting
On 2 Jan 2016, at 17:20, Kevin Darbyshire-Bryant
Post by Simon Kelley
Post by Kevin Darbyshire-Bryant
Hi Simon,
So this is a pretty vague report of something
lurking in very recent code.#
It's pretty good really. I stared at the
ARP-caching code and found a fault in the linked
list code that could introduce a cycle and create
exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could
you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the
bleeding edge stuff nothing would get spotted & fixed
would it :-) Someone has to try and I'd hardly regard
my home router as life critical (although my niece
would have a different opinion on that if she were
visiting)
Thanks,
Kevin
_______________________________________________
Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________ Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAEBCAAGBQJWiqmcAAoJEBXN2mrhkTWimYMQAI/P0McloHDUNTkh5Nl
clb3y
nutfRAn9mIzuKZCxdEg+ZdbGyHuZZsJe/KXnXIh5Z12blcBsVQ1VJ64Y5Q62rUkg
vuxLVgvToXYfUzLJSwQqs7TJJ0jH0mydyD3rXkoCY+x76IcX0d4d47UqQYce+vs8
ppzqxIN2Wi1AuHYrpSCHU0FXUjqqs7fGL3l7RuV8Lsmw6Hz4sksNuwuG6KZkuc
3P
kmQJcTcXl9Ew0aXeR66MsS0rtN7UUJo0M/DYhijhgmx8MAnGYyA0c6PjNn7BL
SMq
hL1QasBPs2Urb9jYlBi+1LrBB6O5O6vLo/MlqEyQKtAsZUgPXORJeYDFVWuM6F
+N
R5F/Qd1vd+6pvxgCagN2TN9Zvizev29HpO6YSpZ8zGIxuFHpWh9dQQEKnWhe
E0lk
4SdQWLOYrH6ge3zJC6z74MxyNasB4Hmau2Ttwl7/Bmye2rz7vqOmgx4Gfep1Z
Jr5
kqIDPWn8EPpjFnTR/hdxoR0S9LTHVZpHTKERVHtpJHni9P4x2LqVuAmvysWdS
FDY
IP7fRRm3cVvxI+Y3a73f1uerBb/80k4fWAFg7tPTpZDD98FdduRT4zUqziYVrHuE
aKEF5iDN6QHlan3Uj/QhmAsDfjgAgDs5fQWnBa2M2QH/F8CIXefz31UcpKkr0y
B1
IFYcFLCIesL4yt1srpX2
=8frm
-----END PGP SIGNATURE-----
Simon Kelley
2016-01-04 17:10:23 UTC
Permalink
Apologies. Don't waste your time. I've managed to make git lose the
update somehow. Will get test4 fixed ASAP.


Cheers,

Simon.
Hi,
I'll try. Unfortunately I have to provoke the spinning somehow. I
just installed the test version, was happy, and a few minutes back
it was no longer responding. TOP showed 99% CPU.
By the way, box is a VIA C7 standard x86 box (32 bits), not MIPS
like Kevin's.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Simon Kelley
[Dnsmasq-discuss] CPU spin in master
Yes, the fix is in test3. Can you build with debug symbols
make CFLAGS=-g
and run under gdb, to find where it's spinning?
Cheers,
Simon
Hi,
ALARM: I compiled "2.76test3" and now it is spinning with
100% CPU on my box, box responds slow or not at all on DNS
query. Was the fix included in "test3"? I updated from 2.75
to 2.76test3 because of the previously mentioned wildcard
dnssec issue.
Uwe
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen
-----Original Message----- From: Dnsmasq-discuss
On Behalf Of Simon Kelley Sent: Monday, January 04, 2016
Subject: Re: [Dnsmasq-discuss] CPU spin in master
Think that one's fixed then :) Many thanks.
Simon.
Post by Kevin Darbyshire-Bryant
Router survived the night. No obvious problems noted
:-)
-- Cheers,
Kevin Sent from my phone, apologies for brevity,
spelling & top posting
On 2 Jan 2016, at 17:20, Kevin Darbyshire-Bryant
Post by Simon Kelley
Post by Kevin Darbyshire-Bryant
Hi Simon,
So this is a pretty vague report of something
lurking in very recent code.#
It's pretty good really. I stared at the
ARP-caching code and found a fault in the linked
list code that could introduce a cycle and create
exactly the symptoms you're seeing.
Git HEAD or 2.76test2 should do it. Please could
you try it?
It's compiling as I type - will report back :-)
Post by Simon Kelley
And many thanks for testing my new code!
Well if we all played it safe and avoided the
bleeding edge stuff nothing would get spotted & fixed
would it :-) Someone has to try and I'd hardly regard
my home router as life critical (although my niece
would have a different opinion on that if she were
visiting)
Thanks,
Kevin
_______________________________________________
Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________ Dnsmasq-
discuss
Post by Kevin Darbyshire-Bryant
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Loading...