[Dnsmasq-discuss] No DHCP leases handed on bridge interface

[Sébastien Delafond]

Hello,

I have a server with a bridge containing only one interface (ideally
it'll of course include more interfaces, but I've tried to eliminate as
many factors as possible):

$ brctl show
bridge name bridge id STP enabled interfaces
br.eth0-2 8000.1cb72c761568 no eth0.2
$ ip ad show br.eth0-2
15: br.eth0-2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
\ noqueue state UP group default qlen 1000
link/ether 1c:b7:2c:76:15:68 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/16 brd 192.168.255.255 scope global br.eth0-2
valid_lft forever preferred_lft forever

The DHCP-relevant parts of dnsmasq.conf are:

interface=*
dhcp-authoritative
dhcp-lease-max=5000
dhcp-range=set:br.eth0-2,192.168.1.100,192.168.1.200,86400
dhcp-option=tag:br.eth0-2,3,192.168.1.1 # gateway
dhcp-option=tag:br.eth0-2,1,255.255.0.0 # netmask
dhcp-option=tag:br.eth0-2,6,192.168.1.1 # dns

My client, sitting on the other end of eth0.2, is issuing DHCP requests
that correctly make it up to the server (tcpdump sees them on both
eth0.2 and br.eth0-2). However, dnsmasq is acting like it doesn't
receive them at all: it stays silent, even when run with "-d
--log-dhcp". In strace, I don't see anything either after it's started.

The most interesting point is probably that ISC-dhcpd serves DHCP leases
just fine in the same exact setup: simply stopping dnsmasq and starting
ISC-dhcpd with a minimal "subnet 192.168.0.0 netmask 255.255.0.0 { range
192.168.1.100 192.168.1.200;}" configuration results in leases being
handed to the same client.

Also, if I remove the bridge, assign 192.168.1.1/16 to eth0.2, and
restart dnsmasq, then my client gets a DHCP lease.

Another thing I've done is to replay all the aforementioned tests with a
wlan0 interface instead of the eth0.2, with the same results: ISC dhcpd
works fine, dnsmasq doesn't serve DHCP over the bridge containing only
wlan0, but does if I kill the bridge and work with wlan0 alone.

Any clue what's going on ? This is a 4.4.3 kernel, and I've tried both
2.62 from Debian jessie, and 2.76 from stretch.

Cheers,

--Seb

[Albert ARIBAUD]

Hi Seb,

Le Fri, 15 Jul 2016 10:28:33 +0000 (UTC)

Post by Sébastien Delafond
Hello,
I have a server with a bridge containing only one interface (ideally
it'll of course include more interfaces, but I've tried to eliminate
$ brctl show
bridge name bridge id STP enabled interfaces
br.eth0-2 8000.1cb72c761568 no
eth0.2 $ ip ad show br.eth0-2
15: br.eth0-2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
\ noqueue state UP group default qlen 1000
link/ether 1c:b7:2c:76:15:68 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/16 brd 192.168.255.255 scope global br.eth0-2
valid_lft forever preferred_lft forever
interface=*
dhcp-authoritative
dhcp-lease-max=5000
dhcp-range=set:br.eth0-2,192.168.1.100,192.168.1.200,86400
dhcp-option=tag:br.eth0-2,3,192.168.1.1 # gateway
dhcp-option=tag:br.eth0-2,1,255.255.0.0 # netmask
dhcp-option=tag:br.eth0-2,6,192.168.1.1 # dns
My client, sitting on the other end of eth0.2, is issuing DHCP
requests that correctly make it up to the server (tcpdump sees them
on both eth0.2 and br.eth0-2). However, dnsmasq is acting like it
doesn't receive them at all: it stays silent, even when run with "-d
--log-dhcp". In strace, I don't see anything either after it's
started.
The most interesting point is probably that ISC-dhcpd serves DHCP
leases just fine in the same exact setup: simply stopping dnsmasq and
starting ISC-dhcpd with a minimal "subnet 192.168.0.0 netmask
255.255.0.0 { range 192.168.1.100 192.168.1.200;}" configuration
results in leases being handed to the same client.
Also, if I remove the bridge, assign 192.168.1.1/16 to eth0.2, and
restart dnsmasq, then my client gets a DHCP lease.
Another thing I've done is to replay all the aforementioned tests
ISC dhcpd works fine, dnsmasq doesn't serve DHCP over the bridge
containing only wlan0, but does if I kill the bridge and work with
wlan0 alone.
Any clue what's going on ? This is a 4.4.3 kernel, and I've tried both
2.62 from Debian jessie, and 2.76 from stretch.

Just to make sure: "eth0.2" would normally denote a virtual interface
for VLAN 2 traffic on eth0. Are VLANs involved in your network setup?

Post by Sébastien Delafond
Cheers,
--Seb

Amicalement,

[Albert ARIBAUD]

Bonjour,

Le Fri, 15 Jul 2016 16:39:37 +0200

Post by Albert ARIBAUD
Just to make sure: "eth0.2" would normally denote a virtual
interface for VLAN 2 traffic on eth0. Are VLANs involved in your
network setup?

Hi Albert,
the hardware is an ARM board using the roboswitch module: that
basically uses VLANs internally, in order to let users dynamically
create logical groups from the 9 physical ethernet ports available on
the board, by writing to /proc/switch/eth0/*. Does that help ?

That leads to further questions, at least. VLANs do not cause problems
for dnsmasq, but bridging VLANs may be tricky and more prone to network
misconfigurations.

So just to make sure, can you list all interfaces and all bridges in
your system, including ifconfig outputs?

FYI and assuming you can read French :) here is an overview of an
issue I had with bridged VLANs: <http://albert.aribaud.net/fr/node/11>.

The Englih TL;DR of it is: I had eth0 and eth1 bridged (as "br0") and
needed VLAN 100 traffic to pass across transparently. Not only did I
need to also bridge eth0.100 and eth1.100 (as "br100"), but I also
had to use iptables to make br0 reject VLAN 100 packets, otherwise they
were dropped instead of being processed and passed across by br100.

Again: no reason why your problem might be the one I had; I'm just
giving an example of how mixing bridges and VLANs can be tricky.

Cheers,
--Seb

Amicalement,

[Sébastien Delafond]

Post by Albert ARIBAUD
That leads to further questions, at least. VLANs do not cause
problems for dnsmasq, but bridging VLANs may be tricky and more
prone to network misconfigurations.
So just to make sure, can you list all interfaces and all bridges in
your system, including ifconfig outputs?

Sure:

$ brctl show
bridge name bridge id STP enabled interfaces
br.eth0-2 8000.1cb72c761568 no eth0.2
$ ifconfig
br.eth0-2 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
inet addr:192.168.1.1 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:872 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:130641 (127.5 KiB) TX bytes:258 (258.0 B)

eth0 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:140624 errors:0 dropped:0 overruns:0 frame:0
TX packets:103007 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:91383944 (87.1 MiB) TX bytes:74433059 (70.9 MiB)
Interrupt:180 Base address:0x5000

eth0.1 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
inet addr:172.16.25.234 Bcast:172.16.25.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:748 errors:0 dropped:0 overruns:0 frame:0
TX packets:355 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:51266 (50.0 KiB) TX bytes:41482 (40.5 KiB)

eth0.2 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1312 (1.2 KiB) TX bytes:258 (258.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:285419 errors:0 dropped:0 overruns:0 frame:0
TX packets:285419 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:77987874 (74.3 MiB) TX bytes:77987874 (74.3 MiB)

So, only one bridge (that's not bridging much right now since it only
contains eth0.2).

Post by Albert ARIBAUD
FYI and assuming you can read French :)

I most definitely can, heh :)

Post by Albert ARIBAUD
<http://albert.aribaud.net/fr/node/11>.
The Englih TL;DR of it is: I had eth0 and eth1 bridged (as "br0") and
needed VLAN 100 traffic to pass across transparently. Not only did I
need to also bridge eth0.100 and eth1.100 (as "br100"), but I also had
to use iptables to make br0 reject VLAN 100 packets, otherwise they
were dropped instead of being processed and passed across by br100.
Again: no reason why your problem might be the one I had; I'm just
giving an example of how mixing bridges and VLANs can be tricky.

Interesting issue, but it doesn't feel like what I'm facing right now:
the fact that ISC dhcpd serves leases OK seems to indicate that the
network configuration itself is all right.

Plus, there really is not VLAN at all outside of that
software-programmable NIC anyway: tcpdump -e confirms that packets
going in and out (either on br0.eth-2 or eth0.2) are not tagged. I've
tested that by assigning a manual IP to my client and initiating a
telnet connection to the outside.

Does this make any sort of sense ? :)

Cheers,

--Seb

[Albert ARIBAUD]

Bonjour Seb,

Le Fri, 15 Jul 2016 15:48:55 +0000 (UTC)

Post by Sébastien Delafond

Post by Albert ARIBAUD
That leads to further questions, at least. VLANs do not cause
problems for dnsmasq, but bridging VLANs may be tricky and more
prone to network misconfigurations.
So just to make sure, can you list all interfaces and all bridges in
your system, including ifconfig outputs?

$ brctl show
bridge name bridge id STP enabled interfaces
br.eth0-2 8000.1cb72c761568 no
eth0.2 $ ifconfig
br.eth0-2 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
inet addr:192.168.1.1 Bcast:192.168.255.255
Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:872 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:130641 (127.5 KiB) TX bytes:258 (258.0 B)
eth0 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:140624 errors:0 dropped:0 overruns:0 frame:0
TX packets:103007 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:91383944 (87.1 MiB) TX bytes:74433059 (70.9 MiB)
Interrupt:180 Base address:0x5000
eth0.1 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
inet addr:172.16.25.234 Bcast:172.16.25.255
Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:748 errors:0 dropped:0 overruns:0 frame:0
TX packets:355 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:51266 (50.0 KiB) TX bytes:41482 (40.5 KiB)
eth0.2 Link encap:Ethernet HWaddr 1c:b7:2c:76:15:68
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1312 (1.2 KiB) TX bytes:258 (258.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:285419 errors:0 dropped:0 overruns:0 frame:0
TX packets:285419 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:77987874 (74.3 MiB) TX bytes:77987874 (74.3 MiB)
So, only one bridge (that's not bridging much right now since it only
contains eth0.2).

... and no weirdness in the address/mask settings that I can see.

[...]

Post by Sébastien Delafond
the fact that ISC dhcpd serves leases OK seems to indicate that the
network configuration itself is all right.

OTOH, there *is* a link with the networking setup since dnsmasq works
without the bridge and stops working with the bridge. Also, the fact
tha ISC dhcp works might hinge on how ISC dhcp listens to requests
exactly vs. how dnsmasq does. What's the ISC dhcp config exactly?

Post by Sébastien Delafond
Plus, there really is not VLAN at all outside of that
software-programmable NIC anyway: tcpdump -e confirms that packets
going in and out (either on br0.eth-2 or eth0.2) are not tagged. I've
tested that by assigning a manual IP to my client and initiating a
telnet connection to the outside.
Does this make any sort of sense ? :)

Well, it certainly does not give me a definite clue as to what is wrong
with your dnsmasq.

Speaking of tcpdump, can you use it to capture a successful request to
ISC dhcp and a (failed) request to dnsmasq, on eth0.2 as well as on
br.eth0-2, and make the captures files available somewhere?

Post by Sébastien Delafond
Cheers,
--Seb

Amicalement,

[Sébastien Delafond]

Post by Albert ARIBAUD
OTOH, there *is* a link with the networking setup since dnsmasq
works without the bridge and stops working with the bridge.

True.

Post by Albert ARIBAUD
Also, the fact tha ISC dhcp works might hinge on how ISC dhcp
listens to requests exactly vs. how dnsmasq does. What's the ISC
dhcp config exactly?

The shorted functional one is:

subnet 192.168.0.0 netmask 255.255.0.0 {
range 192.168.1.100 192.168.1.200;
}

Post by Albert ARIBAUD
Speaking of tcpdump, can you use it to capture a successful request
to ISC dhcp and a (failed) request to dnsmasq, on eth0.2 as well as
on br.eth0-2, and make the captures files available somewhere?

They're now at https://people.debian.org/~seb/dnsmasq-pcaps

Cheers,

--Seb

[Albert ARIBAUD]

Bonsoir Seb,

Le Fri, 15 Jul 2016 17:09:02 +0000 (UTC)

Post by Sébastien Delafond

Post by Albert ARIBAUD
OTOH, there *is* a link with the networking setup since dnsmasq
works without the bridge and stops working with the bridge.

True.

Post by Albert ARIBAUD
Also, the fact tha ISC dhcp works might hinge on how ISC dhcp
listens to requests exactly vs. how dnsmasq does. What's the ISC
dhcp config exactly?

subnet 192.168.0.0 netmask 255.255.0.0 {
range 192.168.1.100 192.168.1.200;
}

No mention of the interfaces it binds to and how? No functional
equivalent to the interface-related options of dnsmasq?

Post by Sébastien Delafond

Post by Albert ARIBAUD
Speaking of tcpdump, can you use it to capture a successful request
to ISC dhcp and a (failed) request to dnsmasq, on eth0.2 as well as
on br.eth0-2, and make the captures files available somewhere?

They're now at https://people.debian.org/~seb/dnsmasq-pcaps

Thanks. The captures are identical except for obvious differences like
transaction numbers or checksums, which have no influence on acceptance
or rejection of a DHCP request.

Which seems to imply that dnsmasq makes the difference based on the
interface it receives the request on -- hence my asking how ISC dhcp
chooses the interfaces it listens to... and how dnsmasq does it.

You might want to check whether the bridge is brought up before or
after dnsmasq is started. Also, try combinations of interface= and
bind-interfaces. Also, checkout bridge-interface= if it is available in
your version of dnsmasq.

Post by Sébastien Delafond
Cheers,
--Seb

Amicalement,

[Sébastien Delafond]

Post by Albert ARIBAUD
No mention of the interfaces it binds to and how? No functional
equivalent to the interface-related options of dnsmasq?

You can pass interfaces to bind to on the command-line, but it wasn't
necessary in my case.

Post by Albert ARIBAUD
Which seems to imply that dnsmasq makes the difference based on the
interface it receives the request on -- hence my asking how ISC dhcp
chooses the interfaces it listens to... and how dnsmasq does it.
You might want to check whether the bridge is brought up before or
after dnsmasq is started.

It's up before I start dnsmasq.

Post by Albert ARIBAUD
Also, try combinations of interface= and bind-interfaces.

I've tried with "interface=br.eth0-2 bind-interfaces" but the behavior
stays the same.

Post by Albert ARIBAUD
Also, checkout bridge-interface= if it is available in your version of
dnsmasq.

From the man page, I had assumed bridge-interface would play a role only
if the bridge interface wasn't assigned an IP.

With "interface=eth0.2 bind-interfaces bridge-interface=eth0.2,br.eth0-2",
dnsmasq refuses to stop with "unknown interface eth0.2", which seems to
be a generic message saying that the interface doesn't have an IP.

With "interface=br.eth0-2 bind-interfaces bridge-interface=eth0.2,br.eth0-2",
dnsmasq clearly states "dnsmasq-dhcp: DHCP, sockets bound exclusively to
interface br.eth0-2 " when starting, and I don't get DHCP leases either.

With "interface=* bind-interfaces bridge-interface=eth0.2,br.eth0-2", no
obvious changes.

Not passing bind-interfaces doesn't seem to affect the result of those
tests.

Cheers,

--Seb

[Albert ARIBAUD]

Bonjour,

Le Sat, 16 Jul 2016 07:15:55 +0000 (UTC)

Post by Sébastien Delafond

Post by Albert ARIBAUD
No mention of the interfaces it binds to and how? No functional
equivalent to the interface-related options of dnsmasq?

You can pass interfaces to bind to on the command-line, but it wasn't
necessary in my case.

Post by Albert ARIBAUD
Which seems to imply that dnsmasq makes the difference based on the
interface it receives the request on -- hence my asking how ISC dhcp
chooses the interfaces it listens to... and how dnsmasq does it.
You might want to check whether the bridge is brought up before or
after dnsmasq is started.

It's up before I start dnsmasq.

OK.

Post by Sébastien Delafond

Post by Albert ARIBAUD
Also, try combinations of interface= and bind-interfaces.

I've tried with "interface=br.eth0-2 bind-interfaces" but the behavior
stays the same.

Post by Albert ARIBAUD
Also, checkout bridge-interface= if it is available in your version
of dnsmasq.

From the man page, I had assumed bridge-interface would play a role
only if the bridge interface wasn't assigned an IP.

Indeed, but it was still worth trying, in case the manpage and code did
not agree.

Post by Sébastien Delafond
With "interface=eth0.2 bind-interfaces
bridge-interface=eth0.2,br.eth0-2", dnsmasq refuses to stop with
"unknown interface eth0.2", which seems to be a generic message
saying that the interface doesn't have an IP.
With "interface=br.eth0-2 bind-interfaces
bridge-interface=eth0.2,br.eth0-2", dnsmasq clearly states
"dnsmasq-dhcp: DHCP, sockets bound exclusively to interface br.eth0-2
" when starting, and I don't get DHCP leases either.
With "interface=* bind-interfaces bridge-interface=eth0.2,br.eth0-2",
no obvious changes.
Not passing bind-interfaces doesn't seem to affect the result of those
tests.

Alright... I'm out of ideas, sorry -- apart from recompiling dnsmasq
with ad hoc debug code. :/

Post by Sébastien Delafond
Cheers,
--Seb

Amicalement,

[Simon Kelley]

Post by Albert ARIBAUD
Alright... I'm out of ideas, sorry -- apart from recompiling dnsmasq
with ad hoc debug code. :/

I can't see anything obvious from what's been posted so far, but it
might be worth pointing out that this arrangement is used in millions of
openWRT routers and has been for years. Ie my openWRT router looks like


***@melaleuca:~# brctl show
bridge name bridge id STP enabled interfaces
br-lan 7fff.e4956e4061cc no eth1
wlan0

***@melaleuca:~# ifconfig
br-lan Link encap:Ethernet HWaddr E4:95:6E:40:61:CC
inet addr:192.168.150.1 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fd66:67b7:4126::1/60 Scope:Global
inet6 addr: fe80::e695:6eff:fe40:61cc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3009106 errors:0 dropped:0 overruns:0 frame:0
TX packets:3571295 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:390966486 (372.8 MiB) TX bytes:3069836286 (2.8 GiB)

.
.
eth1 Link encap:Ethernet HWaddr E4:95:6E:40:61:CC
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:5

.
.
wlan0 Link encap:Ethernet HWaddr E4:95:6E:40:61:CC
inet6 addr: fe80::e695:6eff:fe40:61cc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3029316 errors:0 dropped:0 overruns:0 frame:0
TX packets:3576669 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:436550756 (416.3 MiB) TX bytes:3129854122 (2.9 GiB)


And has a dhcp-range line in the config

dhcp-range=lan,192.168.150.100,192.168.150.249,255.255.255.0,12h


and it just works.


Note that ISC dhcpd uses a very low level code to access packets,
bypassing the whole of the kernel network stack and iptables. It
essentially does the same as tcpdump or wireshark. Dnsmasq doesn't, it
uses the kernel network stack. That may be a clue as to why ISC works
but dnsmasq doesn't, especially the iptables bit. The usual problem is
dumping packets with destination 255.255.255.255 or packets with source
0.0.0.0


Cheers,

Simon.

[Sébastien Delafond]

Post by Simon Kelley
Note that ISC dhcpd uses a very low level code to access packets,
bypassing the whole of the kernel network stack and iptables. It
essentially does the same as tcpdump or wireshark. Dnsmasq doesn't,
it uses the kernel network stack. That may be a clue as to why ISC
works but dnsmasq doesn't, especially the iptables bit. The usual
problem is dumping packets with destination 255.255.255.255 or
packets with source 0.0.0.0

Thanks Simon and Albert, you guys were a big help: I didn't know ISC
used pcap, so I did not once think about iptables. As soon as I
flushed my rules, dnsmasq worked like a charm. Sorry for wasting your
time, and thanks again.

Cheers,

--Seb

[Sébastien Delafond]

Post by Albert ARIBAUD
Just to make sure: "eth0.2" would normally denote a virtual interface
for VLAN 2 traffic on eth0. Are VLANs involved in your network setup?

Hi Albert,

the hardware is an ARM board using the roboswitch module: that basically
uses VLANs internally, in order to let users dynamically create logical
groups from the 9 physical ethernet ports available on the board, by
writing to /proc/switch/eth0/*. Does that help ?

Cheers,

--Seb