[Dnsmasq-discuss] RFC: allowing arbitrarily long options

I'm in favour, in theory at least, of removing arbitrary limits.
Experience has shown that no matter how big, someone, somewhere, will
always find them. The code originally used a fixed buffer which happened
to be unused at that point, to reduce the memory footprint. Whilst
dnsmasq is still intended to be "small", small is a relative thing, and
absolutely, rather bigger than it was 15 years ago, so allocating a big
enough buffer is fine.

In this case, though, as you hint, it's likely that shell limits will
also be a problem. Even eliminating that by using configuration files,
you still have very long lines, which is ugly.

Can't we solve this problem by allowing repeated interface names, so

--bridge-interface=eth1,alias-1,alias-2

becomes identical to

--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2

the patch to implement that is probably smaller than your offering.

Maybe we should do both?

Cheers,

Simon.

Post by Neil Jerram
Calico [1] with OpenStack
(https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses
--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,
where each occurrence of ",<alias-if-name>" occupies 15 characters, and
there can in principle be as many <alias-if-name>s as you can have VMs
on a single OpenStack compute host.Â Currently an option arg is limited
in dnsmasq to 1025 chars overall, which only allows for 67
<alias-if-name>s, which is not necessarily enough, on a powerful compute
host.
So I wonder what folk would think about reallocating as necessary to
allow an option arg to be arbitrarily long?Â (Or at least, as long as
getopt and the containing shell will allow.)Â For reference I've
attached a patch that I think would implement that - but I haven't yet
been able to test it at all, so please don't merge it yet!
Thanks in advance for your thoughts!
Â Â Â Neil
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Neil Jerram

2018-01-14 18:14:34 UTC

Thanks for looking at this, Simon. Some thoughts below...

Post by Simon Kelley
I'm in favour, in theory at least, of removing arbitrary limits.
Experience has shown that no matter how big, someone, somewhere, will
always find them. The code originally used a fixed buffer which happened
to be unused at that point, to reduce the memory footprint. Whilst
dnsmasq is still intended to be "small", small is a relative thing, and
absolutely, rather bigger than it was 15 years ago, so allocating a big
enough buffer is fine.
In this case, though, as you hint, it's likely that shell limits will
also be a problem. Even eliminating that by using configuration files,
you still have very long lines, which is ugly.
Can't we solve this problem by allowing repeated interface names, so
--bridge-interface=eth1,alias-1,alias-2
becomes identical to
--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2
the patch to implement that is probably smaller than your offering.

It looks like a nice alternative, but note that it doesn't help at all with
any shell limit. (In fact it would hit any such limit sooner.) So I think
this is a matter of what you think is more elegant for dnsmasq itself;
particularly in the configuration file context where shell limits don't
apply.

Post by Simon Kelley
Maybe we should do both?

In principle I'm happy to code up and test multiple solutions here; it's
not a large amount of work in any case. So please do let me know what you
would prefer.

Best wishes - Neil

Post by Neil Jerram
Calico [1] with OpenStack
(https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses
--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,
where each occurrence of ",<alias-if-name>" occupies 15 characters, and
there can in principle be as many <alias-if-name>s as you can have VMs
on a single OpenStack compute host. Currently an option arg is limited
in dnsmasq to 1025 chars overall, which only allows for 67
<alias-if-name>s, which is not necessarily enough, on a powerful compute
host.
So I wonder what folk would think about reallocating as necessary to
allow an option arg to be arbitrarily long? (Or at least, as long as
getopt and the containing shell will allow.) For reference I've
attached a patch that I think would implement that - but I haven't yet
been able to test it at all, so please don't merge it yet!
Thanks in advance for your thoughts!
Neil
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Simon Kelley

2018-01-14 22:35:04 UTC

Thanks for looking at this, Simon. Some thoughts below...

I'm in favour, in theory at least, of removing arbitrary limits.
Experience has shown that no matter how big, someone, somewhere, will
always find them. The code originally used a fixed buffer which happened
to be unused at that point, to reduce the memory footprint. Whilst
dnsmasq is still intended to be "small", small is a relative thing, and
absolutely, rather bigger than it was 15 years ago, so allocating a big
enough buffer is fine.
In this case, though, as you hint, it's likely that shell limits will
also be a problem. Even eliminating that by using configuration files,
you still have very long lines, which is ugly.
Can't we solve this problem by allowing repeated interface names, so
--bridge-interface=eth1,alias-1,alias-2
becomes identical to
--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2
the patch to implement that is probably smaller than your offering.
It looks like a nice alternative, but note that it doesn't help at all
with any shell limit. (In fact it would hit any such limit sooner.) So
I think this is a matter of what you think is more elegant for dnsmasq
itself; particularly in the configuration file context where shell
limits don't apply.

Maybe we should do both?
In principle I'm happy to code up and test multiple solutions here; it's
not a large amount of work in any case. So please do let me know what
you would prefer.

I'll take both, for preference.

Actually - no. You test the long-options patch and I'll take that. I'll
do the split-into-multiple lines one.

Cheers,

Simon.

Best wishes - Neil

Post by Neil Jerram
Calico [1] with OpenStack
(https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses

--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,

Post by Neil Jerram
where each occurrence of ",<alias-if-name>" occupies 15

characters, and

Post by Neil Jerram
there can in principle be as many <alias-if-name>s as you can have VMs
on a single OpenStack compute host. Currently an option arg is

limited

Post by Neil Jerram
in dnsmasq to 1025 chars overall, which only allows for 67
<alias-if-name>s, which is not necessarily enough, on a powerful

compute

Post by Neil Jerram
host.
So I wonder what folk would think about reallocating as necessary to
allow an option arg to be arbitrarily long? (Or at least, as long as
getopt and the containing shell will allow.) For reference I've
attached a patch that I think would implement that - but I haven't yet
been able to test it at all, so please don't merge it yet!
Thanks in advance for your thoughts!
Neil
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Neil Jerram

2018-01-18 14:15:32 UTC

Post by Neil Jerram
Thanks for looking at this, Simon. Some thoughts below...
I'm in favour, in theory at least, of removing arbitrary limits.
Experience has shown that no matter how big, someone, somewhere, will
always find them. The code originally used a fixed buffer which

happened

Post by Neil Jerram
to be unused at that point, to reduce the memory footprint. Whilst
dnsmasq is still intended to be "small", small is a relative thing,

and

Post by Neil Jerram
absolutely, rather bigger than it was 15 years ago, so allocating a

big

Post by Neil Jerram
enough buffer is fine.
In this case, though, as you hint, it's likely that shell limits will
also be a problem. Even eliminating that by using configuration

files,

Post by Neil Jerram
you still have very long lines, which is ugly.
Can't we solve this problem by allowing repeated interface names, so
--bridge-interface=eth1,alias-1,alias-2
becomes identical to
--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2
the patch to implement that is probably smaller than your offering.
It looks like a nice alternative, but note that it doesn't help at all
with any shell limit. (In fact it would hit any such limit sooner.) So
I think this is a matter of what you think is more elegant for dnsmasq
itself; particularly in the configuration file context where shell
limits don't apply.
Maybe we should do both?
In principle I'm happy to code up and test multiple solutions here; it's
not a large amount of work in any case. So please do let me know what
you would prefer.

I'll take both, for preference.
Actually - no. You test the long-options patch and I'll take that. I'll
do the split-into-multiple lines one.

I have (or in fact my colleague has) successfully tested this now (with a
real OpenStack rig, 90 cirros VMs at the same time on a GCE n1-standard-64
compute node), with no further changes to the patch that I attached before.

So I believe the patch is good - please would you consider it for merging?

Many thanks - Neil

Post by Simon Kelley
Cheers,
Simon.

Post by Neil Jerram
Best wishes - Neil

Post by Neil Jerram
Calico [1] with OpenStack
(https://docs.projectcalico.org/v2.6/getting-started/openstack/)

uses
--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,

Post by Neil Jerram
where each occurrence of ",<alias-if-name>" occupies 15

characters, and

Post by Neil Jerram
there can in principle be as many <alias-if-name>s as you can have

VMs

Post by Neil Jerram
on a single OpenStack compute host. Currently an option arg is

limited

Post by Neil Jerram
in dnsmasq to 1025 chars overall, which only allows for 67
<alias-if-name>s, which is not necessarily enough, on a powerful

compute

Post by Neil Jerram
host.
So I wonder what folk would think about reallocating as necessary

Post by Neil Jerram
allow an option arg to be arbitrarily long? (Or at least, as long

Post by Neil Jerram
getopt and the containing shell will allow.) For reference I've
attached a patch that I think would implement that - but I haven't

yet

Post by Neil Jerram
been able to test it at all, so please don't merge it yet!
Thanks in advance for your thoughts!
Neil
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Simon Kelley

2018-01-18 22:50:49 UTC

Patch applied.

Cheers,

Simon.

Thanks for looking at this, Simon. Some thoughts below...

On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley
I'm in favour, in theory at least, of removing arbitrary limits.
Experience has shown that no matter how big, someone,

somewhere, will

always find them. The code originally used a fixed buffer

which happened

to be unused at that point, to reduce the memory footprint. Whilst
dnsmasq is still intended to be "small", small is a relative

thing, and

absolutely, rather bigger than it was 15 years ago, so

allocating a big

enough buffer is fine.
In this case, though, as you hint, it's likely that shell

limits will

also be a problem. Even eliminating that by using

configuration files,

you still have very long lines, which is ugly.
Can't we solve this problem by allowing repeated interface

names, so

--bridge-interface=eth1,alias-1,alias-2
becomes identical to
--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2
the patch to implement that is probably smaller than your

offering.

It looks like a nice alternative, but note that it doesn't help at all
with any shell limit. (In fact it would hit any such limit

sooner.) So

I think this is a matter of what you think is more elegant for dnsmasq
itself; particularly in the configuration file context where shell
limits don't apply.

Maybe we should do both?
In principle I'm happy to code up and test multiple solutions

here; it's

not a large amount of work in any case. So please do let me know what
you would prefer.

I'll take both, for preference.
Actually - no. You test the long-options patch and I'll take that. I'll
do the split-into-multiple lines one.
I have (or in fact my colleague has) successfully tested this now (with
a real OpenStack rig, 90 cirros VMs at the same time on a GCE
n1-standard-64 compute node), with no further changes to the patch that
I attached before.
So I believe the patch is good - please would you consider it for merging?
Many thanks - Neil
Cheers,
Simon.

Best wishes - Neil

> Calico [1] with OpenStack
>

(https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses

>
>

--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,

>
> where each occurrence of ",<alias-if-name>" occupies 15
characters, and
> there can in principle be as many <alias-if-name>s as you

can have VMs

> on a single OpenStack compute host. Currently an option arg is
limited
> in dnsmasq to 1025 chars overall, which only allows for 67
> <alias-if-name>s, which is not necessarily enough, on a powerful
compute
> host.
>
> So I wonder what folk would think about reallocating as

necessary to

> allow an option arg to be arbitrarily long? (Or at least,

as long as

> getopt and the containing shell will allow.) For reference I've
> attached a patch that I think would implement that - but I

haven't yet

> been able to test it at all, so please don't merge it yet!
>
> Thanks in advance for your thoughts!
> Neil
>
>
>
> _______________________________________________
> Dnsmasq-discuss mailing list
> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
>
_______________________________________________
Dnsmasq-discuss mailing list
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Neil Jerram

2018-01-19 11:19:30 UTC

Thanks Simon!

Post by Simon Kelley
Patch applied.
Cheers,
Simon.

Post by Neil Jerram
Thanks for looking at this, Simon. Some thoughts below...
On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley
I'm in favour, in theory at least, of removing arbitrary

limits.

Post by Neil Jerram
Experience has shown that no matter how big, someone,

somewhere, will

Post by Neil Jerram
always find them. The code originally used a fixed buffer

which happened

Post by Neil Jerram
to be unused at that point, to reduce the memory footprint.

Whilst

Post by Neil Jerram
dnsmasq is still intended to be "small", small is a relative

thing, and

Post by Neil Jerram
absolutely, rather bigger than it was 15 years ago, so

allocating a big

Post by Neil Jerram
enough buffer is fine.
In this case, though, as you hint, it's likely that shell

limits will

Post by Neil Jerram
also be a problem. Even eliminating that by using

configuration files,

Post by Neil Jerram
you still have very long lines, which is ugly.
Can't we solve this problem by allowing repeated interface

names, so

Post by Neil Jerram
--bridge-interface=eth1,alias-1,alias-2
becomes identical to
--bridge-interface=eth1,alias-1
--bridge-interface=eth1,alias-2
the patch to implement that is probably smaller than your

offering.

Post by Neil Jerram
It looks like a nice alternative, but note that it doesn't help at

all

Post by Neil Jerram
with any shell limit. (In fact it would hit any such limit

sooner.) So

Post by Neil Jerram
I think this is a matter of what you think is more elegant for

dnsmasq

Post by Neil Jerram
itself; particularly in the configuration file context where shell
limits don't apply.
Maybe we should do both?
In principle I'm happy to code up and test multiple solutions

here; it's

Post by Neil Jerram
not a large amount of work in any case. So please do let me know

what

Post by Neil Jerram
you would prefer.

I'll take both, for preference.
Actually - no. You test the long-options patch and I'll take that.

I'll

Post by Simon Kelley
do the split-into-multiple lines one.
I have (or in fact my colleague has) successfully tested this now (with
a real OpenStack rig, 90 cirros VMs at the same time on a GCE
n1-standard-64 compute node), with no further changes to the patch that
I attached before.
So I believe the patch is good - please would you consider it for

merging?

Post by Simon Kelley
Many thanks - Neil
Cheers,
Simon.

Post by Neil Jerram
Best wishes - Neil

Post by Neil Jerram
Calico [1] with OpenStack

(https://docs.projectcalico.org/v2.6/getting-started/openstack/)

uses
--bridge-interface=<context-if-name>,<alias-if-name>,<alias-if-name>,...,

Post by Neil Jerram
where each occurrence of ",<alias-if-name>" occupies 15

characters, and

Post by Neil Jerram
there can in principle be as many <alias-if-name>s as you

can have VMs

Post by Neil Jerram
on a single OpenStack compute host. Currently an option arg

Post by Neil Jerram
limited

Post by Neil Jerram
in dnsmasq to 1025 chars overall, which only allows for 67
<alias-if-name>s, which is not necessarily enough, on a

powerful

Post by Neil Jerram
compute

Post by Neil Jerram
host.
So I wonder what folk would think about reallocating as

necessary to

Post by Neil Jerram
allow an option arg to be arbitrarily long? (Or at least,

as long as

Post by Neil Jerram
getopt and the containing shell will allow.) For reference

I've

Post by Neil Jerram
attached a patch that I think would implement that - but I

haven't yet

Post by Neil Jerram
been able to test it at all, so please don't merge it yet!
Thanks in advance for your thoughts!
Neil
_______________________________________________
Dnsmasq-discuss mailing list

http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss