exim, local resolver, host name lookups and IPv6
 

Hi,

Exim has the habit of trying to find out about its host names and IP
addresses when it starts up. This has, in the past, been an issue for
the Debian packages, since a Debian system might be on a
dial-on-demand modem line with expensive costs and thus should not do
unnecessary DNS lookup when the MTA is started.

A very similiar mail has been on Debian Planet two weeks ago, but I
haven't received responses that got me clued up. Therefore, I am using
the last resort, debian-devel.

I'd like to solicit opinions from people who are more experienced than
me with Unix, the local resolver library including /etc/hosts and
/etc/nsswitch.conf, DNS, and - especially - the customs that apply on
a system running IPv6.

To avoid the extra DNS lookups, the Exim packages have a Debconf
option to configure exim for "minimal DNS usage", which hardcodes the
hostname into Exim's configuration at package configuration time. This
was necessary since - without this option - exim looks up its own host
name in the DNS even when a completely local operation is invoked.

In some cases, exim still looks up its IP address when a listening
daemon starts up. This is why the Debian installer configures
127.0.1.1 (not 127.0.0.1) for the local hostname on installation,
yielding /etc/hosts files like

127.0.0.1 localhost
127.0.1.1 myfoo.localdomain myfoo

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

However, in the last few weeks I have heard a few cases where exim
does IPv6 AAAA lookups when a listening daemon starts up. An strace
shows a gethostbyname2 call for AF_INET6, and if we want to continue
the line we went in the past, we'd need an IPv6 address for
myfoo.localdomain in /etc/hosts as well.

I am now wondering how this could be implemented. In IPv4, we have
127.0.0.0/8 available for the local host and could arbitrarily choose
127.0.1.1 to configure the local host name on. In IPv6, there is only
::1, which is a single address. Would it be possible to choose an
arbitrary "link local" address on lo, the loopback interface? Or is
there any better way?

This being said, I consider the entire 127.0.1.1 business a horrible
hack which is one of the most ugly things I have ever seen. Do we have
a chance to implement this in a more cleaner way, or is it still the
way to go for the distribution, where we don't know zilch about the
environment where an installed system is going to be used?

This issue leads to people adding their local host name to ::1 in
/etc/hosts, which might re-introduce other issues that we experienced
in a phase when we did the same for 127.0.0.1, eventually ending up
with 127.0.1.1, or to disabling IPv6 altogether, which is a bad thing
in a time where IPv6 should be enabled, not disabled. So I'd like to
find a clean solution which could then be implemented in whatever part
of Debian might be responsible.

I feel that the IPv6 issue is the same that led us to invoke the
127.0.1.1 hack for IPv4, and if the answer to the IPv6 issue is "fix
exim", then _how_ should exim be fixed, and why wasn't the answer to
the IPv4 version of the issue "fix exim"?

Any hints will be appreciated.

Greetings
Marc

--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

exim, local resolver, host name lookups and IPv6
 

On Fri, Apr 11, 2008 at 04:41:26PM +0200, Marc Haber <mh+debian-devel@zugschlus.de> wrote:
> Hi,
>
> Exim has the habit of trying to find out about its host names and IP
> addresses when it starts up. This has, in the past, been an issue for
> the Debian packages, since a Debian system might be on a
> dial-on-demand modem line with expensive costs and thus should not do
> unnecessary DNS lookup when the MTA is started.
(...)

The main question to be able to answer your question correctly is:
what does it need these information for ?
The alternate question being: Does it really need this information from
DNS ?

Host name can be returned by gethostname(2), for example, and you can
add the result from getdomainname(2) for an FQDN.

IP address could be taken from network interfaces configuration, too.

Mike


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

* Marc Haber <mh+debian-devel@zugschlus.de> [080411 16:41]:
> To avoid the extra DNS lookups, the Exim packages have a Debconf
> option to configure exim for "minimal DNS usage", which hardcodes the
> hostname into Exim's configuration at package configuration time. This
> was necessary since - without this option - exim looks up its own host
> name in the DNS even when a completely local operation is invoked.

I think the main problem is that Debian is by default setting up those
ipv6 stuff into the interface even when you are in an pure ipv4
environment. That way exim4 cannot do anything to avoid ipv6 stuff
and evil things like this can happen.

I don't think that is only limited to additional lookups. I think I've
also seen a message not being sent on etch, because the target host
also had a AAAA record. (At least I think that is the reason, after
disabling ipv6 in exim4.conf it was sent).

> I feel that the IPv6 issue is the same that led us to invoke the
> 127.0.1.1 hack for IPv4, and if the answer to the IPv6 issue is "fix
> exim", then _how_ should exim be fixed, and why wasn't the answer to
> the IPv4 version of the issue "fix exim"?

I think it is not an issue of only exim. Many services want to have a
name and prefer some canonical or fully qualified hostname.
Which useally means something like a gethostbyname or doing a resolv
forward and backward manually.

So I fear the solution might not easily be reachable globally. If exim
can be fixed here depends on why it does does lookups. (As far as I
remember it does not only lookup the hostname, but also things like
"localhost", dunno whether this is caused by the term showing up in
some configuration file or if it is something hardcoded).

Hochachtungsvoll,
Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

* Mike Hommey <mh@glandium.org> [080411 16:51]:
> The main question to be able to answer your question correctly is:
> what does it need these information for ?
> The alternate question being: Does it really need this information from
> DNS ?
>
> Host name can be returned by gethostname(2), for example, and you can
> add the result from getdomainname(2) for an FQDN.

Which seems to not even be used by "hostname -f", which (according to
ltrace) calls gethostbyname on the output of gethostname instead.

And the "CONFORMING TO" part getdomainname(2) does not look very
promissing, too.

Hochachtungsvoll,
Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

* Bernhard R. Link <brlink@debian.org> [080411 18:16]:
> And the "CONFORMING TO" part getdomainname(2) does not look very
> promissing, too.

actually, the manpage seems to miss the most important part:
getdomainname is supposed to return "(none)", if the system is not
using NIS according to LSB and marked as depreceated...

Hochachtungsvoll,
Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

* Mike Hommey:

> The main question to be able to answer your question correctly is:
> what does it need these information for ?

It needs to know all of its own host names in order to recognize local
mail. At least I think this is the motivation; obviously, using reverse
lookup to gather this data is not fully reliable.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

* Bernhard R. Link:

> I think the main problem is that Debian is by default setting up those
> ipv6 stuff into the interface even when you are in an pure ipv4
> environment. That way exim4 cannot do anything to avoid ipv6 stuff
> and evil things like this can happen.

Yes, I agree this is a problem, because IPv6 still doesn't work out of
the box because IPv6 networking is initialized too late in the boot
process.

We really, really should fix the latter, or completely disable IPv6 by
default, IOW, somehow prevent that the ipv6 module is loaded.

However, this is kind of secondary to Marc's question. I think we might
get away without this type of auto-discovery of host names in Exim.
This would sidestep the issue.

Boy, I've never used sendmail that much, but I keep thinking about this
as related to Cw macro initialization. Weired.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

On 2008-04-11, Marc Haber <mh+debian-devel@zugschlus.de> wrote:
> Hi,
>
> Exim has the habit of trying to find out about its host names and IP
> addresses when it starts up. This has, in the past, been an issue for
> the Debian packages, since a Debian system might be on a
> dial-on-demand modem line with expensive costs and thus should not do
> unnecessary DNS lookup when the MTA is started.
>
> A very similiar mail has been on Debian Planet two weeks ago, but I
> haven't received responses that got me clued up. Therefore, I am using
> the last resort, debian-devel.
> I'd like to solicit opinions from people who are more experienced than
> me with Unix, the local resolver library including /etc/hosts and
> /etc/nsswitch.conf, DNS, and - especially - the customs that apply on
> a system running IPv6.
>
> To avoid the extra DNS lookups, the Exim packages have a Debconf
> option to configure exim for "minimal DNS usage", which hardcodes the
> hostname into Exim's configuration at package configuration time. This
> was necessary since - without this option - exim looks up its own host
> name in the DNS even when a completely local operation is invoked.
>
> In some cases, exim still looks up its IP address when a listening
> daemon starts up. This is why the Debian installer configures
> 127.0.1.1 (not 127.0.0.1) for the local hostname on installation,
> yielding /etc/hosts files like
>
> 127.0.0.1 localhost
> 127.0.1.1 myfoo.localdomain myfoo
>
> # The following lines are desirable for IPv6 capable hosts
>::1 ip6-localhost ip6-loopback
> fe00::0 ip6-localnet
> ff00::0 ip6-mcastprefix
> ff02::1 ip6-allnodes
> ff02::2 ip6-allrouters
> ff02::3 ip6-allhosts
>
> However, in the last few weeks I have heard a few cases where exim
> does IPv6 AAAA lookups when a listening daemon starts up. An strace
> shows a gethostbyname2 call for AF_INET6, and if we want to continue
> the line we went in the past, we'd need an IPv6 address for
> myfoo.localdomain in /etc/hosts as well.

Hi, Marc:

Why is exim using gethost* calls? If you look in the
exim-4.69/src/host.c file, you'll see implementation details like:

#if HAVE_GETIPNODEBYNAME
hostdata = getipnodebyname(CS host->name, af, 0, &error_num);
#else
hostdata = gethostbyname2(CS host->name, af);
error_num = h_errno;
#endif

or

#if HAVE_GETIPNODEBYADDR
hosts = getipnodebyaddr(CS &addr6, sizeof(addr6), AF_INET6, &h_errno);
#else
hosts = gethostbyaddr(CS &addr6, sizeof(addr6), AF_INET6);
#endif

glibc has had these new getipnode* calls for quite some time; there's no
reason not to use them. Why doesn't exim-4.69/OS/os.h-Linux define
these feature macros?

> I am now wondering how this could be implemented. In IPv4, we have
> 127.0.0.0/8 available for the local host and could arbitrarily choose
> 127.0.1.1 to configure the local host name on. In IPv6, there is only
>::1, which is a single address. Would it be possible to choose an
> arbitrary "link local" address on lo, the loopback interface? Or is
> there any better way?

Yes, there is a much better way: do not perform name resolution to
determine the host's FQDN. It is wrong. The MTA needs to know the
"mail name" or FQDN of the system, and it may need to know specific IPv4
or IPv6 addresses to bind to if it is running an SMTP server, but it
does not need to know any particular mapping between the two.

SMTP clients (in this case, MTAs which do not necessarily have an SMTP
server running, but which may initiate SMTP connections to remote
servers) need something plausible for the HELO verb of the SMTP
transaction, and they also need a plausible domain name to use in the
right hand side of the email address given in the MAIL FROM stage.

It looks like there are functions in src/host.c for performing DNS-based
determination of the system's FQDN. I don't know exactly under which
circumstances these functions are invoked, but policy 11.6 implies that
they are superfluous in the presence of the /etc/mailname file.

> This being said, I consider the entire 127.0.1.1 business a horrible
> hack which is one of the most ugly things I have ever seen. Do we have
> a chance to implement this in a more cleaner way, or is it still the
> way to go for the distribution, where we don't know zilch about the
> environment where an installed system is going to be used?
>
> This issue leads to people adding their local host name to ::1 in
> /etc/hosts, which might re-introduce other issues that we experienced
> in a phase when we did the same for 127.0.0.1, eventually ending up
> with 127.0.1.1, or to disabling IPv6 altogether, which is a bad thing
> in a time where IPv6 should be enabled, not disabled. So I'd like to
> find a clean solution which could then be implemented in whatever part
> of Debian might be responsible.
>
> I feel that the IPv6 issue is the same that led us to invoke the
> 127.0.1.1 hack for IPv4, and if the answer to the IPv6 issue is "fix
> exim", then _how_ should exim be fixed, and why wasn't the answer to
> the IPv4 version of the issue "fix exim"?

I don't see how this issue is analogous to the 127.0.1.1 hack. From
reading the archived discussion[0], the problem is applications which
use a sequence of legacy gethostname(), gethostbyname(), etc. calls to
construct an FQDN, and avoiding accidentally using 'localhost' or
'localhost.localdomain' as the system hostname. If you're using the
newer getipnode* functions, it's possible that you'll get an AI_V4MAPPED
address even when asking for an AF_INET6 address.

The analogous IPv6 hack, btw, would be something atrocious in /etc/hosts
like:

::ffff:127.0.1.1 hostname.domainname

> Any hints will be appreciated.

IME, nullmailer and postfix seem to get along fine without generating
spurious DNS traffic, so

#include <flame/default/mta.h>

> Greetings
> Marc
>

[0] http://lists.debian.org/debian-boot/2005/06/msg00938.html

--
Robert Edmonds
edmonds@debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

On 2008-04-11, Florian Weimer <fw@deneb.enyo.de> wrote:
> * Bernhard R. Link:
>
>> I think the main problem is that Debian is by default setting up those
>> ipv6 stuff into the interface even when you are in an pure ipv4
>> environment. That way exim4 cannot do anything to avoid ipv6 stuff
>> and evil things like this can happen.
>
> Yes, I agree this is a problem, because IPv6 still doesn't work out of
> the box because IPv6 networking is initialized too late in the boot
> process.

How so? Do you mean a situation such as

* no inet6 stanzas configured in /etc/network/interfaces
* a daemon attempts to bind to an AF_INET6 socket late in the boot
process
* the kernel helpfully autoloads ipv6.ko
* the kernel helpfully binds a "scope link" inet6 address to each
network interface and performs router solicitation

?

It's primarily the configuration of those "scope link" inet6 addresses
which cause problems, right? Would loading ipv6.ko with
net.ipv6.conf.all.autoconf = 0 help?

--
Robert Edmonds
edmonds@debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

exim, local resolver, host name lookups and IPv6
 

On Fri, Apr 11, 2008 at 05:48:19PM +0000, Robert Edmonds wrote:

> Why is exim using gethost* calls? If you look in the
> exim-4.69/src/host.c file, you'll see implementation details like:
>
> #if HAVE_GETIPNODEBYNAME
> hostdata = getipnodebyname(CS host->name, af, 0, &error_num);
> #else
> hostdata = gethostbyname2(CS host->name, af);
> error_num = h_errno;
> #endif
[...]
> glibc has had these new getipnode* calls for quite some time; there's no
> reason not to use them. Why doesn't exim-4.69/OS/os.h-Linux define
> these feature macros?

From the getipnodebyname(3) manpage:

NOTES
These functions were present in glibc 2.1.91-95, but were removed
again. Several Unix-like systems support them, but all call them
deprecated.

The best way is to use getaddrinfo() and getnameinfo().

> Yes, there is a much better way: do not perform name resolution to
> determine the host's FQDN. It is wrong.

I agree.

--
Met vriendelijke groet / with kind regards,
Guus Sliepen <guus@debian.org>