RFC: Making mail-transport-agent Priority: optional
 

I recently booted up a Debian Live "standard" image on a test system,
and noticed that it included a running instance of exim. Curious why a
live system would need an MTA, I found Debian Live's policy that the
"standard" image contains everything installed as part of a standard
Debian system (and similarly, the desktop images include everything
installed by the corresponding tasks or metapackages). This seems like
a sensible policy, but it reminded me again that Debian still has an MTA
in standard. I'd like to change that.

Not every system needs an MTA, and I'd argue that today most systems
don't. End-user systems (desktops, laptops) typically handle mail via
one or more smarthosts elsewhere, driven by MUAs that know how to talk
SMTP. Other tools which send mail have learned to send SMTP as well,
and tools that cannot still have the option of recommending or depending
on an MTA without needing one in standard. And many servers don't need
an MTA either, unless they run programs which need to send mail by
calling sendmail.

I realize that some people use a personal mail configuration which
involves a local MTA, even on a laptop or desktop. However, this
represents a less common configuration, and one which requires somewhat
more difficulty for a single user to set up than the configuration of a
MUA (given that they normally need to do the latter as well). It also
introduces an extra layer end-users would have to monitor to ensure that
mail gets sent, even when their MUA says "sent"; MUAs don't know how to
do that monitoring themselves. I don't think having an MTA in priority
standard makes such a setup any easier for the people who want it; we
have a marvellous packaging system that makes it easy for users to
install the numerous packages they want above-and-beyond standard to
reach a system with everything they expect, and I doubt many users who
use a local MTA will otherwise remain entirely satisfied by the contents
of standard with no other additions.

I also understand that many developers have an attachment to having a
properly configured local MTA. Debian has had an MTA in standard for a
long time, and people feel that MTA ought to actually know how to send
email. Traditional UNIX multiuser systems had a local MTA, and I enjoy
thinking of the days when people regularly sent email to addresses
without an '@' in them. I've certainly observed a certain level of "fix
your MTA" in response to users who rely on smarthosts instead. However,
I think we need to consider how our users actually *use* their systems
today, and I don't think most of our users really want or need a local
MTA on their systems by default.

Popcon shows that ~65-70% of Debian systems have exim4 installed. How
many of those users really use a local MTA, and how many just have a
forlorn MTA checking an empty queue for mail that will never appear?
30-35% of users cared enough to remove exim, and another 7% or so seem to
have configured their systems to stop running it (at boot or otherwise)
without actually removing it. How many more just don't notice one more
process starting at boot time, and don't bother removing it even though
they'll never use it?

I checked into what packages we have with priority >= standard that want
an MTA. Turns out that only bsd-mailx (also standard) has a Depends on
an MTA, but bsd-mailx seems exactly as "standard" as an MTA in the first
place; if you don't have one you don't need the other. (And from what I
can tell, most scripts which want to send mail use sendmail directly,
not mail; either way, someone can always install bsd-mailx if they need
a "mail" command.) People do sometimes use mail/mailx as a personal
MUA, but installing bsd-mailx seems just as easy as installing the
numerous MUAs in optional, and I don't think anyone will argue that more
people use mailx than one of the more user-friendly MUAs. :) I'd
propose moving bsd-mailx to priority optional as well.

The following 6 packages with priority >= standard have a Recommends on
an MTA:

- apt-listchanges, which has the option of mailing changelogs or news to
root. However, apt-listchanges can just as easily display those
changes at install time, and many end-users will never see mail sent
to root or any other local user. Perhaps apt-listchanges should just
Suggests an MTA.

- procmail: Normally only needed if you actually process mail locally on
a system using more than just a MUA; also, procmail has an alternative
recommends on fetchmail, and will work easily enough with any other
means of getting mail on a system. Easily installed by anyone using
it. (May want to move to optional as well, by the same arguments,
since it primarily goes with an MTA-based mail setup.)

- cron and at, which really want to send mail rather than just logging.
The output of these can potentially prove useful, but only if it goes
somewhere that will actually get read. Many end-users will never see
mail sent to root or any other local user. We don't currently help
users forward mail for root off to an email address they'll actually
read, so they'll only see if it they know to configure forwarding
themselves, or they run a MUA which looks at the local mail store,
which most MUAs don't. I've personally run cron for years without an
MTA; the few times I've written a cron job whose output I cared about,
I piped its output to logger or otherwise captured it in a logfile.
Anyone counting on cron to send them mail for their own cron jobs need
only fire up apt and install an MTA, which does not seem particularly
difficult.

- mutt, which also knows how to speak SMTP, POP, and IMAP. Seems like
mutt should just Suggests an MTA.

A couple other packages in standard and above which might care:

- bsdmainutils contains "calendar", as well as a disabled-by-default
daily cronjob to mail users about events on their calendar. Anyone
enabling that cronjob and using calendar can always install an MTA.
Perhaps bsdmainutils should suggest an MTA.

- reportbug suggests an MTA. However, reportbug knows how to speak
SMTP, and it has a very sensible default configuration using
reportbug.debian.org, which works without either a local MTA *or*
smarthost.


Regarding people seeing local mail, Joey Hess wrote:
> there are three types of systems: 1. local mail is read, active admin
> (a DD's own box). 2. local mail is forward, active remote admin (a
> DD's family or work box) 3. everybody else
>
> and 1 and 2 need knowledgeable people for whom apt-get install postfix
> is not exactly hard while they're editing the aliases file, QED


Based on all of the above, I'd like to propose the following changes:

- Change exim4-daemon-light, bsd-mailx, and possibly procmail to
Priority: optional.
- Change apt-listchanges, mutt, cron, and at to Suggests: default-mta |
mail-transport-agent, rather than the current Recommends. (In
addition to reflecting the concerns above, this change also needs to
happen to avoid pulling in an MTA by default anyway via the Recommends
of standard packages.)


What would it take to make this change? Have I missed any important
points? Would any other packages need changes, other than the ones I've
mentioned above?

I will happily work to coordinate this transition.

- Josh Triplett


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111012213912.GB1964@leaf">http://lists.debian.org/20111012213912.GB1964@leaf

RFC: Making mail-transport-agent Priority: optional
 

Josh Triplett <josh@joshtriplett.org> writes:

> What would it take to make this change?

Changing the LSB. Or you need to keep the sendmail interface. Which is
what mail-transport-agent provides.

> Have I missed any important points?

You forgot to explain the upside, reason, why, gain, whatever.

> Would any other packages need changes, other than the ones I've
> mentioned above?

all packages with cron jobs, all 3rd party applications assuming an UNIX
environment, ++

The reasons are all explained in the release notes.

Why not remove syslog as well? I'm pretty sure there are plenty of end
users never ever looking at a log file.


Bjørn


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 871uuhhly0.fsf@nemi.mork.no">http://lists.debian.org/871uuhhly0.fsf@nemi.mork.no

RFC: Making mail-transport-agent Priority: optional
 

On Wed, Oct 12, 2011 at 02:39:13PM -0700, Josh Triplett wrote:
> Popcon shows that ~65-70% of Debian systems have exim4 installed.
> 30-35% of users cared enough to remove exim, and another 7% or so seem to
> have configured their systems to stop running it (at boot or otherwise)
> without actually removing it.

That would break their system as daemons have no way to notify the user
something is wrong. Instead, I bet that you read popcon's vote<installed
that way -- which at least in this case nearly always means the user has the
filesystem mounted noatime.

Which is a damn reasonable thing to do, as it prevents every write from
having its metadata cost doubled. Current Debian's default, relatime,
sacrifices performance and causes unnecessary spin-ups, with the stated
explanation being certain uses of mbox, a long-obsolete format that might be
adequate at most for rare mail from daemons but not actual mail from live
people[1]. And even that is fixable by making mail readers manually set the
access time.


[1]. How many non-technical folks you know who do not add fat attachments to
a majority of mails?

--
1KB // Yo momma uses IPv4!


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111012224525.GA16178@angband.pl">http://lists.debian.org/20111012224525.GA16178@angband.pl

RFC: Making mail-transport-agent Priority: optional
 

Adam Borowski wrote:
> On Wed, Oct 12, 2011 at 02:39:13PM -0700, Josh Triplett wrote:
>> Popcon shows that ~65-70% of Debian systems have exim4 installed.
>> 30-35% of users cared enough to remove exim, and another 7% or so seem to
>> have configured their systems to stop running it (at boot or otherwise)
>> without actually removing it.
>
> That would break their system as daemons have no way to notify the user
> something is wrong.

Not true. Daemons can write to log files, as almost all daemons other than
cron do.

> Instead, I bet that you read popcon's vote<installed
> that way -- which at least in this case nearly always means the user has the
> filesystem mounted noatime.

Fair enough. I took a guess, based on exim running at boot, but noatime
would certainly produce the same effect. I use noatime myself. (I also
don't run popcon myself.)

The 30-35% figure for users who have removed exim still make sense,
though, to the extent that popcon numbers for a package with priority >=
standard can make sense.

In any case, I didn't intend the popcon numbers as any kind of proof,
just a bit of very rough evidence about how many people have removed an
MTA. For comparison, ~15% of users don't have wamerican (standard, with
numerous alternatives), ~15% of users don't have mutt (standard), ~15%
of users don't have w3m (standard for some reason, with numerous
alternatives), ~14% of users don't have ncurses-term (standard), ~10% of
users don't have telnet (standard), and ~4% of users don't have pciutils
(standard). I haven't checked the full list of standard packages, but
it does seem notable that of standard packages, exim gets removed much
more frequently than others.

That said, I'd hardly argue for popcon numbers as any significant reason
against having an MTA in standard. I provided a pile of much better
reasons in my original mail.

> Which is a damn reasonable thing to do, as it prevents every write from
> having its metadata cost doubled. Current Debian's default, relatime,
> sacrifices performance and causes unnecessary spin-ups, with the stated
> explanation being certain uses of mbox, a long-obsolete format that might be
> adequate at most for rare mail from daemons but not actual mail from live
> people[1]. And even that is fixable by making mail readers manually set the
> access time.

Agreed on all counts. The one other argument I've seen for relatime
involves tmpreaper and similar programs which prune files by age. The
original commit that put relatime into the kernel
(11ff6f05f1e836a6a02369a4c4b64757e484adc1) cited tmpreaper as a
rationale.

- Josh Triplett


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111013011802.GA2626@leaf">http://lists.debian.org/20111013011802.GA2626@leaf

RFC: Making mail-transport-agent Priority: optional
 

Bjørn Mork wrote:
> Josh Triplett <josh@joshtriplett.org> writes:
>> What would it take to make this change?
>
> Changing the LSB. Or you need to keep the sendmail interface. Which is
> what mail-transport-agent provides.

lsb-core provides the LSB interface, and it has priority extra, not
standard. It already has a dependency on an MTA, along with
dependencies on a large pile of other programs and libraries with
priorities lower than standard. So, as far as I can tell the LSB does
not specify what appears in standard, and removing an MTA from standard
would not affect lsb-core in any way.

>> Have I missed any important points?
>
> You forgot to explain the upside, reason, why, gain, whatever.

Re-reading my original mail, you're right, I do seem to have missed
covering that point explicitly. Thanks. :)

The main reasons to stop having an MTA in standard:

- Starting a daemon at boot time, which slows down booting. This led me
to notice the problem in Debian Live: it took a non-trivial amount of
time for the boot process to finish starting exim and move on.

- Listening on ports by default, which exposes the system to any
potential vulnerabilities, as well as potentially allowing the sending
of spam. I've checked, and out of all the packages with priority
standard or above, only exim and isc-dhcp-client listen on ports by
default. Removing an MTA significantly reduces the attack surface of
a default Debian system.

- Asking configuration questions via debconf at install time, which
increases the amount of work and complexity required to install
Debian. For most users, these questions will duplicate the process
they later go through to configure their MUA.

- Taking time to download and install, which increases the time and
bandwidth needed to install or upgrade a Debian system.

- Running a daemon all the time, which takes up RAM.

- Taking up space on disk, as with any other package installed but not used.

- Taking up space in the process listing; the more programs a system
runs that it doesn't use, the longer it takes to look over the output
of "ps auxf" or top.

- Similarly, taking up space in the list of installed packages, the
apt-listchanges output, and so on. Any package installed but not used
incurs a small but non-zero amount of mental overhead.


Users can easily install an MTA; why do they need one *by default* on
every Debian system they install?

>> Would any other packages need changes, other than the ones I've
>> mentioned above?
>
> all packages with cron jobs,

...which produce output to somewhere other than a log file, in some
scenario other than "being buggy and accidentally producing output", and
which expect end users to read their output, and which therefore expect
that the end user has configured root's mail to go somewhere they'll
actually read. In any case, cron can still suggest an MTA, and any
package which absolutely needs a working MTA can depend on one (and add
giant warnings that they require a *working* MTA configuration, which a
depends does not guarantee).

> all 3rd party applications assuming an UNIX
> environment, ++

By which you mean having a sendmail binary? If you mean the LSB again,
LSB support requires installation of lsb-core, which depends on an MTA.
If you mean third-party applications in .deb form, they should depend on
an MTA if they need one. And otherwise, installing third-party
applications outside the package manager frequently requires installing
additional supporting packages to provide expected interfaces, and
having a working sendmail seems no different.

And on top of all of that, nothing guarantees that the sendmail binary
can actually send mail outside the local system. The admin will still
need to know that the program they install wants to send mail with
sendmail, so that they know not to say "local delivery only".

> The reasons are all explained in the release notes.

Which release notes do you mean? I don't see anything about exim or
mail-transport-agent in the Debian squeeze release notes (other than the
large table of various package versions in Debian, which includes
notable packages of many different priorities).

> Why not remove syslog as well? I'm pretty sure there are plenty of end
> users never ever looking at a log file.

Among other things, most daemons log information to log files (as
opposed to mail). In any case, I'm specifically proposing moving an MTA
out of standard, not anything else at the moment. :)

- Josh Triplett


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111013033450.GA2975@leaf">http://lists.debian.org/20111013033450.GA2975@leaf

RFC: Making mail-transport-agent Priority: optional
 

As someone who runs Debian on his smartphone, I completely agree with
making an MTA optional.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6G1so+Zp-YSc5VyU1TZYp5C-nMyc2o_SthaTDzck_9xnQ@mail.gmail.com">http://lists.debian.org/CAKTje6G1so+Zp-YSc5VyU1TZYp5C-nMyc2o_SthaTDzck_9xnQ@mail.gmail.com

RFC: Making mail-transport-agent Priority: optional
 

On Thu, Oct 13, 2011 at 6:45 AM, Adam Borowski wrote:

> That would break their system as daemons have no way to notify the user
> something is wrong.

The user will not be notified even if the daemons send a mail to them.
I don't think any of the desktops GUIs that we ship know anything
about the local mail queue unless explicitly configured in an MUA, nor
do they notify the user when there is new mail.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6HD1xvwk7B5QHn=RYJa+6N-zHQRYAt7asLWTMNhYRpVuA@mail.gmail.com">http://lists.debian.org/CAKTje6HD1xvwk7B5QHn=RYJa+6N-zHQRYAt7asLWTMNhYRpVuA@mail.gmail.com

RFC: Making mail-transport-agent Priority: optional
 

Hi there!

On Thu, 13 Oct 2011 03:18:04 +0200, Josh Triplett wrote:
> The 30-35% figure for users who have removed exim still make sense,
> though, to the extent that popcon numbers for a package with priority >=
> standard can make sense.
>
> In any case, I didn't intend the popcon numbers as any kind of proof,
> just a bit of very rough evidence about how many people have removed an
> MTA.

I disagree on that, according to popcon we have 19.27% of users has
postfix installed, which could mean that ~90% of users has an MTA
installed.

FWIW, I would say that the postfix numbers are more representative,
because they are the result of a specific choice.

Thx, bye,
Gismo / Luca

RFC: Making mail-transport-agent Priority: optional
 

* Josh Triplett <josh@joshtriplett.org> [111013 05:51]:
> Users can easily install an MTA; why do they need one *by default* on
> every Debian system they install?

Because the system is not in a useful state without. If you want to
cripple your system, just deinstall it.

> The main reasons to stop having an MTA in standard:
>
> - Listening on ports by default, which exposes the system to any
> potential vulnerabilities, as well as potentially allowing the sending
> of spam. I've checked, and out of all the packages with priority
> standard or above, only exim and isc-dhcp-client listen on ports by
> default. Removing an MTA significantly reduces the attack surface of
> a default Debian system.

Last I checked, exim does not listen to things on the outside by
default. (Though I had nothing against it no longer listening
on tcp, as long as it still accepts mails)

> - Asking configuration questions via debconf at install time, which
> increases the amount of work and complexity required to install
> Debian. For most users, these questions will duplicate the process
> they later go through to configure their MUA.

People have different needs here. There really is no "one size fits
all". Your whole argument of not wanting one being started at most
means there is one option missing.

> - Taking time to download and install, which increases the time and
> bandwidth needed to install or upgrade a Debian system.

Please drop the "upgrade". If you deinstall it there is no cost at
upgrading.

> - Running a daemon all the time, which takes up RAM.

Then do not start it.

> - Taking up space on disk, as with any other package installed but not used.

Then deinstall it.

> - Taking up space in the process listing; the more programs a system
> runs that it doesn't use, the longer it takes to look over the output
> of "ps auxf" or top.

Then do not start it.

> - Similarly, taking up space in the list of installed packages, the
> apt-listchanges output, and so on. Any package installed but not used
> incurs a small but non-zero amount of mental overhead.

Then deinstall it.

Please note that an MTA perfectly fits into the
'If the expectation is that an experienced Unix person who found it
missing would say "What on earth is going on, where is foo?", it must
be an important package.' criterion. So only having priority standard
is already a compromise (and I guess historically due to there being
alternative implementations).

I'm all for having minimal dependencies to make deinstalling stuff one
does not like or want easier. But defaults should be something
reasonable.

Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111013081738.GB28246@server.brlink.eu">http://lists.debian.org/20111013081738.GB28246@server.brlink.eu

RFC: Making mail-transport-agent Priority: optional
 

On Thu, 13 Oct 2011 09:46:19 +0200
Luca Capello <luca@pca.it> wrote:

> Hi there!
>
> On Thu, 13 Oct 2011 03:18:04 +0200, Josh Triplett wrote:
> > The 30-35% figure for users who have removed exim still make sense,
> > though, to the extent that popcon numbers for a package with priority >=
> > standard can make sense.
> >
> > In any case, I didn't intend the popcon numbers as any kind of proof,
> > just a bit of very rough evidence about how many people have removed an
> > MTA.
>
> I disagree on that, according to popcon we have 19.27% of users has
> postfix installed, which could mean that ~90% of users has an MTA
> installed.
>
> FWIW, I would say that the postfix numbers are more representative,
> because they are the result of a specific choice.

The whole popcon thing is misleading - in many situations where you
don't want an MTA you also will not install popularity-contest.

The only time I see issues with things depending on an MTA is when
someone mistakenly depends on 'at' or puts 'at' into a multistrap
config then wonders where the extra Mb have gone....

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/