On 04/29/2012 03:13 AM, Marco d'Itri wrote:
> Is this the right time to do it?

First lets fix all RC bugs and get other more important things done than
discussing - yet again - the replacement of a well working MTA by a
different well working MTA. Both are equally easy to setup and configure
with debconf for the "normal" user - and that is what counts. Those who
know what they want/need also know how to install exactly that.


--
Bernd Zeimetz Debian GNU/Linux Developer
http://bzed.de http://www.debian.org
GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4F9E614E.8070900@bzed.de">http://lists.debian.org/4F9E614E.8070900@bzed.de

As I'm not involved in developing dma at all, neither upstream nor in
Debian, I'm not the right one to discuss implementation details in depth
with.


* Russ Allbery [2012-04-29 17:32 -0700]:
> Adam Borowski <kilobyte@angband.pl> writes:
> > On Sun, Apr 29, 2012 at 10:50:45PM +0200, Carsten Hey wrote:
>
> >> Looks like the DragonFly Mail Agent (dma), which already has been
> >> mentioned in this thread, could become a decent default for Wheezy+1
> >> after some small changes.
> >>
> >> In a nutshell: it's able to deliver locally and remotely, has a queue,
> >> supports TLS/SSL, does not listen on port 25 and instead of running as
> >> daemon, it if run every 5 minutes via cron to flush the queue.
>
> > I hope you mean: to run retries (in which case every 5 minutes is an
> > overkill).
>
> > Delaying every outgoing mail by 5 minutes doesn't sound like a good idea
> > to me.
>
> ... incron ... You'd still need timed handling of queued mail for retries.

There are two modes dma can run in, one is the deferred mode, which
seems to be basically how you think dma always works. The other is the
immediate mode that is default in Debian and upstream and as the name
suggests it delivers immediately if possible and it forks for mails that
can not be delivered immediately. The resulting processes then handle
besides obvious other things the timed handling of queued mail for
retries.


The rest of this mail is likely not interesting for most of you since it
only tries to answer the natural follow up question "Why does it need
a cronjob then?" and explains why I don't think anymore that a switch to
incron should be considered.


Two reasons for running dma -q via cronjob in my own words but stolen
from README.Debian are:
* If the queue is not empty after reboot, dma -q needs to be run at
least once to start delivering these mails. A @reboot cronjob or an
init script would also to this job.
* Delivery processes might die for various reasons, but the mails still
need to be delivered in a timely manner.

If dma would be the default MTA, then it should IMHO be as reliable as
possible and even try to prevent user errors. If a user would
unintentionally enables deferred mode (which is useful if you are behind
a dial-up line) but would not set up dma -q to run periodically, then
the mails would not be delivered without such a default cronjob.
A comment that reminds users to adapt the cronjob if needed should be
added to the config file. If dma -q is run every 5 minutes be default
anyway, the option -bq does not make that much sense anymore; this can
possibly be solved by implementing different ways of processing queued
mails. All in all, enabling the cronjob by default, as it is already
done in Debian, seems to be sane.


> I think that was Carsten's point about switching to incron, which
> would then do the right thing for new outgoing mail.

This is a reasonable and logical assumption, but it is wrong

Actually the reason to mention incron was that I thought that running
dma -q if triggered by inotify could be a bit cheaper than running it
every five minutes. For a default MTA, the amount of systems that run
it could make considering even minimal differences in efficiency
worthwhile.

The idea was to use incron to restart failed delivery processes, if this
would be possible at all depends on details of dma and incron/inotify
I'm not aware off. An additional reason to the explanation above not to
use incron is that in rare cases dma might fail for example with ENOMEM
whilst reading its configuration file before it is able to open any file
in the spool dir, which would render running it by incron to be not 100%
bullet proof anyway.


Regards
Carsten


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120430095818.GA6834@furrball.stateful.de">http://lists.debian.org/20120430095818.GA6834@furrball.stateful.de

On Sat, Apr 28, 2012 at 07:12:42PM -0700, Russ Allbery wrote:
> I'm not sure that I see the point, and I say that as someone who
> replaces Exim with Postfix on all of my boxes.

Nobody's suggesting you need to change to anything. The worst you
have to do if debian changed default MTA, would be to
"apt-get install exim4" (gasp, what horrible pain) when doing new
installations.

> There's nothing particularly wrong with Exim; it works just fine.

Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
forcing quoted-printable conversions to make emails "7bit clean" is quite
horribly wrong.

Debian is the main source of Exim installs in internet, it is also our fault.
According to one old stat[1], 34% of mx records were exim, most probably almost
all simply because it came by default in debian and it was "good enough"
so people didnt' switch away from it.

So yes, switching to postfix by default would reduce the workload of email
servers around the globe (no need to burn cpu cycles and thus co2 to convert
emails to quoted-printable).

Yes that was a bit of a hyperbole, but this is my pet issue. I complained
last about it in 2009 [2] with no change from upstream since...

Riku

[1] http://www.securityspace.com/s_survey/data/man.201007/mxsurvey.html
[2] http://suihkulokki.blogspot.com/2009/07/cult-of-workarounds.html


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120430101419.GA30431@afflict.kos.to">http://lists.debian.org/20120430101419.GA30431@afflict.kos.to

On Mon, Apr 30, 2012 at 11:58:18AM +0200, Carsten Hey wrote:
> * Russ Allbery [2012-04-29 17:32 -0700]:
> > Adam Borowski <kilobyte@angband.pl> writes:
> > > On Sun, Apr 29, 2012 at 10:50:45PM +0200, Carsten Hey wrote:
> >
> > >> Looks like the DragonFly Mail Agent (dma), which already has been
> > >> mentioned in this thread, could become a decent default for Wheezy+1
> > >> after some small changes.
> > >>
> > >> In a nutshell: it's able to deliver locally and remotely, has a queue,
> > >> supports TLS/SSL, does not listen on port 25 and instead of running as
> > >> daemon, it if run every 5 minutes via cron to flush the queue.
>
> If dma would be the default MTA, then it should IMHO be as reliable as
> possible and even try to prevent user errors. If a user would
> unintentionally enables deferred mode (which is useful if you are behind
> a dial-up line) but would not set up dma -q to run periodically, then
> the mails would not be delivered without such a default cronjob.
> A comment that reminds users to adapt the cronjob if needed should be
> added to the config file. If dma -q is run every 5 minutes be default
> anyway, the option -bq does not make that much sense anymore; this can
> possibly be solved by implementing different ways of processing queued
> mails. All in all, enabling the cronjob by default, as it is already
> done in Debian, seems to be sane.

Not on a laptop or any machine that has to conserve power and avoid
unnecessary wakeups / disk spin-ups.

A cronjob every 5 minutes means you need to start up the process, which adds
quite a bit of churn. Worse, it will spam the logs, and since at least
auth.log is fsync()ed after every write, it needs to spin up the disk.

That's too big a price for a MTA on a system that typically goes months or
years without a single mail.

--
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets. Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable and Non-Discriminatory prices.

On 30.04.2012 16:55, Adam Borowski wrote:
> On Mon, Apr 30, 2012 at 11:58:18AM +0200, Carsten Hey wrote:
>> * Russ Allbery [2012-04-29 17:32 -0700]:
[]
>> If dma would be the default MTA, then it should IMHO be as reliable as
>> possible and even try to prevent user errors. If a user would
>> unintentionally enables deferred mode (which is useful if you are behind
>> a dial-up line) but would not set up dma -q to run periodically, then
>> the mails would not be delivered without such a default cronjob.
>> A comment that reminds users to adapt the cronjob if needed should be
>> added to the config file. If dma -q is run every 5 minutes be default
>> anyway, the option -bq does not make that much sense anymore; this can
>> possibly be solved by implementing different ways of processing queued
>> mails. All in all, enabling the cronjob by default, as it is already
>> done in Debian, seems to be sane.
>
> Not on a laptop or any machine that has to conserve power and avoid
> unnecessary wakeups / disk spin-ups.
>
> A cronjob every 5 minutes means you need to start up the process, which adds
> quite a bit of churn. Worse, it will spam the logs, and since at least
> auth.log is fsync()ed after every write, it needs to spin up the disk.
>
> That's too big a price for a MTA on a system that typically goes months or
> years without a single mail.

Hmm.. Now when you mentioned it...

On all our postfix servers (yes we use postfix), I mount a tmpfs over
/var/spool/postfix/run, create subdirs "pid", "private" and "public"
in there and change corresponding dirs in /var/spool/postfix/ into
symlinks to run/$subdir.

Exactly in order to avoid extra disk wakeups every so often, by default
every 5min -- when qmgr gets woken up to re-scan its queue. This is
done by master process according to master.cf, master writes a byte
into corresponding /var/spool/postfix/public/qmgr FIFO, which results
in mtime of that inode being updated every 5 minutes. Oh well.

(And when I create these symlinks, `postfix check' starts reporting
wrong permissions on /var/spool/postfix/p* - since they becomes
"world writable").

FWIW, but Postfix also has this issue, unless it is set up very
carefully and in a non-standard way

(This is something I use since about 2002)

Thanks,

/mjt


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4F9E9014.8020009@msgid.tls.msk.ru">http://lists.debian.org/4F9E9014.8020009@msgid.tls.msk.ru

On Sun, Apr 29, 2012 at 07:18:54PM +0200, Marco d'Itri wrote:
> On Apr 29, Russ Allbery <rra@debian.org> wrote:
> > The giant endless flamewars on debian-devel required to make a decision to
> > change anything.
>
> Unrelated: you have just shown what poisons Debian and has been keeping
> us behind innovation for the last years.
> Not the flamewars themselves, most of us are grown ups and can handle
> them, but the fear that proposing a change will cause endless
> discussions and no results.

Still unrelated, but let me AOL the above. Russ was just joking, but I
completely agree with Marco's point here. We're often scared of
"starting a thread on -devel" to propose changes that are far reaching
enough not to be directly implementable. Every now and then I got asked
advice on whether proposing some change on -devel is a good idea or not.
I'm always happy to give advice, but the fact we sometime worry to
propose changes is worrisome in itself. And it can induce project-wide
inertia as much as worrying too much about performing NMUs to fix bugs.

But we also need to convince ourselves that -devel discussions are
useful and lead to progress. For that to happen, we need more people
that look back at past discussions, summarize their conclusions (if
there have been) or relaunch them (if not), and take concrete actions as
the natural next step of discussing. There are people doing that, but
not nearly enough.

Cheers.
--
Stefano Zacchiroli zack@{upsilon.cc,pps.jussieu.fr,debian.org} . o .
Maître de conférences ...... http://upsilon.cc/zack ...... . . o
Debian Project Leader ....... @zack on identi.ca ....... o o o
« the first rule of tautology club is the first rule of tautology club »

Riku Voipio <riku.voipio@iki.fi> writes:
> On Sat, Apr 28, 2012 at 07:12:42PM -0700, Russ Allbery wrote:

>> I'm not sure that I see the point, and I say that as someone who
>> replaces Exim with Postfix on all of my boxes.

> Nobody's suggesting you need to change to anything. The worst you have
> to do if debian changed default MTA, would be to "apt-get install exim4"
> (gasp, what horrible pain) when doing new installations.

Did you miss the bit where I said that I replace Exim with Postfix on all
of my boxes, despite quoting it? Logically, you can draw the inference
that I know exactly how hard it is to replace the default MTA with a
different one.

>> There's nothing particularly wrong with Exim; it works just fine.

> Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
> forcing quoted-printable conversions to make emails "7bit clean" is
> quite horribly wrong.

I didn't realize that. I agree, that's an annoying missing feature. Has
someone talked with upstream about whether they have plans to implement
it?

> Yes that was a bit of a hyperbole, but this is my pet issue. I
> complained last about it in 2009 [2] with no change from upstream
> since...

> [2] http://suihkulokki.blogspot.com/2009/07/cult-of-workarounds.html

Okay, let me rephrase that: has anyone talked with upstream about this
without making sweeping, confrontational statements about a "cult of
workarounds" and otherwise insulting them?

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87397lcl42.fsf@windlord.stanford.edu">http://lists.debian.org/87397lcl42.fsf@windlord.stanford.edu

* Russ Allbery (rra@debian.org) [120430 17:09]:
> Riku Voipio <riku.voipio@iki.fi> writes:

> > Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
> > forcing quoted-printable conversions to make emails "7bit clean" is
> > quite horribly wrong.
>
> I didn't realize that. I agree, that's an annoying missing feature. Has
> someone talked with upstream about whether they have plans to implement
> it?

Quoting the manual
| accept_8bitmime Use: main Type: boolean Default: false
|
| This option causes Exim to send 8BITMIME in its response to an SMTP EHLO
| command, and to accept the BODY= parameter on MAIL commands. However, though
| Exim is 8-bit clean, it is not a protocol converter, and it takes no steps to
| do anything special with messages received by this route. Consequently, this
| option is turned off by default.

So the *default* configuration doesn't advertise 8BITMIME for the
reason that exim won't do conversions later on when relaying to other
hosts (which might be a possible RFC violation, depending where the
mail is relayed to). One could certainly argue that isn't the best /
recommended state, but it's not that bad either.

(And this is one of the first hits while searching for 8BITMIME and
exim.)


Andi


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120430152149.GP2385@mails.so.argh.org">http://lists.debian.org/20120430152149.GP2385@mails.so.argh.org

On Mon, Apr 30, 2012 at 01:55:24PM BST, Adam Borowski wrote:
> Not on a laptop or any machine that has to conserve power and avoid
> unnecessary wakeups / disk spin-ups.

Or any device with an SSD or SD card (more and more popular net-tops
nowadays).

> A cronjob every 5 minutes means you need to start up the process, which adds
> quite a bit of churn. Worse, it will spam the logs, and since at least
> auth.log is fsync()ed after every write, it needs to spin up the disk.

This is the reason why I got rid of DMA on my systems, the defaults -
cron job and unnecessary entries in the logs.
Other than that I don't really have anything else against it.
If those two get fixed it could be a sane default MTA (IMHO).

Cheers,
--
Raf


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120430171517.GA3896@linuxstuff.pl">http://lists.debian.org/20120430171517.GA3896@linuxstuff.pl

On Monday, April 30, 2012 06:14:19, Riku Voipio wrote:
> On Sat, Apr 28, 2012 at 07:12:42PM -0700, Russ Allbery wrote:
...
> > There's nothing particularly wrong with Exim; it works just fine.
>
> Exim in 2012 not supporting 8BITMIME and thus being the last Major MTA
> forcing quoted-printable conversions to make emails "7bit clean" is quite
> horribly wrong.

I think it would be useful to describe what issue(s) there are concerning
8BITMIME and why this is important. I've found some information [1] about
this, but it isn't clear what problems are actially *caused* by the lack of
8BITMIME support by default in Exim. Is it just slow sending of outbound
attachments?

> Debian is the main source of Exim installs in internet, it is also our
> fault. According to one old stat, 34% of mx records were exim, most
> probably almost all simply because it came by default in debian and it was
> "good enough" so people didnt' switch away from it.

The quoted 2010 survey [2] showed Exim was the most popular MTA (which I found
surprising), deployment of Exim growing just slightly faster than Postfix, and
everything else falling in popularity. I don't know how one would verify (or
dispute) the claim that Debian was the main source of Exim installs, and I'm
not sure that's a "problem" that needs fixing. (Also if you look more closely
at the survey, ~55% of responding MTAs didn't identify themselves and are thus
not counted in the statistics, which is a potential wide margin of error.)

> So yes, switching to postfix by default would reduce the workload of email
> servers around the globe (no need to burn cpu cycles and thus co2 to
> convert emails to quoted-printable).

The statistics quoted showed that Exim was most popular, so wouldn't switching
to Postfix by default actually be more CPU costly than the reverse? :-/ [I'm
not saying you're wrong, just that I don't see the logic in the argument.]




I administer both Postfix and Exim and greatly prefer Exim (specifically
exim4-daemon-heavy and using a single config file), but I wouldn't mind if the
default were Postifx. Whatever the default MTA is should IMHO be whatever DDs
supporting Debian think is the most supportable and "the best default".

I've likewise often wondered if a low-resource MTA like DMA or ssmtp could be
the default MTA for Desktop installs (and I've occasionally tried them), but
as has been discussed there seem to be some issues with the idea. In my case
for Desktops I want the local MTA to be able to handle sending local outbound
mail to a server via port 587 over TLS with authentication, to retry sending
at increasing time intervals, using a "queue runner" but without a daemon
listening, and to notify the sender on a permanent failure. Thusfar I've only
been able to find all of that in a full-fledged MTA.



[1] http://cr.yp.to/smtp/8bitmime.html

[2] http://www.securityspace.com/s_survey/data/man.201007/mxsurvey.html

-- Chris

--
Chris Knadle
Chris.Knadle@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74