popen(), sendmail: Success?

[Chuck Swiger]

Thomas Bellman wrote:
> Chuck Swiger <cswiger at mac.com> wrote:
>> If you plan on sending any significant amount of mail, it is
>> recommended that your code talk SMTP to port 25 and pipeline all of
>> the messages you need to send in one continuous stream.  If you
>> fork a sendmail subprocess for each message you send, that will
>> create a wastefully huge amount of system load.
> 
> If you write your application to talk SMTP directly, instead of using
> the proper Unix API for sending mail (i.e, calling 
> /usr/lib/sendmail), you have a number of problems.

Which "proper Unix API"?  The closest thing is RFC-2822, since 
/usr/lib/sendmail doesn't exist on many Unix platforms:

18-sec% which -a sendmail
/usr/sbin/sendmail
19-sec% uname -a
FreeBSD sec.codefab.com 4.8-STABLE FreeBSD 4.8-STABLE #0: Wed Apr  9
08:07:21 EDT 2003     root at sec.codefab.com:/usr/obj/usr/src/sys/SEC  i386
20-sec% ls -l /usr/lib/sendmail
ls: /usr/lib/sendmail: No such file or directory

> First, which machine should you talk SMTP with?  Localhost?

A site which intends to handle a non-trivial amount of electronic mail
needs to have one or more designated mail servers, very commonly
identified via MX records in the DNS.  Sites with enough traffic likely
have both inbound and outbound SMTP boxes.

> Not portable.  There are lots of Unix machines which do not have any 
> MTA running listening to port 25.

True, but I don't recall insisting that one need to do SMTP specificly 
to localhost:25.  If you want to use another machine to queue the mail,
such as a dedicated mail server, instead of locahost, that's fine by me.

> Every machine I administrate run Sendmail in queue mode only (except
> for the mail servers, of course).

By this you mean, "you do not configure your clients run sendmail as a 
daemon listening on port 25" and not "I always run sendmail in 'actual 
delivery is deferred until the next queue run' delivery mode (-odq)"...?

> Thus, you need your application to be configurable for
> the mail server to use.  If the system changes, and another mail 
> server should be used, the sysadmin might not remember, or know, that
> that particular application must be reconfigured.

That's why sysadmins need to document how systems are configured and 
note what the dependencies are.  Just as developers need to write 
applications to be configurable enough to select a MTA-- possibly not 
sendmail, and possibly not running on the localhost-- to handle email 
generated on behalf of the system.

> It becomes even worse if it is an application where each user on the 
> machine has their own configuration, like Netscape or Mozilla...

Try specifying "option smtp-server _IP_;" within your DHCP server's 
configuration, or perhaps use port forwarding, IPFW divert sockets, etc 
so that connections made to localhost:25 get forwarded to a machine 
which will handle such mail.

> Second, you better be prepared to queue the mails in case of network
> or mail server problems and retry sending them at a later time.

Certainly.  If your app can hand email off to a local MTA which is 
always available, it doesn't have to queue the mail itself-- let the MTA 
do that.

[ ... ]
> If you talk SMTP directly, you run the risk of skipping any local 
> policy for sending email, like masquerading as a specific domain 
> instead of as the FQDN of the local host, or a number of other 
> settings.

I think you may have misunderstood my suggestion: I don't believe or 
recommend that someone writing a python program should perform MX 
lookups and try to do SMTP to the mail server of the addressee, retry 
undelivered messages, etc.

Talking SMTP to a local mailserver (25/tmp, or via the MSP, 587/tcp) 
lets your local mailserver perform masquerading, mailertable lookups, 
virus scanning, forwarding mail through a firewall which would otherwise 
block outbound SMTP, etc.

> There are many subtleties involved in the game of being, or 
> pretending to be, an MTA, many of which are not handled by the 
> smtplib Python module, and many of them are subject to local policy,
> decided by the system administrator.  Sendmail, Postfix and Qmail are
> written to take care of those; you are unlikely to do a better job
> when writing an application whose primary purpose *isn't* being an
> MTA.

Agreed.  Where we seem to disagree is how to get the email to an MTA:

I suggest that invoking /usr/lib/sendmail -t via popen() is a poor 
approach for sending more than a trivial amount of mail.  By any chance, 
Thomas, have you ever tried delivering more than 1000 messages or so at 
a time by forking sendmail subprocesses?  :-)

> I know many other sysadmins that run their systems similarly to the
> way I run mine.  Any application that talks SMTP on their own instead
> of calling /usr/lib/sendmail to do it, is considered broken by us..

If doing so makes you happy, I guess that's okay.  Certainly, there are 
less reasonable positions that one could insist upon.

> So, when you write any program that sends email, at *least* give the
> option of configuring it to use /usr/lib/sendmail, and preferably to
> use that by default.  It is at least as likely to work out of the box
> as any other setting is.

Unless you write code for Windows, or Unix flavors that use a different 
path to sendmail, or use qmail/smail/postfix/etc _instead_ of sendmail.

[ Or M$-Exchange, DEC's Pathworks for VMS SMTP connector, Cisco's 
wretched SMTP firewall proxy (MailGuard?), or any number of other things 
that handle SMTP the way a "Do Not Fold, Spindle, or Mutilate" stamp on 
a real letter seems to be handled: ie, by redirecting such email to the 
virtual equivalent of a paper shredder.  :-) ]

Portability sometimes means more than just "This works under Red Hat, 
SuSE, and Debian."

-- 
-Chuck