an opinion about using port 25
Something's been bothering me about the use of connecting to port 25 to deliver mail.
The goal of this endeavor, if i remember correctly, was to make mailman more portable in it's interface to MTA's. However, I think that it may well make that problem even worse, and that it lies outside what mailman should be doing.
Here's the argument:
mailman is a mailing list manager, not an MTA. If mailman developers decide to make delivery occur by SMTP, it has to do a LOT of the work that an MTA should be doing: queuing messages, retrying every so often, bouncing messages with appropriate messages under appropriate circumstances (like deferral messages every 4 hours, etc). All this should be the work of the MTA.
I also think that it will eventually exacerbate the problems of portability. As a part of test group for a new MTA, and an administrator of a qmail and a sendmail system, I know many work-years of labor goes into making an MTA be able to successfully deliver mail to all the other MTA's out there. This happens because different MTAs act very differently. There are still problems occuring between code as mature as qmail and other MTA's like certain microsoft based products (postoffice, LSOFT for nt, etc).
Do we really want to take on that burden?
There are other ways we could pursue this. There are basically 2 interfaces to an MTA mailman has to consider: updating aliases and delivering messages. For each of these interfaces, we could create a class hierarchy into which mailman's communication with new MTA's could be easily dropped. Such a structure, coupled with good documentation about how to add support for a new MTA seems to me like it will go further faster than putting a lot of work into SMTP transactions. Such an approach could even include smtp transactions, as a bonus for the daring or those who want mail delivered via a separate machine than the one on which the list manager resides.
As far as portability goes, and as far allowing mailman developers to focus on what a list manager should do, i really think that we would do better not depending on a universally reliable way of doing smtp transactions.
scott
On Thu, 23 Jul 1998, Scott wrote:
Something's been bothering me about the use of connecting to port 25 to deliver mail.
The goal of this endeavor, if i remember correctly, was to make mailman more portable in it's interface to MTA's. However, I think that it may well make that problem even worse, and that it lies outside what mailman should be doing.
Here's the argument:
mailman is a mailing list manager, not an MTA. If mailman developers decide to make delivery occur by SMTP, it has to do a LOT of the work that an MTA should be doing: queuing messages, retrying every so often, bouncing messages with appropriate messages under appropriate circumstances (like deferral messages every 4 hours, etc). All this should be the work of the MTA.
I also think that it will eventually exacerbate the problems of portability. As a part of test group for a new MTA, and an administrator of a qmail and a sendmail system, I know many work-years of labor goes into making an MTA be able to successfully deliver mail to all the other MTA's out there. This happens because different MTAs act very differently. There are still problems occuring between code as mature as qmail and other MTA's like certain microsoft based products (postoffice, LSOFT for nt, etc).
Do we really want to take on that burden?
Most definitely not! While i adamantly agree that Mailman should not be taking over the duties of an MTA, i do not believe that the recent change amounts to that.
The change is in the way that Mailman passes messages to the *local* MTA
- it uses SMTP instead of execing the MTA program. This falls far, far short of the general problem of delivery to a general host out on the net. The local MTA still handles all the challenges of getting the messages to the remote site!
There are a few significant benefits to this change. Directly contrary to Scott's next point, this drastically reduces portability burdens, because Mailman doesn't need to know *anything* about the specific MTA the local host is running, except that it obeys SMTP.
Another big benefit actually addresses the concerns Gergely Madarasz (urgently) raised yesterday. By using the specialized protocol, we don't need to expose any of the addresses or other incidentals to processing by the system shell. (The addresses were still being exposed in other popen's, further upstream, but i changed that in the development code, which should be released soon. I did a quick check for other os.popen's and and os.systems, and most looked safe, but we still have yet to do a thorough examination for security sake.)
There are related literal-mangling problems associated with exposing strings to the shell, which we also circumvent by direct protocol transmission.
This change may also enable us to exploit advanced SMTP capabilities, like the DSN mod that dan ohnesorg has proposed (inhibiting those annoying "Warning: not yet delivered after 4 hours, will keep trying (and telling you about it)" messages).
There are two drawbacks i can see to this approach, both easily surmountable.
One is that we *do* have to make provisions for systems where the MTA is not always available. This amounts to a drastically scaled-down version of the general delivery-to-an-unavailable host, and has none of the nuances of the more general problem. It is addressed in the working code.
The other is in handling delivery failures for destinations that happen to be local to the local MTA. These are the only deliveries that will be immediately recognized and refused. In this case we don't get the benefit of bounce messages to recognize bad recipients, and we have to make sure that the deliveries to valid recipients on the local host are not disrupted. The latter part has been taken care of in the working copy, and the former part has to be, but no great disruption is caused by it, in any case.
There are other ways we could pursue this. There are basically 2 interfaces to an MTA mailman has to consider: updating aliases and delivering messages. For each of these interfaces, we could create a class hierarchy into which mailman's communication with new MTA's could be easily dropped. Such a structure, coupled with good documentation about how to add support for a new MTA seems to me like it will go further faster than putting a lot of work into SMTP transactions. Such an approach could even include smtp transactions, as a bonus for the daring or those who want mail delivered via a separate machine than the one on which the list manager resides.
I don't agree. I'd rather solve, once and for all, connecting via SMTP to the local MTA, and having to handle queueing messages at times the MTA is unavailable, than developing a general plug in framework for a command-based interface with every MTA that's ever going to come down the road. That doesn't sound like an tidy problem to me.
Ken
On Fri, 24 Jul 1998, Ken Manheimer wrote:
The change is in the way that Mailman passes messages to the *local* MTA
- it uses SMTP instead of execing the MTA program. This falls far, far short of the general problem of delivery to a general host out on the net. The local MTA still handles all the challenges of getting the messages to the remote site!
There are a few significant benefits to this change. Directly contrary to Scott's next point, this drastically reduces portability burdens, because Mailman doesn't need to know *anything* about the specific MTA the local host is running, except that it obeys SMTP.
As I said in one of my previous postings... sendmail can be called from the command line to talk SMTP just like as connected to port 25 (sendmail -bs). I expect most other MTA's have the same features, with perhaps a bit different command line. So the only difference between using a local MTA and connecting to port 25 would be to open a two way pipe to 'sendmail -bs' or whatever (all sendmail-compatible mta-s, like exim or smail should have sendmail -bs) instead of connecting to port 25. All the rest is the same. So the only change needed here would be in the connect function of smtplib -> not a mailman issue but a python issue :) This would be much-much nicer than mailman trying to queue mails.
Greg
-- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/
On Sat, 25 Jul 1998, Gergely Madarasz wrote:
'sendmail -bs' or whatever (all sendmail-compatible mta-s, like exim or smail should have sendmail -bs) instead of connecting to port 25. All the
Of course, all these optional, depending on some parameters passed to smtplib or some defaults... with fallback to port 25.
-- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/
On 25 Jul 98, at 0:59, Gergely Madarasz wrote:
As I said in one of my previous postings... sendmail can be called from the command line to talk SMTP just like as connected to port 25 (sendmail -bs). I expect most other MTA's have the same features, with perhaps a bit different command line. So the only difference between using a local MTA and connecting to port 25 would be to open a two way pipe to 'sendmail -bs' or whatever (all sendmail-compatible mta-s, like exim or smail should have sendmail -bs) instead of connecting to port 25. All the rest is the same. So the only change needed here would be in the connect function of smtplib -> not a mailman issue but a python issue :) This would be much-much nicer than mailman trying to queue mails.
I have made patch, which uses delivery status notification. This will be probably included in some of next revision. I have not found some posibility, how detect ob MTA knows this feature, other than parsing respond to EHLO request. Delivery status notification is very nice feature, I am using only notify=failure and the list is shielded before warnigs like, mail was not delivered within 4 hours and so.
cheers dan
________________________________________DDDDDD
DD DD Dan Ohnesorg, supervisor on POWER
DD OOOO Dan@feld.cvut.cz
DD OODDOO Dep. of Power Engineering
DDDDDD OO CTU FEL Prague, Bohemia
OO OO work: +420 2 24352785;+420 2 24972109
OOOO home: +420 311 679679;+420 311 679311
________________________________________
Pesimista vidi v ementalskem syru jen ty diry.
On Sat, 25 Jul 1998, Dan Ohnesorg, admin of POWER wrote:
On 25 Jul 98, at 0:59, Gergely Madarasz wrote:
As I said in one of my previous postings... sendmail can be called from the command line to talk SMTP just like as connected to port 25 (sendmail -bs). I expect most other MTA's have the same features, with perhaps a bit different command line. So the only difference between using a local MTA and connecting to port 25 would be to open a two way pipe to 'sendmail -bs' or whatever (all sendmail-compatible mta-s, like exim or smail should have sendmail -bs) instead of connecting to port 25. All the rest is the same. So the only change needed here would be in the connect function of smtplib -> not a mailman issue but a python issue :) This would be much-much nicer than mailman trying to queue mails.
I have made patch, which uses delivery status notification. This will be probably included in some of next revision. I have not found some posibility, how detect ob MTA knows this feature, other than parsing respond to EHLO request. Delivery status notification is very nice feature, I am using only notify=failure and the list is shielded before warnigs like, mail was not delivered within 4 hours and so.
hmmm... Well, parsing the ehlo responce _is_ the proper way todetect support for any esmtp feature, like DSN. I also want to point out that the latest version of the smtplib included in the python distribution (which is NOT the same as mailman's smtplib) has full ESMTP support. This latest version (http://www.integral.org/~dragondm/python/smtplib.py) will be the smtplib included in the python 1.5.2 distribution. Also, unless someone else does it first, as soon as I can get the latest 1.0b5 code [1], I will be submitting patches for mailman to substitute this smtplib for the one currently included with mailman. The reason for that is that way, any improvements/bugfixes to the module could eventually be folded into the next python distribution.
Now, as for 'sendmail -bs' well, yes you could do that. with thepython smtplib, it would be a matter of subclassing smtplib.SMTP and overriding the methods that open, read from, and write to the socket, however I would reccomend against it. If the local smtp listener is refusing messages, 9 times out of ten there is a good reason for it (perhaps the server's overloaded). The best choice is for mailman to queue the message, and fer a cron job or such to try again later. Port 25 smtp is the 'universal' interface for mta's, and trying to cater to everything else out there is asking fer coding nightmares.
Notes: [1] Erm, whhen's this going to be out? What happened to 'Release early, release often?' I've had things I would have liked to have coded and submitted by now, but I've been holding off on fer nearly a month wating for the latest code. Mebbe we need stable/unstable tracks like the linux kernel? With unstable releases VERY often.
-The Dragon De Monsyne
On Mon, Jul 27, 1998 at 05:20:18AM -0500, The Dragon De Monsyne wrote:
Notes: [1] Erm, whhen's this going to be out? What happened to 'Release early, release often?' I've had things I would have liked to have coded and submitted by now, but I've been holding off on fer nearly a month wating for the latest code. Mebbe we need stable/unstable tracks like the linux kernel? With unstable releases VERY often.
I almost resent this comment. I'd expect you to have a bit more consideration and tolerance. We said we'd *try* to release last week, and once we did, we'd release more often. However, we were not willing to release at all until we got something that worked for some reasonable value of working. Please also be considerate to the fact that we don't have as much time as we'd like to work on this project.
I'm sorry if you feel frustrated; however, we have said in the past that we would provide you with a more current snapshot on request. I would not have refused any resonable request for a snapshot from someone interested in helping out. Some sort of private communication would have been a far more apropriate forum, until that avenue proved futile.
On Sat, Jul 25, 1998 at 12:59:24AM +0200, Gergely Madarasz wrote:
As I said in one of my previous postings... sendmail can be called from the command line to talk SMTP just like as connected to port 25 (sendmail -bs). I expect most other MTA's have the same features, with perhaps a bit different command line.
That's definitely not the case for a few MTA's, and it's definitely not something I would want to count on even if it were.
On Sun, 26 Jul 1998, John Viega wrote:
On Sat, Jul 25, 1998 at 12:59:24AM +0200, Gergely Madarasz wrote:
As I said in one of my previous postings... sendmail can be called from the command line to talk SMTP just like as connected to port 25 (sendmail -bs). I expect most other MTA's have the same features, with perhaps a bit different command line.
That's definitely not the case for a few MTA's, and it's definitely not something I would want to count on even if it were.
Don't count on it, just make it an option :) I'd use it here and now. Unfortunatelly I'm just a beginner in python...
-- Madarasz Gergely gorgo@caesar.elte.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://www.cab.u-szeged.hu/local/linux/
participants (6)
-
Dan Ohnesorg, admin of POWER -
Gergely Madarasz -
John Viega -
Ken Manheimer -
Scott -
The Dragon De Monsyne