
Hello,
I've fixed now the MODE reader problems in the debian package of mailman (using the cvs nntplib.py in Mailman/pythonlib)... And now I was wondering... what happens if the newserver does not accept connections when mailman wants to gateway a post from mail to news... is there a queueing mechanism for nntp just like there is for smtp in case the MTA is down ?
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> I've fixed now the MODE reader problems in the debian package
GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)...
GM> And now I was wondering... what happens if the newserver does
GM> not accept connections when mailman wants to gateway a post
GM> from mail to news... is there a queueing mechanism for nntp
GM> just like there is for smtp in case the MTA is down ?
Nope.

On Wed, 23 Feb 2000, Barry A. Warsaw wrote:
Ahm... perhaps the queue should be expanded to handle the NNTP case too then ? :) We don't want mails get lost on their way to newsgroups... :)
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> Ahm... perhaps the queue should be expanded to handle the NNTP
GM> case too then ? :) We don't want mails get lost on their way
GM> to newsgroups... :)
Except that Mailman's mail queuing is gone too now in the current CVS snapshot. The 1.1 implementation caused a terrible performance hit so I ripped it out, figuring a well configured MTA should be able to do that job, and do it better. It could be added back, but I haven't found a need. If someone wants to re-implement this against the current architecture, then I agree that nntp queuing should be part of the picture.
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
Ah, indeed, now mailman calls the MTA directly, and no smtp on port 25, right ? Perhaps there is a way to do something like that with the news server, so no need to talk nntp... with inn rnews seems to be the program to call, I don't know about other nntpservers...
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on
GM> port 25, right ?
Not quite. There are actually two interchangable modules which do delivery. Sendmail.py expects sendmail command line interface and uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 and blasts the message to the MTA. The latter does no queuing because it is my observation that 1) assuming the SMTP is running correctly, and 2) you are not doing DSN, the SMTP delivery happens asynchronously so still only be notified of errors by the bounce mechanism. 1.1 had all the queuing stuff because with DSN enabled (for those MTAs that support this ESMTP feature), delivery was synchronous and Mailman had to deal with exceptions instead of letting the MTA do it.
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
But it happened quite often that mailman couldn't connect to port 25 because sendmail wasn't accepting connections due to high load (loads above 25 are usual on this machine...), so queueing was needed anyway...
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> But it happened quite often that mailman couldn't connect to
GM> port 25 because sendmail wasn't accepting connections due to
GM> high load (loads above 25 are usual on this machine...), so
GM> queueing was needed anyway...
One possible solution is to use a better MTA (i.e. not sendmail). Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system.
Some form of queuing may still be necessary, but the previous implementation did not work. I don't have time to implement a new system myself but if someone wants to work on it, let me know.
-Barry

On Wed, 23 Feb 2000 13:15:35 -0500 (EST) bwarsaw <bwarsaw@cnri.reston.va.us> wrote:
"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway...
I potentially have the same problem -- the system occassionally gets very high load spikes, NOT due to MailMan, and Exim is configured to refuse connections when load gets too high.
Some form of queuing may still be necessary...
If the new system also allows mail to be delivered to the local MTA by handingit off to a sendmail executable, there should be little to no problem. I forget the new design tho.
-- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=--

On Wed, Feb 23, 2000 at 10:20:38AM -0800, J C Lawrence wrote:
I got loads of up to 60 when i used exim with Mailman 1.1 ... and then slowly the system started to die ... this was happening when sending about 20 approved posts to 1500+ users. I configured exim not to send out anything when getting a higher load than 5, but then my fcgi scripts would die anyway during posting approvals... Since i'm using postfix with MM 1.2, loads don't get any higher than 2 or 3. So i'm very happy with the changes in 1.2 :)
Ricardo.
--

"RK" == Ricardo Kustner <ricardo@rixhq.nu> writes:
RK> Since i'm using postfix with MM 1.2, loads don't get any
RK> higher than 2 or 3. So i'm very happy with the changes in 1.2
RK> :)
This is very good to hear! :)
Who else is using any of the 1.2 CVS snapshots?
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
Sorry, but I can't accept this answer :) It seems quite normal for _any_ MTA to refuse connections when the load is high (protect the machine from even higher loads), and it is quite normal for machines to have high load... you don't want to say "don't use mailman on machines wich have load above 5", do you ? :) I use it, and it works nicely :)
Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system.
Well... at least it took part of it. I'm really looking forward to the 1.2 release to test the performance :)
I don't really see the need of a new smtp queueing system if there is the possibility of executing the MTA directly, so I'm not asking anyone to do it now :) I'm just saying that queueing was _needed_ before, and it _worked_ (I was one of those who requested this feature when 1.0b4 lost almost all my digests, because of the high load early morning :)) So there is no problem now :)
And at the beginning of this thread I just warned that the same might be needed for the nntp server too, I didn't want to go into the smtp stuff now :)
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> I've fixed now the MODE reader problems in the debian package
GM> of mailman (using the cvs nntplib.py in Mailman/pythonlib)...
GM> And now I was wondering... what happens if the newserver does
GM> not accept connections when mailman wants to gateway a post
GM> from mail to news... is there a queueing mechanism for nntp
GM> just like there is for smtp in case the MTA is down ?
Nope.

On Wed, 23 Feb 2000, Barry A. Warsaw wrote:
Ahm... perhaps the queue should be expanded to handle the NNTP case too then ? :) We don't want mails get lost on their way to newsgroups... :)
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> Ahm... perhaps the queue should be expanded to handle the NNTP
GM> case too then ? :) We don't want mails get lost on their way
GM> to newsgroups... :)
Except that Mailman's mail queuing is gone too now in the current CVS snapshot. The 1.1 implementation caused a terrible performance hit so I ripped it out, figuring a well configured MTA should be able to do that job, and do it better. It could be added back, but I haven't found a need. If someone wants to re-implement this against the current architecture, then I agree that nntp queuing should be part of the picture.
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
Ah, indeed, now mailman calls the MTA directly, and no smtp on port 25, right ? Perhaps there is a way to do something like that with the news server, so no need to talk nntp... with inn rnews seems to be the program to call, I don't know about other nntpservers...
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> Ah, indeed, now mailman calls the MTA directly, and no smtp on
GM> port 25, right ?
Not quite. There are actually two interchangable modules which do delivery. Sendmail.py expects sendmail command line interface and uses os.popen() to invoke a subprocess. SMTPDirect.py opens port 25 and blasts the message to the MTA. The latter does no queuing because it is my observation that 1) assuming the SMTP is running correctly, and 2) you are not doing DSN, the SMTP delivery happens asynchronously so still only be notified of errors by the bounce mechanism. 1.1 had all the queuing stuff because with DSN enabled (for those MTAs that support this ESMTP feature), delivery was synchronous and Mailman had to deal with exceptions instead of letting the MTA do it.
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
But it happened quite often that mailman couldn't connect to port 25 because sendmail wasn't accepting connections due to high load (loads above 25 are usual on this machine...), so queueing was needed anyway...
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/

"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> But it happened quite often that mailman couldn't connect to
GM> port 25 because sendmail wasn't accepting connections due to
GM> high load (loads above 25 are usual on this machine...), so
GM> queueing was needed anyway...
One possible solution is to use a better MTA (i.e. not sendmail). Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system.
Some form of queuing may still be necessary, but the previous implementation did not work. I don't have time to implement a new system myself but if someone wants to work on it, let me know.
-Barry

On Wed, 23 Feb 2000 13:15:35 -0500 (EST) bwarsaw <bwarsaw@cnri.reston.va.us> wrote:
"GM" == Gergely Madarasz <gorgo@sztaki.hu> writes:
GM> But it happened quite often that mailman couldn't connect to GM> port 25 because sendmail wasn't accepting connections due to GM> high load (loads above 25 are usual on this machine...), so GM> queueing was needed anyway...
I potentially have the same problem -- the system occassionally gets very high load spikes, NOT due to MailMan, and Exim is configured to refuse connections when load gets too high.
Some form of queuing may still be necessary...
If the new system also allows mail to be delivered to the local MTA by handingit off to a sendmail executable, there should be little to no problem. I forget the new design tho.
-- J C Lawrence Home: claw@kanga.nu ----------(*) Other: coder@kanga.nu --=| A man is as sane as he is dangerous to his environment |=--

On Wed, Feb 23, 2000 at 10:20:38AM -0800, J C Lawrence wrote:
I got loads of up to 60 when i used exim with Mailman 1.1 ... and then slowly the system started to die ... this was happening when sending about 20 approved posts to 1500+ users. I configured exim not to send out anything when getting a higher load than 5, but then my fcgi scripts would die anyway during posting approvals... Since i'm using postfix with MM 1.2, loads don't get any higher than 2 or 3. So i'm very happy with the changes in 1.2 :)
Ricardo.
--

"RK" == Ricardo Kustner <ricardo@rixhq.nu> writes:
RK> Since i'm using postfix with MM 1.2, loads don't get any
RK> higher than 2 or 3. So i'm very happy with the changes in 1.2
RK> :)
This is very good to hear! :)
Who else is using any of the 1.2 CVS snapshots?
-Barry

On Wed, 23 Feb 2000 bwarsaw@cnri.reston.va.us wrote:
Sorry, but I can't accept this answer :) It seems quite normal for _any_ MTA to refuse connections when the load is high (protect the machine from even higher loads), and it is quite normal for machines to have high load... you don't want to say "don't use mailman on machines wich have load above 5", do you ? :) I use it, and it works nicely :)
Also, it's very possible that all that extra load was /caused/ by Mailman 1.1's queuing system.
Well... at least it took part of it. I'm really looking forward to the 1.2 release to test the performance :)
I don't really see the need of a new smtp queueing system if there is the possibility of executing the MTA directly, so I'm not asking anyone to do it now :) I'm just saying that queueing was _needed_ before, and it _worked_ (I was one of those who requested this feature when 1.0b4 lost almost all my digests, because of the high load early morning :)) So there is no problem now :)
And at the beginning of this thread I just warned that the same might be needed for the nntp server too, I didn't want to go into the smtp stuff now :)
-- Madarasz Gergely gorgo@sztaki.hu gorgo@linux.rulez.org It's practically impossible to look at a penguin and feel angry. Egy pingvinre gyakorlatilag lehetetlen haragosan nezni. HuLUG: http://mlf.linux.rulez.org/
participants (5)
-
Barry A. Warsaw
-
bwarsaw@cnri.reston.va.us
-
Gergely Madarasz
-
J C Lawrence
-
Ricardo Kustner