From norbert.aschendorff at yahoo.de  Thu Nov  1 09:54:50 2012
From: norbert.aschendorff at yahoo.de (Norbert Aschendorff)
Date: Thu, 01 Nov 2012 09:54:50 +0100
Subject: [Mailman-Users] DEFAULT_EMAIL_HOST
Message-ID: <509238DA.9090102@yahoo.de>

Hi all,

I wanted to figure out which effect DEFAULT_EMAIL_HOST has but I didn't
have much success. The only place where I found this variable was
add_virtualhost() in mailman-install.pdf and in Defaults.py.

My question: Does DEFAULT_EMAIL_HOST have an effect on the mailman
instance when I am not using (e.g. Postfix) virtual hosts? Does it have
an effect _when_ I'm using virtual domains? And if yes, which? Or should
it always be set to a sensible value? Does it influence to which
hostname the "aliases" file belongs? And if yes, does that make sense?

I ask because when I change the value of DEFAULT_EMAIL_HOST, nothing
happens except of some address changes on the web interface. Sending
mails to the lists works without any problems.

I'm pretty confused :)

Norbert

From mark at msapiro.net  Fri Nov  2 03:35:33 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 1 Nov 2012 19:35:33 -0700
Subject: [Mailman-Users] DEFAULT_EMAIL_HOST
In-Reply-To: <509238DA.9090102@yahoo.de>
Message-ID: <PC19520121101193533006231b1709f@MSAPIRO>

Norbert Aschendorff wrote:
>
>I wanted to figure out which effect DEFAULT_EMAIL_HOST has but I didn't
>have much success. The only place where I found this variable was
>add_virtualhost() in mailman-install.pdf and in Defaults.py.


You missed

cron/mailpasswds
Mailman/MailList.py
Mailman/Cgi/create.py
bin/fix_url.py
bin/newlist

and also references to the VIRTUAL_HOSTS dictionary which always
contains at leasy the one add_virtualhost(DEFAULT_URL_HOST,
DEFAULT_EMAIL_HOST) entry in

Mailman/Utils.py
Mailman/Cgi/create.py
bin/fix_url.py
bin/newlist


>My question: Does DEFAULT_EMAIL_HOST have an effect on the mailman
>instance when I am not using (e.g. Postfix) virtual hosts?


Yes.


>Does it have
>an effect _when_ I'm using virtual domains?


Yes.


>And if yes, which?


See below.


>Or should
>it always be set to a sensible value?


Yes.


>Does it influence to which
>hostname the "aliases" file belongs? And if yes, does that make sense?



What do you mean by 'to which hostname the "aliases" file belongs?'. I
don't think the "aliases" file "belongs" to any host. Please clarify?


>I ask because when I change the value of DEFAULT_EMAIL_HOST, nothing
>happens except of some address changes on the web interface. Sending
>mails to the lists works without any problems.


DEFAULT_EMAIL_HOST affects the value assigned to the list's host_name
attribute at list creation time depending on exactly how and with what
options the list is created.

The list's host_name in turn is used as the domain part of every list
email address exposed by Mailman. It also determines in a Postfix
integration environment whether virtual mappings for list addresses
are created in data/virtual-mailman depending on whether or not the
lists host_name matches an entry in POSTFIX_STYLE_VIRTUAL_DOMAINS.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From blablubb at divepage.net  Fri Nov  2 11:24:03 2012
From: blablubb at divepage.net (Michael)
Date: Fri, 02 Nov 2012 11:24:03 +0100
Subject: [Mailman-Users] Login to private archive
Message-ID: <c71abe414aac956b1b5d47b03b540ca2@divepage.net>

 

Hello mailman-users, 

I am new to mailman and ran into a problem
with my fresh 2.1.13 installation of mailman. 

My first created list is
defined with private option for archive, so list members have to login
before they can view the archive.
The login for users works well on the
edit profile page, but not on the archive page. 

After entering the
login credencials I receive a 301 redirect to the archive page.
I
consider there should be the archive shown now.
Instead there is only
the login form again.
Taking a look at the cookies in the browser I can
see that there has no cookie been set. 

Do you have any idea about this
problem? 

Regards from Augsburg, Germany,
Michael 

From mark at msapiro.net  Fri Nov  2 18:32:44 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 2 Nov 2012 10:32:44 -0700
Subject: [Mailman-Users] Login to private archive
In-Reply-To: <c71abe414aac956b1b5d47b03b540ca2@divepage.net>
Message-ID: <PC195201211021032440328557ef434@MSAPIRO>

Michael wrote:
>
>After entering the
>login credencials I receive a 301 redirect to the archive page.
>I
>consider there should be the archive shown now.
>Instead there is only
>the login form again.


See the FAQ at <http://wiki.list.org/x/ioA9>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From misterbhatt at gmail.com  Sat Nov  3 15:30:43 2012
From: misterbhatt at gmail.com (Amit Bhatt)
Date: Sat, 3 Nov 2012 20:00:43 +0530
Subject: [Mailman-Users] error message while adding members
Message-ID: <4D129683B69C474EB67E65FB55DDB43F@amit>

Hello experts,

I am unable to add or subscribe any member using mass subscription option under membership management of my mailing list. I am facing the below error since yesterday:

Error: The form lifetime has expired. (request forgery check)

Please suggest me the remedy for it.

Thanks,

Amit Bhatt

From mark at msapiro.net  Sat Nov  3 15:52:56 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 3 Nov 2012 07:52:56 -0700
Subject: [Mailman-Users] error message while adding members
In-Reply-To: <4D129683B69C474EB67E65FB55DDB43F@amit>
Message-ID: <PC19520121103075256089036e2bc32@MSAPIRO>

Amit Bhatt wrote:
>
>I am unable to add or subscribe any member using mass subscription option under membership management of my mailing list. I am facing the below error since yesterday:
>
>Error: The form lifetime has expired. (request forgery check)


This message comes from the new in 2.1.15 CSRF mitigation features.

When did you upgrade to 2.1.15?

Have you set a value for FORM_LIFETIME in mm_cfg.py? If so, what? (The
default setting in Defaults.py is 1 hour.)

What the message is telling you is that the mass subscribe form you are
submitting was retrieved from the host longer than FORM_LIFETIME
before it was submitted. If this is not the case, there is some issue
in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
is applicable.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From misterbhatt at gmail.com  Sat Nov  3 17:16:20 2012
From: misterbhatt at gmail.com (Amit Bhatt)
Date: Sat, 3 Nov 2012 21:46:20 +0530
Subject: [Mailman-Users] error message while adding members
References: <PC19520121103075256089036e2bc32@MSAPIRO>
Message-ID: <36C36184026C4ED798B3AE61A683FA1B@amit>

yes, surprisingly, the mailing list has been updated in to new version 
1.1.15 without any effort made by me.
I am surprised because I have never tried to update the list, is it possible 
that the updation is done from our web server end?

I have noticed this change after seeing your reply.

Regards,

Amit Bhatt
----- Original Message ----- 
From: "Mark Sapiro" <mark at msapiro.net>
To: "Amit Bhatt" <misterbhatt at gmail.com>; <Mailman-Users at python.org>
Sent: Saturday, November 03, 2012 8:22 PM
Subject: Re: [Mailman-Users] error message while adding members


> Amit Bhatt wrote:
>>
>>I am unable to add or subscribe any member using mass subscription option 
>>under membership management of my mailing list. I am facing the below 
>>error since yesterday:
>>
>>Error: The form lifetime has expired. (request forgery check)
>
>
> This message comes from the new in 2.1.15 CSRF mitigation features.
>
> When did you upgrade to 2.1.15?
>
> Have you set a value for FORM_LIFETIME in mm_cfg.py? If so, what? (The
> default setting in Defaults.py is 1 hour.)
>
> What the message is telling you is that the mass subscribe form you are
> submitting was retrieved from the host longer than FORM_LIFETIME
> before it was submitted. If this is not the case, there is some issue
> in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
> is applicable.
>
> -- 
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> 


From misterbhatt at gmail.com  Sat Nov  3 17:17:20 2012
From: misterbhatt at gmail.com (Amit Bhatt)
Date: Sat, 3 Nov 2012 21:47:20 +0530
Subject: [Mailman-Users] error message while adding members
Message-ID: <1D46F30AE5344345A3D29EF449FE80EE@amit>

I am sorry, it is version 2.1.1.15.

----- Original Message ----- 
From: "Amit Bhatt" <misterbhatt at gmail.com>
To: "Mark Sapiro" <mark at msapiro.net>; <Mailman-Users at python.org>
Sent: Saturday, November 03, 2012 9:46 PM
Subject: Re: [Mailman-Users] error message while adding members


> yes, surprisingly, the mailing list has been updated in to new version 
> 1.1.15 without any effort made by me.
> I am surprised because I have never tried to update the list, is it 
> possible that the updation is done from our web server end?
>
> I have noticed this change after seeing your reply.
>
> Regards,
>
> Amit Bhatt
> ----- Original Message ----- 
> From: "Mark Sapiro" <mark at msapiro.net>
> To: "Amit Bhatt" <misterbhatt at gmail.com>; <Mailman-Users at python.org>
> Sent: Saturday, November 03, 2012 8:22 PM
> Subject: Re: [Mailman-Users] error message while adding members
>
>
>> Amit Bhatt wrote:
>>>
>>>I am unable to add or subscribe any member using mass subscription option 
>>>under membership management of my mailing list. I am facing the below 
>>>error since yesterday:
>>>
>>>Error: The form lifetime has expired. (request forgery check)
>>
>>
>> This message comes from the new in 2.1.15 CSRF mitigation features.
>>
>> When did you upgrade to 2.1.15?
>>
>> Have you set a value for FORM_LIFETIME in mm_cfg.py? If so, what? (The
>> default setting in Defaults.py is 1 hour.)
>>
>> What the message is telling you is that the mass subscribe form you are
>> submitting was retrieved from the host longer than FORM_LIFETIME
>> before it was submitted. If this is not the case, there is some issue
>> in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
>> is applicable.
>>
>> -- 
>> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>> San Francisco Bay Area, California    better use your sense - B. Dylan
>>
> 


From fmouse-mailman at fmp.com  Sat Nov  3 17:51:16 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Sat, 03 Nov 2012 11:51:16 -0500
Subject: [Mailman-Users] error message while adding members
In-Reply-To: <1D46F30AE5344345A3D29EF449FE80EE@amit>
References: <1D46F30AE5344345A3D29EF449FE80EE@amit>
Message-ID: <1351961476.29611.155.camel@pudina.fmp.com>

On Sat, 2012-11-03 at 21:47 +0530, Amit Bhatt wrote:
> I am sorry, it is version 2.1.1.15.
> 
I believe the form timeout feature, and the related error message, were
introduced in 2.1.15 and aren't present in earlier versions.  Mailman
uses the standard version numbering scheme of Major.Minor.Revision and a
4th number, if it's there, may possibly be a distribution (downstream)
number.  From a cmd line:

~mailman/bin/version

Will give you the correct number, assuming you're using a Unix-like OS.

Also http://<server_name>/mailman/listinfo will show it to you at the
bottom of the page.

-- 
Lindsay Haisley       | "Real programmers use butterflies"
FMP Computer Services |
512-259-1190          |       - xkcd
http://www.fmp.com    |


From fmouse-mailman at fmp.com  Sat Nov  3 18:06:42 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Sat, 03 Nov 2012 12:06:42 -0500
Subject: [Mailman-Users] error message while adding members
In-Reply-To: <PC19520121103075256089036e2bc32@MSAPIRO>
References: <PC19520121103075256089036e2bc32@MSAPIRO>
Message-ID: <1351962402.29611.166.camel@pudina.fmp.com>

On Sat, 2012-11-03 at 07:52 -0700, Mark Sapiro wrote:
> What the message is telling you is that the mass subscribe form you are
> submitting was retrieved from the host longer than FORM_LIFETIME
> before it was submitted. If this is not the case, there is some issue
> in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
> is applicable.

Could this be caused by a web proxy that's improperly caching the form
page and serving it to the client in spite of the fact that csrf_token
has changed?  Or improperly indicating to the client that the page
hasn't changed so that the client displays from its its own cache
instead of requesting it again from the server?

Amit, I think a good test here would be to clear your browser's cache,
reload the subscription form, and see if you get the same error.  The
experts may have a better suggestion, but this is what I'd try.

-- 
Lindsay Haisley       | "In an open world, who needs  
FMP Computer Services |    Windows or Gates"
512-259-1190          |
http://www.fmp.com    |


From misterbhatt at gmail.com  Sat Nov  3 18:49:07 2012
From: misterbhatt at gmail.com (Amit Bhatt)
Date: Sat, 3 Nov 2012 23:19:07 +0530
Subject: [Mailman-Users] error message while adding members
References: <PC19520121103075256089036e2bc32@MSAPIRO>
	<1351962402.29611.166.camel@pudina.fmp.com>
Message-ID: <C4301E1A1463454D8345BAB19D97FE7E@amit>

Dear Lindsay,

Thanks for your excellent suggestion, it has worked for me!

Now the subscription is being done smoothly.

Regards,

Amit Bhatt
----- Original Message ----- 
From: "Lindsay Haisley" <fmouse-mailman at fmp.com>
To: <mailman-users at python.org>
Sent: Saturday, November 03, 2012 10:36 PM
Subject: Re: [Mailman-Users] error message while adding members


> On Sat, 2012-11-03 at 07:52 -0700, Mark Sapiro wrote:
>> What the message is telling you is that the mass subscribe form you are
>> submitting was retrieved from the host longer than FORM_LIFETIME
>> before it was submitted. If this is not the case, there is some issue
>> in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
>> is applicable.
>
> Could this be caused by a web proxy that's improperly caching the form
> page and serving it to the client in spite of the fact that csrf_token
> has changed?  Or improperly indicating to the client that the page
> hasn't changed so that the client displays from its its own cache
> instead of requesting it again from the server?
>
> Amit, I think a good test here would be to clear your browser's cache,
> reload the subscription form, and see if you get the same error.  The
> experts may have a better suggestion, but this is what I'd try.
>
> -- 
> Lindsay Haisley       | "In an open world, who needs
> FMP Computer Services |    Windows or Gates"
> 512-259-1190          |
> http://www.fmp.com    |
>
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> http://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: 
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: 
> http://mail.python.org/mailman/options/mailman-users/misterbhatt%40gmail.com 


From fmouse-mailman at fmp.com  Sat Nov  3 19:54:02 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Sat, 03 Nov 2012 13:54:02 -0500
Subject: [Mailman-Users] error message while adding members
In-Reply-To: <C4301E1A1463454D8345BAB19D97FE7E@amit>
References: <PC19520121103075256089036e2bc32@MSAPIRO>
	<1351962402.29611.166.camel@pudina.fmp.com>
	<C4301E1A1463454D8345BAB19D97FE7E@amit>
Message-ID: <1351968842.29611.202.camel@pudina.fmp.com>

On Sat, 2012-11-03 at 23:19 +0530, Amit Bhatt wrote:
> Dear Lindsay,
> 
> Thanks for your excellent suggestion, it has worked for me!
> 
> Now the subscription is being done smoothly.

This addresses the symptom, but not the underlying problem, which may
recur.  HTTP caching can be fairly complex, and your local browser cache
is simply the last link in the chain.  Clearing your browser's cache
each time before loading this form will avoid the problem, but not solve
it, and it shouldn't be necessary.

Deleting your browser cache forces a refresh from the server, or proxy
server, but the underlying problem is that your browser _thinks_ that it
has the most recent version of the form page in its cache, so it
displays it.  Mailman's CGI mechanism that issues the form page should
always inform the user agent, or proxy server, that the page is new, so
that any agent in the chain, proxy server or browser, will discard a
cached version and reload it.  So what's broken here is the exchange of
this information.  I suppose this _could_ be a browser problem, but IMHO
it's more likely to be a problem further back toward the server, or on
the server itself.

Every mass subscription page form has a token, labeled "csrf_token", the
value of which is a string of letters and numbers, and is different each
time the page is generated by the CGI script.  This token is sent back
to Mailman when you submit the form.  This token is interpreted by
Mailman, which uses it to determine how long ago the page was generated,
and to refuse it if the page is older than FORM_LIFETIME, set in
Defaults.py or mm_cfg.py.  So because this token changes with every
service of this form, the page is always "new" and every element of the
connection between the CGI script and your browser should be told that
this is the case.

A couple of questions might help to understand this:

*  Are you using a HTTP proxy server?  This information would have been
intentionally set in your browser's configuration.  If so, do you know
anything about this proxy server?

*  What kind and version of browser are you using?

*  On what kind of system is Mailman running and what kind (Apache,
MSIIS, etc.) and version is the web server?

> Regards,
> 
> Amit Bhatt
> ----- Original Message ----- 
> From: "Lindsay Haisley" <fmouse-mailman at fmp.com>
> To: <mailman-users at python.org>
> Sent: Saturday, November 03, 2012 10:36 PM
> Subject: Re: [Mailman-Users] error message while adding members
> 
> 
> > On Sat, 2012-11-03 at 07:52 -0700, Mark Sapiro wrote:
> >> What the message is telling you is that the mass subscribe form you are
> >> submitting was retrieved from the host longer than FORM_LIFETIME
> >> before it was submitted. If this is not the case, there is some issue
> >> in your web server. Perhaps the FAQ at <http://wiki.list.org/x/ioA9>
> >> is applicable.
> >
> > Could this be caused by a web proxy that's improperly caching the form
> > page and serving it to the client in spite of the fact that csrf_token
> > has changed?  Or improperly indicating to the client that the page
> > hasn't changed so that the client displays from its its own cache
> > instead of requesting it again from the server?
> >
> > Amit, I think a good test here would be to clear your browser's cache,
> > reload the subscription form, and see if you get the same error.  The
> > experts may have a better suggestion, but this is what I'd try.
> >
> > -- 
> > Lindsay Haisley       | "In an open world, who needs
> > FMP Computer Services |    Windows or Gates"
> > 512-259-1190          |
> > http://www.fmp.com    |
> >
> > ------------------------------------------------------
> > Mailman-Users mailing list Mailman-Users at python.org
> > http://mail.python.org/mailman/listinfo/mailman-users
> > Mailman FAQ: http://wiki.list.org/x/AgA3
> > Security Policy: http://wiki.list.org/x/QIA9
> > Searchable Archives: 
> > http://www.mail-archive.com/mailman-users%40python.org/
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/mailman-users/misterbhatt%40gmail.com 



From khillo100 at hotmail.com  Sun Nov  4 01:44:09 2012
From: khillo100 at hotmail.com (Khalil Abbas)
Date: Sun, 4 Nov 2012 02:44:09 +0200
Subject: [Mailman-Users] Moderation issue..
Message-ID: <DUB118-DS9EA41CA8E3E5968911297FB650@phx.gbl>

Hello ..

This is the second time that happens with me and I changed the server the first time.. I?ve been using my lists for months without any problems, I post to all lists from one address ?which is moderated- and I accept all messages manually from the web admin..

but then after sending to the lists, the messages does not appear waiting for moderation although maillogs show that they?ve been delivered to the lists addresses.. also I noticed a warning that I used to see when thing were working fine, I don?t know if it?s the cause of the issue..

the mail log is below..

please advise..

Thanks..


Nov  4 08:44:19 ns1 postfix/cleanup[4236]: 01AD818A0041: message-id=<011a01cdba1c$d1d06ac0$75714040$@domain.com>
Nov  4 08:44:19 ns1 postfix/qmgr[3917]: 01AD818A0041: from=<dailymail at domain.com>, size=106552, nrcpt=10 (queue active)
Nov  4 08:44:19 ns1 postfix/smtpd[4227]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 Ok: queued as 01AD818A0041; from=<dailymail at domain.com> to=<lista at domain.com> proto=ESMTP helo=<KhalilAbbasPC>
Nov  4 08:44:19 ns1 postfix/smtpd[4235]: disconnect from localhost.localdomain[127.0.0.1]
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains
Nov  4 08:44:19 ns1 /usr/lib64/plesk-9.0/psa-pc-remote[19459]: Message aborted.
Nov  4 08:44:19 ns1 postfix/pipe[4260]: 01AD818A0041: to=<listc at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4266]: 01AD818A0041: to=<listf at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4258]: 01AD818A0041: to=<listb at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4262]: 01AD818A0041: to=<listd at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4264]: 01AD818A0041: to=<liste at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4272]: 01AD818A0041: to=<listi at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4270]: 01AD818A0041: to=<listh at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4257]: 01AD818A0041: to=<lista at domain.com>, relay=mailman, delay=6.8, delays=6.3/0/0/0.51, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4268]: 01AD818A0041: to=<listg at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/pipe[4274]: 01AD818A0041: to=<listj at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.51, dsn=2.0.0, status=sent (delivered via mailman service)
Nov  4 08:44:19 ns1 postfix/qmgr[3917]: 01AD818A0041: removed

From mark at msapiro.net  Sun Nov  4 06:36:51 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 03 Nov 2012 22:36:51 -0700
Subject: [Mailman-Users] Moderation issue..
In-Reply-To: <DUB118-DS9EA41CA8E3E5968911297FB650@phx.gbl>
References: <DUB118-DS9EA41CA8E3E5968911297FB650@phx.gbl>
Message-ID: <5095FEF3.8050909@msapiro.net>

On 11/3/2012 5:44 PM, Khalil Abbas wrote:
> Hello ..
> 
> This is the second time that happens with me and I changed the server the first time.. I?ve been using my lists for months without any problems, I post to all lists from one address ?which is moderated- and I accept all messages manually from the web admin..
> 
> but then after sending to the lists, the messages does not appear waiting for moderation although maillogs show that they?ve been delivered to the lists addresses.. also I noticed a warning that I used to see when thing were working fine, I don?t know if it?s the cause of the issue..


What are you saying? Are you saying that the message was delivered to
the list members without being held for approval, or that the message
just "disappeared". It seems like the latter and I will respond based on
that.


> the mail log is below..

> Nov  4 08:44:19 ns1 postfix/trivial-rewrite[4230]: warning: do not list domain domain.com in BOTH mydestination and virtual_mailbox_domains


This message from Postfix says your Postfix configuration is in error.
domain.com is either a local domain or a virtual domain. You have told
Postfix it is both which is wrong.

> Nov  4 08:44:19 ns1 postfix/pipe[4260]: 01AD818A0041: to=<listc at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4266]: 01AD818A0041: to=<listf at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4258]: 01AD818A0041: to=<listb at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4262]: 01AD818A0041: to=<listd at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4264]: 01AD818A0041: to=<liste at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4272]: 01AD818A0041: to=<listi at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4270]: 01AD818A0041: to=<listh at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.49, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4257]: 01AD818A0041: to=<lista at domain.com>, relay=mailman, delay=6.8, delays=6.3/0/0/0.51, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4268]: 01AD818A0041: to=<listg at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.01/0/0.5, dsn=2.0.0, status=sent (delivered via mailman service)
> Nov  4 08:44:19 ns1 postfix/pipe[4274]: 01AD818A0041: to=<listj at domain.com>, relay=mailman, delay=6.8, delays=6.3/0.02/0/0.51, dsn=2.0.0, status=sent (delivered via mailman service)


So your message was delivered via the "mailman" service. Presumably this
is postfix_to_mailman.py, and presumably postfix_to_mailman.py actually
queued the message for Mailman. If that's true, and it disappeared after
that, there will be evidence in Mailman's logs. So what's in Mailman's
logs (particularly 'error' and 'vette')? Also, depending on what 3rd
party packager provided your version of postfix_to_mailman.py, it would
probably log any problems it found to its stderr which should result in
their appearing in the Postfix log, so again, it probably did the
expected delivery.

Other possibilities include things like IncomingRunner not running. See
the FAQ at <http://wiki.list.org/x/A4E9>, but ignore the part about
aliases because it's not applicable in your environment.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Sat Nov  3 23:47:45 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Sat, 03 Nov 2012 15:47:45 -0700
Subject: [Mailman-Users] Delete all members with nomail[B]
Message-ID: <50959F11.2010509@agliodbs.com>

Howdy,

I have a mailman list with 3200 members, about 900 of which are now
nomail[B] from bounces.  I'd like to know how to purge all the nomail[B]
members from the list -- without spending an entire day doing it through
the GUI.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From eviljafar at gmail.com  Sun Nov  4 04:19:41 2012
From: eviljafar at gmail.com (Anthony Wilson (Jaf))
Date: Sat, 3 Nov 2012 23:19:41 -0400
Subject: [Mailman-Users] Mailman archive - archive date does not match the
 date the message was posted
Message-ID: <CAEC=f8f6HZzW5F8Ppj7VWji=BQmK1fCf8RAbz7AjqxgyE=pMGA@mail.gmail.com>

There is an existing Mailman list with an archive which needs to be moved
to a different server, however it seems that the archive is a bit messed
up. I assume that the list was set up in 2006, but for some reason there is
a 2002 archive, and all of the posts in there seem to have been posted in
2012. It's a public archive so I assume there's no harm in posting a link
to it - https://secure.neap.net/pipermail/montrealers/.  Is there some way
to fix this?

From mark at msapiro.net  Sun Nov  4 17:32:10 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 4 Nov 2012 08:32:10 -0800
Subject: [Mailman-Users] Delete all members with nomail[B]
In-Reply-To: <50959F11.2010509@agliodbs.com>
Message-ID: <PC195201211040832100796a880ec26@MSAPIRO>

Josh Berkus wrote:
>
>I have a mailman list with 3200 members, about 900 of which are now
>nomail[B] from bounces.  I'd like to know how to purge all the nomail[B]
>members from the list -- without spending an entire day doing it through
>the GUI.


If Mailman's cron/disabled script is being run daily by cron as it
should be, a nomail[B] member will receive a total of N warnings at
intervals of D days and then will be automatically removed.

N = the lists bounce_you_are_disabled_warnings setting
D = the lists bounce_you_are_disabled_warnings_interval setting

If members aren't being removed N*D days after delivery is initially
disabled by bounce, cron/disabled isn't running or is encountering
errors. The errors should be mailed by cron to the Mailman user unless
MAILTO is set to something else on the crontab. The problem here is
mail to the Mailman user often ends up being delivered to to site list
and if the site list doesn't accept mail from non-members, that mail
can be lost.

If after digesting the above you still want to remove nomail[B] members
manually, you can do it with withlist as in:

mmp/bin/withlist -l LISTNAME
Loading list LISTNAME (locked)
The variable `m' is the LISTNAME MailList instance
>>> from Mailman import MemberAdaptor
>>> for member in m.getDeliveryStatusMembers(status=[MemberAdaptor.BYBOUNCE]):
...     m.removeMember(member)
...
>>> m.Save()
>>>                    (type control-D here to exit)
Unlocking (but not saving) list: gpc-test
Finalizing

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Sun Nov  4 20:19:29 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Sun, 04 Nov 2012 11:19:29 -0800
Subject: [Mailman-Users] Delete all members with nomail[B]
In-Reply-To: <PC195201211040832100796a880ec26@MSAPIRO>
References: <PC195201211040832100796a880ec26@MSAPIRO>
Message-ID: <5096BFC1.1040300@agliodbs.com>

Mark,

> If Mailman's cron/disabled script is being run daily by cron as it
> should be, a nomail[B] member will receive a total of N warnings at
> intervals of D days and then will be automatically removed.

Hmmm.  "N" is set to zero, so it seems like they should have already
been removed.  No?


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Mon Nov  5 01:37:58 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 4 Nov 2012 16:37:58 -0800
Subject: [Mailman-Users] Delete all members with nomail[B]
In-Reply-To: <5096BFC1.1040300@agliodbs.com>
Message-ID: <PC19520121104163758081243131026@MSAPIRO>

Josh Berkus quoted me and wrote:
>
>> If Mailman's cron/disabled script is being run daily by cron as it
>> should be, a nomail[B] member will receive a total of N warnings at
>> intervals of D days and then will be automatically removed.
>
>Hmmm.  "N" is set to zero, so it seems like they should have already
>been removed.  No?


That depends. If bounce_you_are_disabled_warnings was set to zero at
the time of the user's first bounce, then the user should have been
removed when her bounce score reached bounce_score_threshold rather
than setting her delivery nomail[B]. However, if it was non-zero at
the time of the first recorded bounce, that is the value that was
saved for that user at that time and subsequent changes to
bounce_you_are_disabled_warnings have no effect for that user.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Mon Nov  5 03:22:35 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Sun, 04 Nov 2012 18:22:35 -0800
Subject: [Mailman-Users] Delete all members with nomail[B]
In-Reply-To: <PC19520121104163758081243131026@MSAPIRO>
References: <PC19520121104163758081243131026@MSAPIRO>
Message-ID: <509722EB.5010004@agliodbs.com>

Mark,

> That depends. If bounce_you_are_disabled_warnings was set to zero at
> the time of the user's first bounce, then the user should have been
> removed when her bounce score reached bounce_score_threshold rather
> than setting her delivery nomail[B]. However, if it was non-zero at
> the time of the first recorded bounce, that is the value that was
> saved for that user at that time and subsequent changes to
> bounce_you_are_disabled_warnings have no effect for that user.

Ok, that's probably what happened.  So if I wait a couple weeks, these
users should all gradually go away.

I will keep your withlist script around in case I want to delete users
who are voluntarily nomail.  We have quite a few users who go nomail,
and then forget they ever subscribed ...

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Mon Nov  5 03:51:44 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 04 Nov 2012 18:51:44 -0800
Subject: [Mailman-Users] Delete all members with nomail[B]
In-Reply-To: <509722EB.5010004@agliodbs.com>
References: <PC19520121104163758081243131026@MSAPIRO>
	<509722EB.5010004@agliodbs.com>
Message-ID: <509729C0.3080009@msapiro.net>

On 11/4/2012 6:22 PM, Josh Berkus wrote:
> 
>> However, if it was non-zero at
>> the time of the first recorded bounce, that is the value that was
>> saved for that user at that time and subsequent changes to
>> bounce_you_are_disabled_warnings have no effect for that user.
> 
> Ok, that's probably what happened.  So if I wait a couple weeks, these
> users should all gradually go away.


Assuming cron/disabled is being run, yes, that is the case.


> I will keep your withlist script around in case I want to delete users
> who are voluntarily nomail.  We have quite a few users who go nomail,
> and then forget they ever subscribed ...


You'd need to change MemberAdaptor.BYBOUNCE in that script to
MemberAdaptor.BYUSER to delete members who set themselves to nomail.

However, it occurs to me that there is a better way than my possibly
error prone withlist method. Consider

cd to Mailman's bin directory
./list_members -n bybounce LISTNAME | ./remove_members -f - LISTNAME

or

./list_members -n byuser LISTNAME | ./remove_members -f - LISTNAME

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Mon Nov  5 05:07:43 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 4 Nov 2012 20:07:43 -0800
Subject: [Mailman-Users] Mailman archive - archive date does not match
	thedate the message was posted
In-Reply-To: <CAEC=f8f6HZzW5F8Ppj7VWji=BQmK1fCf8RAbz7AjqxgyE=pMGA@mail.gmail.com>
Message-ID: <PC19520121104200743015692326687@MSAPIRO>

Anthony Wilson (Jaf) wrote:

>There is an existing Mailman list with an archive which needs to be moved
>to a different server, however it seems that the archive is a bit messed
>up. I assume that the list was set up in 2006, but for some reason there is
>a 2002 archive, and all of the posts in there seem to have been posted in
>2012. It's a public archive so I assume there's no harm in posting a link
>to it - https://secure.neap.net/pipermail/montrealers/.  Is there some way
>to fix this?


There are two different groups of messages in the archive for January,
2002. One group is messages numbered 023657 through 023673. These
actually belong in July, 2009 between
<https://secure.neap.net/pipermail/montrealers/2009-July/023656.html>
and
<https://secure.neap.net/pipermail/montrealers/2009-July/023674.html>.

The other group is messages numbered 052646 through 052858. These
actually belong in May, 2012 between
<https://secure.neap.net/pipermail/montrealers/2012-May/052645.html>
and
<https://secure.neap.net/pipermail/montrealers/2012-May/052859.html>.

How they wound up with wrong dates is something of a mystery. My best
guess is that the system clock on the server that archived them was
set wrong during some periods in July 2009 and May 2012, but that
seems far fetched.

I'm assuming that the messages are in correct sequence in the
archives/private/montrealers.mbox/montrealers.mbox file. The first
step is to find some of these messages in that file ans see what the
time stamps are in their From_ lines and Date: headers. If the time
stamps appear correct there,

bin/arch --wipe montrealers

will probably fix everything. If those time stamps are from 2002, I
think the answer is manually editing the timestamps in
archives/private/montrealers.mbox/montrealers.mbox file and then
running the above bin/arch command.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From sahil+mailman at tandon.net  Mon Nov  5 04:42:33 2012
From: sahil+mailman at tandon.net (Sahil Tandon)
Date: Sun, 4 Nov 2012 22:42:33 -0500
Subject: [Mailman-Users] suppress duplicate when posting addressed to list
	and its alias name
Message-ID: <20121105034233.GA1839@magic.hamla.org>

Many of our lists have full addresses in the form of 'foo-bar at baz.org',
with corresponding alias names 'bar at baz.org'.

When someone sends a message To:foo-bar at baz.org and also Cc:bar at baz.org,
his or her MUA sends two separate messages to the list which transmits
these duplicates to list members.  However, only one copy makes it to
the pipermail HTML archives.

Is there a Mailman way to activate, in the context of delivery, the same
duplicate suppression that occurs when archiving?  If not, I will have
to do this based on Message-IDs on input at the MTA, but I would prefer
to do it via the mailing list manager if this is already possible.

Thanks,
-- 
Sahil Tandon

From Peter.STUMPF at unoosa.org  Mon Nov  5 16:01:21 2012
From: Peter.STUMPF at unoosa.org (Peter STUMPF)
Date: Mon, 5 Nov 2012 16:01:21 +0100
Subject: [Mailman-Users] AUTO: Peter STUMPF is out of the office.
Message-ID: <OF6EB6891E.3D63E858-ONC1257AAD.00528576-C1257AAD.00528576@unvienna.org>



I am out of the office from Fri 10/05/2012 until Thu 01/31/2013.

Dear Sir or Madam,
this is to inform you that my assignment with UN-SPIDER expired.
I'll keep an eye on this mailbox though for the transition time and try to
reply timely.
For any matters regarding the UN-SPIDER Knowledge Portal please contact
either Anne Knauer (anne.knauer at unoosa.org) or Juan Carlos Villagran de
Leon (juan-carlos.villagran at unoosa.org)
Best regards,

Peter Stumpf


Note: This is an automated response to your message  "[Mailman-Users]
suppress duplicate when posting addressed to list	and its alias name"
sent on 05/11/2012 4:42:33.

This is the only notification you will receive while this person is away.

From josh at agliodbs.com  Mon Nov  5 19:23:50 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Mon, 05 Nov 2012 10:23:50 -0800
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
Message-ID: <50980436.4050809@agliodbs.com>

Folks,

I deleted a mailing list using rmlist.  However, mailman continues to
send out messages to the recipients on this list, and I can't seem to
make it stop.  There's ~~ 3000 pending deliveries, so I'd really rather
that they not go out.

For now mailman is shut down.  How do I clean out its pending messages
queue entirely?


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From josh at agliodbs.com  Mon Nov  5 23:35:11 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Mon, 05 Nov 2012 14:35:11 -0800
Subject: [Mailman-Users] withlist docs?
Message-ID: <50983F1F.9040806@agliodbs.com>

Folks,

I'd like to use withlist more to troubleshoot certain issues, but I
can't find any documentation on the objects and methods available in the
withlist context.  Do such docs exist somewhere?

For example, I'd like to figure out how to set all addresses in a
certain domain nomail.  But I can't figure out:

- what the parameters and outputs of getDeliveryStatusMembers are
- what the method is to set someone nomail

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Tue Nov  6 01:51:21 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 5 Nov 2012 16:51:21 -0800
Subject: [Mailman-Users] suppress duplicate when posting addressed to
	listand its alias name
In-Reply-To: <20121105034233.GA1839@magic.hamla.org>
Message-ID: <PC1952012110516512100152641cee4@MSAPIRO>

Sahil Tandon wrote:

>Is there a Mailman way to activate, in the context of delivery, the same
>duplicate suppression that occurs when archiving?


No. The archiver has intimate knowledge of message-ids because they are
used in message threading so it knows when a message has a duplicate
message-id. Note that it doesn't actually ignore the duplicate. It
creates an nnnnnn.html archive file containing the message and it adds
it to the cumulative .mbox file and (I think, I haven't checked) the
periodic .txt file; it just doesn't link to it from any of the index
files.


>If not, I will have
>to do this based on Message-IDs on input at the MTA, but I would prefer
>to do it via the mailing list manager if this is already possible.


It would be possible to implement a per-list database of processed
message-ids with a custom handler very early in the pipeline, and
discard duplicates there. See <http://wiki.list.org/x/l4A9>.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Tue Nov  6 02:02:24 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 5 Nov 2012 17:02:24 -0800
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <50980436.4050809@agliodbs.com>
Message-ID: <PC195201211051702240687e8c95d47@MSAPIRO>

Josh Berkus wrote:

>Folks,
>
>I deleted a mailing list using rmlist.  However, mailman continues to
>send out messages to the recipients on this list, and I can't seem to
>make it stop.  There's ~~ 3000 pending deliveries, so I'd really rather
>that they not go out.
>
>For now mailman is shut down.  How do I clean out its pending messages
>queue entirely?


First of all, if the list is gone, i.e. there is no lists/LISTNAME/
directory in Mailman's hierarchy, no currently queued messages will be
processed. Outgoing runner or some other runner may pick up queue
entries for this list, but the list won't be there, so the queued
message will be shunted with an error log message 'Dequeuing message
destined for missing list: LISTNAME'. Therefore, you can just restart
Mailman and any messages in Mailman's queues from the affected list
will just wind up in the shunt queue and won't be processed.

Thus, any 'pending' deliveries which will go out have already been
delivered from Mailman to the MTA, and stopping them has to be done in
the MTA (unless that's moot at this point).

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Tue Nov  6 02:21:51 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 5 Nov 2012 17:21:51 -0800
Subject: [Mailman-Users] withlist docs?
In-Reply-To: <50983F1F.9040806@agliodbs.com>
Message-ID: <PC195201211051721510640df1de873@MSAPIRO>

Josh Berkus wrote:
>
>I'd like to use withlist more to troubleshoot certain issues, but I
>can't find any documentation on the objects and methods available in the
>withlist context.  Do such docs exist somewhere?


There are two basic objects in Mailman. These are list objects and
message objects.

Message objects are a sub-class of the python email.message class which
is documented in the Python docs (e,g.
<http://docs.python.org/2/library/email.message.html#email.message>.
Mailman adds a few methods in the Message.Message class definition in
Mailman/Message.py.

List objects have many methods defined in several different modules,
but a good starting point is Mailman/MailList.py. Also, virtually
every method having to do with list members, including the low level
addNewMember(), removeMember() and changeMemberAddress() methods
called by higher level methods defined in Mailman/MailList.py, is
documented in Mailman/MemberAdaptor.py.


>For example, I'd like to figure out how to set all addresses in a
>certain domain nomail.  But I can't figure out:
>
>- what the parameters and outputs of getDeliveryStatusMembers are


Not the method you want. You want getMembers(), but both are documented
in Mailman/MemberAdaptor.py.


>- what the method is to set someone nomail


setDeliveryStatus() also documented in Mailman/MemberAdaptor.py.


You might look at the scripts at <http://www.msapiro.net/scripts/> as
examples of some of the things you might want to do. In particular,
<http://www.msapiro.net/scripts/reset_bounce.py> with a very small
change can disable delivery for all members in a given domain and
<http://www.msapiro.net/scripts/set_nomail.py> can disable delivery
for given members.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Tue Nov  6 02:23:13 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Mon, 05 Nov 2012 17:23:13 -0800
Subject: [Mailman-Users] withlist docs?
In-Reply-To: <PC195201211051721510640df1de873@MSAPIRO>
References: <PC195201211051721510640df1de873@MSAPIRO>
Message-ID: <50986681.9020904@agliodbs.com>


> You might look at the scripts at <http://www.msapiro.net/scripts/> as
> examples of some of the things you might want to do. In particular,
> <http://www.msapiro.net/scripts/reset_bounce.py> with a very small
> change can disable delivery for all members in a given domain and
> <http://www.msapiro.net/scripts/set_nomail.py> can disable delivery
> for given members.

Thanks very much; this helps a lot.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From guest2 at sgeinc.com  Tue Nov  6 05:17:17 2012
From: guest2 at sgeinc.com (Richard Shetron)
Date: Mon, 05 Nov 2012 23:17:17 -0500
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <PC195201211051702240687e8c95d47@MSAPIRO>
References: <PC195201211051702240687e8c95d47@MSAPIRO>
Message-ID: <50988F4D.2070309@sgeinc.com>



On 11/5/2012 8:02 PM, Mark Sapiro wrote:
> Josh Berkus wrote:
>
>> Folks,
>>
>> I deleted a mailing list using rmlist.  However, mailman continues to
>> send out messages to the recipients on this list, and I can't seem to
>> make it stop.  There's ~~ 3000 pending deliveries, so I'd really rather
>> that they not go out.
>>
>> For now mailman is shut down.  How do I clean out its pending messages
>> queue entirely?
>
>
> First of all, if the list is gone, i.e. there is no lists/LISTNAME/
> directory in Mailman's hierarchy, no currently queued messages will be
> processed. Outgoing runner or some other runner may pick up queue
> entries for this list, but the list won't be there, so the queued
> message will be shunted with an error log message 'Dequeuing message
> destined for missing list: LISTNAME'. Therefore, you can just restart
> Mailman and any messages in Mailman's queues from the affected list
> will just wind up in the shunt queue and won't be processed.
>
> Thus, any 'pending' deliveries which will go out have already been
> delivered from Mailman to the MTA, and stopping them has to be done in
> the MTA (unless that's moot at this point).

I have a shell script that I use to remove email from the postfix queue 
based on some unique text string.  This is only for postfix and is kinda 
crude, but it works ;)

It should run on any *nix system.



From sahil+mailman at tandon.net  Tue Nov  6 06:20:41 2012
From: sahil+mailman at tandon.net (Sahil Tandon)
Date: Tue, 6 Nov 2012 00:20:41 -0500
Subject: [Mailman-Users] suppress duplicate when posting addressed to
 listand its alias name
In-Reply-To: <PC1952012110516512100152641cee4@MSAPIRO>
References: <20121105034233.GA1839@magic.hamla.org>
	<PC1952012110516512100152641cee4@MSAPIRO>
Message-ID: <20121106052040.GA3780@magic.hamla.org>

On Mon, 2012-11-05 at 16:51:21 -0800, Mark Sapiro wrote:
> ... 
> It would be possible to implement a per-list database of processed
> message-ids with a custom handler very early in the pipeline, and
> discard duplicates there. See <http://wiki.list.org/x/l4A9>.

Thanks Mark, this seems like the ideal approach.  I'll try to hack
something together borrowing from the various handlers (namely
AvoidDuplicates.py) that are already in use.  If I can understand how
Mailman keeps the in-memory dictionary of Message-IDs mentioned in
AvoidDuplicates.py, and implement an analogue for our use-case, that
would do it.  The goal is to check whether a tuple of (message-id,
listname) already exists in the dict and, if it does, raise
Errors.DiscardMessage; otherwise, add the tuple to the dict and do
nothing.

-- 
Sahil Tandon

From mark at msapiro.net  Tue Nov  6 20:26:40 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Nov 2012 11:26:40 -0800
Subject: [Mailman-Users] suppress duplicate when posting addressed
	tolistand its alias name
In-Reply-To: <20121106052040.GA3780@magic.hamla.org>
Message-ID: <PC1952012110611264009377b914591@MSAPIRO>

Sahil Tandon wrote:
>
>Thanks Mark, this seems like the ideal approach.  I'll try to hack
>something together borrowing from the various handlers (namely
>AvoidDuplicates.py) that are already in use.


Actually, AvoidDuplicates.py ccould serve as a good example, but it is
currently not actually used. It is experimental and is bot included in
the default GLOBAL_PIPELINE.


>If I can understand how
>Mailman keeps the in-memory dictionary of Message-IDs mentioned in
>AvoidDuplicates.py, and implement an analogue for our use-case, that
>would do it.


The major problem with keeping these data in-memory other than purging
"old" entries so that the dictionary doesn't grow too large, is that
in-memory data aren't shared between runners so if the incoming queue
is sliced, the multiple copies of IncomingRunner do not have access to
each other's data.

In your case, the input to the hash on which runners are sliced
includes all the message headers and the listname so it is likely that
the "equivalent but different" listname messages will be in different
slices of the hash space.

This is not a concern if IncomingRunner is not sliced. It is also not a
concern with a disk based cache as long as buffers are flushed after
writing because IncomingRunner locks the list whose message is being
processed which should prevent race conditions between different
slices of IncomingRunner.


>The goal is to check whether a tuple of (message-id,
>listname) already exists in the dict and, if it does, raise
>Errors.DiscardMessage; otherwise, add the tuple to the dict and do
>nothing.


I would make a dictionary keyed on message-id + the cannonical listname
with value = the time seen. Then I could just check if the key for the
current message exists and proceed as above, and I also have time
stamps so I can periodically remove old entries.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Tue Nov  6 21:00:18 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Nov 2012 12:00:18 -0800
Subject: [Mailman-Users] suppress duplicate when posting
	addressedtolistand its alias name
In-Reply-To: <PC1952012110611264009377b914591@MSAPIRO>
Message-ID: <PC195201211061200180343329337e9@MSAPIRO>

Mark Sapiro wrote:

>Actually, AvoidDuplicates.py ccould serve as a good example, but it is
>currently not actually used. It is experimental and is bot included in
>the default GLOBAL_PIPELINE.


I seem to be having more than my usual problems with
typing/proofreading this morning. The above paragraph should say

Actually, AvoidDuplicates.py could serve as a good example, but it is
currently not actually used. It is experimental and is not included in
the default GLOBAL_PIPELINE.

More importantly, I was confused. AvoidDuplicates.py does appear in the
GLOBAL_PIPELINE, but it does not do what its docstring says it does.
All it does in its current form is not send to list recipients who are
explicitly addressed in a To: or Cc: header of the message and you
have selected the nodups option for the list.

Thus, there is no "in-memory dictionary of Message-ID: and recipient
pairs" and no testing to see if a recipient has already received
another copy of the message from Mailman. The intent of the feature
described in the docstring is to enable the elimination of sending
multiple copies of messages which were cross-posted to multiple lists
to a recipient who is a member of more than one of those lists. This
feature was never successfully implemented.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From es at fruitcom.com  Wed Nov  7 00:49:25 2012
From: es at fruitcom.com (Eric Smith)
Date: Wed, 7 Nov 2012 00:49:25 +0100
Subject: [Mailman-Users] Adding custom header
Message-ID: <20121106234925.GA7359@foodsniffer.eu>

I want to receive all mail relating to lists that I create and
manage into my inbox. Because there are arbitrary list names and
also numerous domains, I think that the best way is for all list
mail to carry an identifying mail header.

Having searched the archives, I understand I should 
edit the source.
I am thinking like appending a tag to an existing header like
List-Id:

Is this giving the corre t file to edit.
 $ find /usr/lib/mailman -type f -exec grep -l List-Id {} \;
/usr/lib/mailman/Mailman/Handlers/CookHeaders.py

Any caveats, guidelines, bearing in mind this:
http://mail.python.org/pipermail/mailman-users/2011-December/072595.html

-- 
Eric Smith

From sahil+mailman at tandon.net  Wed Nov  7 02:02:17 2012
From: sahil+mailman at tandon.net (Sahil Tandon)
Date: Tue, 6 Nov 2012 20:02:17 -0500
Subject: [Mailman-Users] suppress duplicate when posting addressed
 tolistand its alias name
In-Reply-To: <PC1952012110611264009377b914591@MSAPIRO>
References: <20121106052040.GA3780@magic.hamla.org>
	<PC1952012110611264009377b914591@MSAPIRO>
Message-ID: <20121107010216.GC3998@magic.hamla.org>

On Tue, 2012-11-06 at 11:26:40 -0800, Mark Sapiro wrote:

> Actually, AvoidDuplicates.py ccould serve as a good example, but it is
> currently not actually used. It is experimental and is bot included in
> the default GLOBAL_PIPELINE.

As you noted in your follow-up, the docstring does not at all describe
what that handler actually does.  I learned this when actually stepping
through the code. :)

> The major problem with keeping these data in-memory other than purging
> "old" entries so that the dictionary doesn't grow too large, is that
> in-memory data aren't shared between runners so if the incoming queue
> is sliced, the multiple copies of IncomingRunner do not have access to
> each other's data.
> 
> In your case, the input to the hash on which runners are sliced
> includes all the message headers and the listname so it is likely that
> the "equivalent but different" listname messages will be in different
> slices of the hash space.
> 
> This is not a concern if IncomingRunner is not sliced. It is also not a
> concern with a disk based cache as long as buffers are flushed after
> writing because IncomingRunner locks the list whose message is being
> processed which should prevent race conditions between different
> slices of IncomingRunner.

Then, would it make sense (or be overkill) to have the handler populate
a dict of key, value = message-id, timestamp?  And, store that dict in a
pickle whose filename is derived from mlist.internal_name()?

Obviously, this would result in a lot of pickles that are constantly
opened, edited (and, periodically cleansed), and closed.  Is the
performance cost/benefit prohibitive?  I would also be relying on the
fact that a handler is never concurrently called for the same list -- is
that understanding accurate? -- which avoids the scenario in which we
are trying to simultaneously manipulate the same pickle.

-- 
Sahil Tandon

From mark at msapiro.net  Wed Nov  7 02:49:35 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Nov 2012 17:49:35 -0800
Subject: [Mailman-Users] Adding custom header
In-Reply-To: <20121106234925.GA7359@foodsniffer.eu>
Message-ID: <PC195201211061749350437a29a6798@MSAPIRO>

Eric Smith wrote:

>I want to receive all mail relating to lists that I create and
>manage into my inbox. Because there are arbitrary list names and
>also numerous domains, I think that the best way is for all list
>mail to carry an identifying mail header.
>
>Having searched the archives, I understand I should 
>edit the source.


Not really. There's a better way.


>I am thinking like appending a tag to an existing header like
>List-Id:


I would not do that. List-Id: is defined by RFC 2919, and while you may
not care if your list's List-Id: header is not fully compliant, other
members of your lists might.

I would use a X- user defined header, e.g. X-MyList: for this purpose.

I would not edit CookHeaders.py, even if I were going to modify the
contents of the List-Id: header. I would create a custom handler to
add my header or modify the List-Id: header added by CookHeaders. See
the FAQ at <http://wiki.list.org/x/l4A9> for more about implementing
custom handlers and why this is the right way to do what you want.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Wed Nov  7 03:02:27 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 6 Nov 2012 18:02:27 -0800
Subject: [Mailman-Users] suppress duplicate when posting
	addressedtolistand its alias name
In-Reply-To: <20121107010216.GC3998@magic.hamla.org>
Message-ID: <PC195201211061802270281f9aa9ef4@MSAPIRO>

Sahil Tandon wrote:

>On Tue, 2012-11-06 at 11:26:40 -0800, Mark Sapiro wrote:
>
>> In your case, the input to the hash on which runners are sliced
>> includes all the message headers and the listname so it is likely that
>> the "equivalent but different" listname messages will be in different
>> slices of the hash space.
>> 
>> This is not a concern if IncomingRunner is not sliced. It is also not a
>> concern with a disk based cache as long as buffers are flushed after
>> writing because IncomingRunner locks the list whose message is being
>> processed which should prevent race conditions between different
>> slices of IncomingRunner.
>
>Then, would it make sense (or be overkill) to have the handler populate
>a dict of key, value = message-id, timestamp?  And, store that dict in a
>pickle whose filename is derived from mlist.internal_name()?
>
>Obviously, this would result in a lot of pickles that are constantly
>opened, edited (and, periodically cleansed), and closed.  Is the
>performance cost/benefit prohibitive?


Whether the cost is prohibitive depends on how many messages per
minute, hour, day, etc you process through the list. I think it could
work. The 'in-memory dictionary would also work as long as you are
running with the default single qrunner per queue except for the rare
case where the duplicates are processed one on each side of a restart.

Note as an implementation for the file name (path) derived from the
list's internal_name, I would just use a fixed file name, e.g.,
message-ids.pck in the existing lists/internal_name()/ directory.


>I would also be relying on the
>fact that a handler is never concurrently called for the same list -- is
>that understanding accurate? -- which avoids the scenario in which we
>are trying to simultaneously manipulate the same pickle.


Yes, that is accurate. IncomingRunner locks the list before processing
the pipeline and doesn't unlock it until it's done, so processing of
the pipeline for a given message and list is complete before any other
runner can begin processing a message for that list.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From eviljafar at gmail.com  Wed Nov  7 03:32:28 2012
From: eviljafar at gmail.com (Anthony Wilson (Jaf))
Date: Tue, 6 Nov 2012 21:32:28 -0500
Subject: [Mailman-Users] Mailman archive - archive date does not match
 thedate the message was posted
In-Reply-To: <PC19520121104200743015692326687@MSAPIRO>
References: <CAEC=f8f6HZzW5F8Ppj7VWji=BQmK1fCf8RAbz7AjqxgyE=pMGA@mail.gmail.com>
	<PC19520121104200743015692326687@MSAPIRO>
Message-ID: <CAEC=f8fH4WzKcJohUxmU8S+x4E+GsQ1fdDvfXz6PhbGneUY6dA@mail.gmail.com>

Thanks for your reply Mark.

I took a look in the .mbox file and unfortunately the dates in the From and
Date headers are 2000, so it seems a manual fix is in order. There's 230
messages so I'll put that on hold for now.

On Sun, Nov 4, 2012 at 11:07 PM, Mark Sapiro <mark at msapiro.net> wrote:

> Anthony Wilson (Jaf) wrote:
>
> >There is an existing Mailman list with an archive which needs to be moved
> >to a different server, however it seems that the archive is a bit messed
> >up. I assume that the list was set up in 2006, but for some reason there
> is
> >a 2002 archive, and all of the posts in there seem to have been posted in
> >2012. It's a public archive so I assume there's no harm in posting a link
> >to it - https://secure.neap.net/pipermail/montrealers/.  Is there some
> way
> >to fix this?
>
>
> There are two different groups of messages in the archive for January,
> 2002. One group is messages numbered 023657 through 023673. These
> actually belong in July, 2009 between
> <https://secure.neap.net/pipermail/montrealers/2009-July/023656.html>
> and
> <https://secure.neap.net/pipermail/montrealers/2009-July/023674.html>.
>
> The other group is messages numbered 052646 through 052858. These
> actually belong in May, 2012 between
> <https://secure.neap.net/pipermail/montrealers/2012-May/052645.html>
> and
> <https://secure.neap.net/pipermail/montrealers/2012-May/052859.html>.
>
> How they wound up with wrong dates is something of a mystery. My best
> guess is that the system clock on the server that archived them was
> set wrong during some periods in July 2009 and May 2012, but that
> seems far fetched.
>
> I'm assuming that the messages are in correct sequence in the
> archives/private/montrealers.mbox/montrealers.mbox file. The first
> step is to find some of these messages in that file ans see what the
> time stamps are in their From_ lines and Date: headers. If the time
> stamps appear correct there,
>
> bin/arch --wipe montrealers
>
> will probably fix everything. If those time stamps are from 2002, I
> think the answer is manually editing the timestamps in
> archives/private/montrealers.mbox/montrealers.mbox file and then
> running the above bin/arch command.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>
>

From cite at incertum.net  Wed Nov  7 06:38:31 2012
From: cite at incertum.net (Stefan Foerster)
Date: Wed, 7 Nov 2012 06:38:31 +0100
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <PC195201211051702240687e8c95d47@MSAPIRO>
References: <50980436.4050809@agliodbs.com>
	<PC195201211051702240687e8c95d47@MSAPIRO>
Message-ID: <20121107053831.GB3552@mail.incertum.net>

* Mark Sapiro <mark at msapiro.net>:
> First of all, if the list is gone, i.e. there is no lists/LISTNAME/
> directory in Mailman's hierarchy, no currently queued messages will be
> processed. Outgoing runner or some other runner may pick up queue
> entries for this list, but the list won't be there, so the queued
> message will be shunted with an error log message 'Dequeuing message
> destined for missing list: LISTNAME'. Therefore, you can just restart
> Mailman and any messages in Mailman's queues from the affected list
> will just wind up in the shunt queue and won't be processed.

I am aware of how to get rid of shunted messages but I'd like to know
if there are performance penalites for keeping a moderate amount
(<1k) of messages around in the shunted queue for about a month.

As far as I understand, the shunted queue is not directly involved in
message delivery, so there should be no performance hit, right? This
way I could "play it safe", keep those messages around for a certain
time (they might turn out important after all) and then have a cron
job delete them.


Thanks
Stefan

From mark at msapiro.net  Wed Nov  7 15:56:01 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 07 Nov 2012 06:56:01 -0800
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <20121107053831.GB3552@mail.incertum.net>
References: <50980436.4050809@agliodbs.com>
	<PC195201211051702240687e8c95d47@MSAPIRO>
	<20121107053831.GB3552@mail.incertum.net>
Message-ID: <509A7681.1050208@msapiro.net>

On 11/6/2012 9:38 PM, Stefan Foerster wrote:

> I am aware of how to get rid of shunted messages but I'd like to know
> if there are performance penalites for keeping a moderate amount
> (<1k) of messages around in the shunted queue for about a month.
> 
> As far as I understand, the shunted queue is not directly involved in
> message delivery, so there should be no performance hit, right? This
> way I could "play it safe", keep those messages around for a certain
> time (they might turn out important after all) and then have a cron
> job delete them.


You are correct that the shunt queue is not involved in normal delivery.
The only hit would be when shunting an additional message. If the
physical size of the qfiles/shunt/ directory in the file system is
large, there is additional overhead each time another message is shunted
because the entire physical directory must be searched, even if it is
empty, to ensure the new entry is not a duplicate name.

Regarding a cron job, the following is from Mailman's NEWS file

2.1.11 (30-Jun-2008)

  New Features

    - Added a new cron/cull_bad_shunt script to cull and optionally
      archive old entries from the bad and shunt queues. This is controlled
      by new Defaults.py/mm_cfg.py settings BAD_SHUNT_STALE_AFTER (default
      7 days) and BAD_SHUNT_ARCHIVE_DIRECTORY (default None) which determine
      how long to keep bad and shunt queue entries and optionally, where to
      archive removed entries.

Note that this means that in a default configuration since 2.1.11, shunt
queue entries are deleted after 7 days. Do disable this, set

BAD_SHUNT_STALE_AFTER = 0

in mm_cfg.py. To keep them for a month, set

BAD_SHUNT_STALE_AFTER = days(30)

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Wed Nov  7 19:06:27 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Wed, 07 Nov 2012 10:06:27 -0800
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <509A7681.1050208@msapiro.net>
References: <50980436.4050809@agliodbs.com>
	<PC195201211051702240687e8c95d47@MSAPIRO>
	<20121107053831.GB3552@mail.incertum.net>
	<509A7681.1050208@msapiro.net>
Message-ID: <509AA323.8020509@agliodbs.com>

All,

FWIW, the queue emptied out after a few hours of my original report.

Since I both deleted the list and emptied the postfix queue, I'm still
mystified as to why it didn't empty out immediately though.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From sahil+mailman at tandon.net  Thu Nov  8 04:02:57 2012
From: sahil+mailman at tandon.net (Sahil Tandon)
Date: Wed, 7 Nov 2012 22:02:57 -0500
Subject: [Mailman-Users] suppress duplicate when posting
 addressedtolistand its alias name
In-Reply-To: <PC195201211061802270281f9aa9ef4@MSAPIRO>
References: <20121107010216.GC3998@magic.hamla.org>
	<PC195201211061802270281f9aa9ef4@MSAPIRO>
Message-ID: <20121108030257.GH6183@magic.hamla.org>

On Tue, 2012-11-06 at 18:02:27 -0800, Mark Sapiro wrote:

> Sahil Tandon wrote:
> ...
> >Obviously, this would result in a lot of pickles that are constantly
> >opened, edited (and, periodically cleansed), and closed.  Is the
> >performance cost/benefit prohibitive?
> 
> Whether the cost is prohibitive depends on how many messages per
> minute, hour, day, etc you process through the list. I think it could
> work. The 'in-memory dictionary would also work as long as you are
> running with the default single qrunner per queue except for the rare
> case where the duplicates are processed one on each side of a restart.
> 
> Note as an implementation for the file name (path) derived from the
> list's internal_name, I would just use a fixed file name, e.g.,
> message-ids.pck in the existing lists/internal_name()/ directory.

Thanks; this custom handler is now in testing and working well so far.
I appreciate your help.

-- 
Sahil Tandon

From mark at msapiro.net  Thu Nov  8 05:08:52 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 07 Nov 2012 20:08:52 -0800
Subject: [Mailman-Users] Mailman continues to deliver to deleted list
In-Reply-To: <509AA323.8020509@agliodbs.com>
References: <50980436.4050809@agliodbs.com>
	<PC195201211051702240687e8c95d47@MSAPIRO>
	<20121107053831.GB3552@mail.incertum.net>
	<509A7681.1050208@msapiro.net> <509AA323.8020509@agliodbs.com>
Message-ID: <509B3054.10306@msapiro.net>

On 11/7/2012 10:06 AM, Josh Berkus wrote:
> 
> Since I both deleted the list and emptied the postfix queue, I'm still
> mystified as to why it didn't empty out immediately though.


If you delete a list and Mailman's OutgoingRunner is processing a
message for that list, as long as the runner is still running, it will
continue to process that one message until all recipients have been
delivered to the MTA. The only way to stop that is to SIGKILL the runner.

If you are referring to additional message entries in qfiles/out/, those
will be processed in sequence by OutgoingRunner, but nothing will be
delivered from them because the runner will determine that the list is
gone and just log the fact and move the queue entry to qfiles/shunt/.

If neither of the above explains your observation, then I don't know (at
least without more information) what was going on.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From igoetrich at yahoo.de  Thu Nov  8 07:54:12 2012
From: igoetrich at yahoo.de (E Kogler)
Date: Thu, 8 Nov 2012 06:54:12 +0000 (GMT)
Subject: [Mailman-Users] mailman across firewall
Message-ID: <1352357652.58591.YahooMailNeo@web29403.mail.ird.yahoo.com>

Hello,
my task is to establish access to mailman across a firewall.
The configuration looks (of course ;-) ) like that:

INTERNET---FW--INTERNAL

On the Firewall there is a mail-relay with postfix installed, on the Internal server the courier-mta does the job.
I'm wondering how to cross the firewall without much work for me as the sysadmin.
What I came up with is to have each list with all its addresses on the valid recipients list on the mail-relay but it seems to me a little bit complicated because I'm planning to have a few lists running on the internal server.
The other solution which came to my mind is to have mailman on the FW and tweaking the mail-relay.


Is there a better way doing this ?
Edgar

From josh at agliodbs.com  Thu Nov  8 18:53:24 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 09:53:24 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
Message-ID: <509BF194.3010607@agliodbs.com>

Folks,

1) I set bounce_you_are_disabled_warnings=0

2) Using a withlist script, I removed all list members with any bounce
stats at all.

Yet, when I checked today, there were 98 outgoing bounce warnings
(undeliverable) in the queue.  Huh?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From josh at agliodbs.com  Thu Nov  8 18:56:48 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 09:56:48 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <509BF194.3010607@agliodbs.com>
References: <509BF194.3010607@agliodbs.com>
Message-ID: <509BF260.1050608@agliodbs.com>

On 11/8/12 9:53 AM, Josh Berkus wrote:
> Folks,
> 
> 1) I set bounce_you_are_disabled_warnings=0
> 
> 2) Using a withlist script, I removed all list members with any bounce
> stats at all.
> 
> Yet, when I checked today, there were 98 outgoing bounce warnings
> (undeliverable) in the queue.  Huh?

Clarification: 98 *new* bounce messages, from after the withlist purge.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Thu Nov  8 22:57:19 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 8 Nov 2012 13:57:19 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <509BF260.1050608@agliodbs.com>
Message-ID: <PC1952012110813571907815ed5fc60@MSAPIRO>

Josh Berkus wrote:
>> 
>> 1) I set bounce_you_are_disabled_warnings=0
>> 
>> 2) Using a withlist script, I removed all list members with any bounce
>> stats at all.
>> 
>> Yet, when I checked today, there were 98 outgoing bounce warnings
>> (undeliverable) in the queue.  Huh?
>
>Clarification: 98 *new* bounce messages, from after the withlist purge.


I am guessing that what you are saying is these messages are queued in
Postfix for addresses that give a Postfix retryable error. If you are
saying something different, please clarify as to where they are queued
and perhaps what their content is.

If my guess is correct, what is happening is you have removed any
bouncing members, so any subsequent bounces for current members are
initial bounces. Thus, I am also guessing that bounce_processing is
Yes, and bounce_score_threshold is <= 1.0 so that an initial bounce
will trigger immediate removal of the member from the list, and your
98 messages are the goodbye messages sent to the removed members
because the list's General Options -> send_goodbye_msg is Yes.

Assuming these messages were in fact generated as a result of Mailman's
bounce processing, there will be entries in Mailman's 'bounce' log
corresponding to the messages.

If the above is not the explanation, please give more detail so we can
understand what the problem is.	
	
-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Thu Nov  8 23:09:03 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 14:09:03 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <PC1952012110813571907815ed5fc60@MSAPIRO>
References: <PC1952012110813571907815ed5fc60@MSAPIRO>
Message-ID: <509C2D7F.6020908@agliodbs.com>


> I am guessing that what you are saying is these messages are queued in
> Postfix for addresses that give a Postfix retryable error. If you are
> saying something different, please clarify as to where they are queued
> and perhaps what their content is.

Correct.

> If my guess is correct, what is happening is you have removed any
> bouncing members, so any subsequent bounces for current members are
> initial bounces. Thus, I am also guessing that bounce_processing is
> Yes, and bounce_score_threshold is <= 1.0 so that an initial bounce
> will trigger immediate removal of the member from the list, and your
> 98 messages are the goodbye messages sent to the removed members
> because the list's General Options -> send_goodbye_msg is Yes.

That's quite likely the correct explanation.  I will play with settings
and see if I can get bounce queuing down.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From josh at agliodbs.com  Thu Nov  8 23:11:41 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 14:11:41 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <509C2D7F.6020908@agliodbs.com>
References: <PC1952012110813571907815ed5fc60@MSAPIRO>
	<509C2D7F.6020908@agliodbs.com>
Message-ID: <509C2E1D.9080906@agliodbs.com>


> That's quite likely the correct explanation.  I will play with settings
> and see if I can get bounce queuing down.

BTW, how does bounce scoring work?  I couldn't find any documentation on it.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Thu Nov  8 23:24:15 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 8 Nov 2012 14:24:15 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <509C2E1D.9080906@agliodbs.com>
Message-ID: <PC1952012110814241502506544e6c9@MSAPIRO>

Josh Berkus wrote:
>
>BTW, how does bounce scoring work?  I couldn't find any documentation on it.


The documentation is at the "(Details for bounce_score_threshold)" link
on the web admin Bounce processing page, but it lies.

The lie is that there is no distinction between hard and soft bounces.
Any recognized DSN returned to Mailman with a status of undeliverable
is a hard bounce (score = 1.0), even if the reason is something like
full mailbox. Anything that indicates the MTA is still trying to
deliver is ignored.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Fri Nov  9 00:36:01 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 15:36:01 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <PC1952012110814241502506544e6c9@MSAPIRO>
References: <PC1952012110814241502506544e6c9@MSAPIRO>
Message-ID: <509C41E1.7020905@agliodbs.com>


> The lie is that there is no distinction between hard and soft bounces.
> Any recognized DSN returned to Mailman with a status of undeliverable
> is a hard bounce (score = 1.0), even if the reason is something like
> full mailbox. Anything that indicates the MTA is still trying to
> deliver is ignored.

So Mailman could have already marked a user as bounced, even though the
MTA still has mail to them in the queue?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From mark at msapiro.net  Fri Nov  9 01:37:09 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 8 Nov 2012 16:37:09 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <509C41E1.7020905@agliodbs.com>
Message-ID: <PC1952012110816370903905824e74f@MSAPIRO>

Josh Berkus wrote:
>
>So Mailman could have already marked a user as bounced, even though the
>MTA still has mail to them in the queue?


No. Mailman will only score a bounce for a DSN that indicates failure.
Any MTA that keeps a message queued for delivery after returning a
failure DSN is seriously broken.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From josh at agliodbs.com  Fri Nov  9 01:49:51 2012
From: josh at agliodbs.com (Josh Berkus)
Date: Thu, 08 Nov 2012 16:49:51 -0800
Subject: [Mailman-Users] Why is Mailman still sending bounce messages?
In-Reply-To: <PC1952012110816370903905824e74f@MSAPIRO>
References: <PC1952012110816370903905824e74f@MSAPIRO>
Message-ID: <509C532F.7010307@agliodbs.com>

On 11/8/12 4:37 PM, Mark Sapiro wrote:
> Josh Berkus wrote:
>>
>> So Mailman could have already marked a user as bounced, even though the
>> MTA still has mail to them in the queue?
> 
> 
> No. Mailman will only score a bounce for a DSN that indicates failure.
> Any MTA that keeps a message queued for delivery after returning a
> failure DSN is seriously broken.

OK, whew.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

From odhiambo at gmail.com  Fri Nov  9 13:00:34 2012
From: odhiambo at gmail.com (Odhiambo Washington)
Date: Fri, 9 Nov 2012 15:00:34 +0300
Subject: [Mailman-Users] mailman across firewall
In-Reply-To: <1352357652.58591.YahooMailNeo@web29403.mail.ird.yahoo.com>
References: <1352357652.58591.YahooMailNeo@web29403.mail.ird.yahoo.com>
Message-ID: <CAAdA2WNtQ506FbiGrVVrtcwwxfE_aXd_gkyTirfZvadVDt8Zyg@mail.gmail.com>

On Thu, Nov 8, 2012 at 9:54 AM, E Kogler <igoetrich at yahoo.de> wrote:

> Hello,
> my task is to establish access to mailman across a firewall.
> The configuration looks (of course ;-) ) like that:
>
> INTERNET---FW--INTERNAL
>
> On the Firewall there is a mail-relay with postfix installed, on the
> Internal server the courier-mta does the job.
> I'm wondering how to cross the firewall without much work for me as the
> sysadmin.
> What I came up with is to have each list with all its addresses on the
> valid recipients list on the mail-relay but it seems to me a little bit
> complicated because I'm planning to have a few lists running on the
> internal server.
> The other solution which came to my mind is to have mailman on the FW and
> tweaking the mail-relay.
>
>
> Is there a better way doing this ?
> Edgar
>
>
Since Mailman has no login accounts, I see it as easy as running it on the
FW.


-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
I can't hear you -- I'm using the scrambler.

From vlad at hollebcons.com  Fri Nov  9 19:02:03 2012
From: vlad at hollebcons.com (Vladimir Mikhelson)
Date: Fri, 09 Nov 2012 12:02:03 -0600
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
Message-ID: <509D451B.1040800@hollebcons.com>

Hi,

Has anybody run into a problem where the "unsub" check-box + Submit Your
Changes  in the Membership List does not work?

Mass Removal works fine.

Thank you,
Vladimir




From mark at msapiro.net  Fri Nov  9 20:25:27 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 09 Nov 2012 11:25:27 -0800
Subject: [Mailman-Users] mailman across firewall
In-Reply-To: <1352357652.58591.YahooMailNeo@web29403.mail.ird.yahoo.com>
References: <1352357652.58591.YahooMailNeo@web29403.mail.ird.yahoo.com>
Message-ID: <509D58A7.30407@msapiro.net>

On 11/7/2012 10:54 PM, E Kogler wrote:
>
> What I came up with is to have each list with all its addresses on the valid recipients list on the mail-relay but it seems to me a little bit complicated because I'm planning to have a few lists running on the internal server.


I'm not sure what you think the complications are, but see the FAQ at
<http://wiki.list.org/x/ZoCj>. This does not directly address your
situation, but it indicates a method you could use to generate the
required recipient maps file on the Mailman server and use rsync or
whatever to copy that or merge it with the mail relay's maps.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Fri Nov  9 20:32:52 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Nov 2012 11:32:52 -0800
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <509D451B.1040800@hollebcons.com>
Message-ID: <PC1952012110911325201405225ecc0@MSAPIRO>

Vladimir Mikhelson wrote:
>
>Has anybody run into a problem where the "unsub" check-box + Submit Your
>Changes  in the Membership List does not work?


I have not seen this except in cases where the user's address is
syntactically invalid. If that's the case here, see the FAQ at
<http://wiki.list.org/x/UIA9>.

Otherwise, is there anything in Mailman's logs, particularly 'error'
and 'subscribe'?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From vlad at hollebcons.com  Fri Nov  9 21:55:06 2012
From: vlad at hollebcons.com (Vladimir Mikhelson)
Date: Fri, 09 Nov 2012 14:55:06 -0600
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <PC195201211091237430544e318a64b@Notebook-09>
References: <PC195201211091237430544e318a64b@Notebook-09>
Message-ID: <509D6DAA.3020204@hollebcons.com>


On 11/9/2012 2:37 PM, Mark Sapiro wrote:
> Vladimir Mikhelson wrote:
>> The address was fine as far as I can tell, no special characters, no
>> mixed case.
>>
>> The only relevant record in subscribe is as follows:
>>
>> Nov 09 11:48:05 2012 (14361) surcharge-letter: deleted user at domain.tld;
>> admin mass unsub
>
> And presumably that's the mass unsub you did after the checkbox unsub
> failed.
>
> Can you set user options frem the membership list, e.g. set mod, ack,
> hide, etc. on and off for a user?
>
Yes, I can.  I have tried "nomail" and "ack" back and forth with no
problems.



From mark at msapiro.net  Fri Nov  9 22:08:16 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Nov 2012 13:08:16 -0800
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <509D6DAA.3020204@hollebcons.com>
Message-ID: <PC195201211091308160217efcf24e9@Notebook-09>

Vladimir Mikhelson wrote:

>Yes, I can.  I have tried "nomail" and "ack" back and forth with no
>problems.


Then I don't know what the issue might be. The unsub checkbox works for
me. What is your Mailman version? Is there an appropriate POST
transaction in your web server logs from when you did the 'unsub'?

If you look at the source of a membership list page in your browser,
does a typical row look as follows except for the actual user name and
address and the available languages?

    <tr>
      <td><center><INPUT name="mark%40msapiro.net_unsub"
type="CHECKBOX" value="off" ></center></td>
      <td><a
href="../../options/gpc-test/mark--at--msapiro.net">mark at msapiro.net</a><br><INPUT
name="mark%40msapiro.net_realname" type="TEXT" value="Mark Sapiro"
size="22" ><INPUT name="user" type="HIDDEN" value="mark%40msapiro.net"
></td>
      <td><center><INPUT name="mark%40msapiro.net_mod" type="CHECKBOX"
value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_hide" type="CHECKBOX"
value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_nomail"
type="CHECKBOX" value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_ack" type="CHECKBOX"
value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_notmetoo"
type="CHECKBOX" value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_nodupes"
type="CHECKBOX" value="on" CHECKED ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_digest"
type="CHECKBOX" value="off" ></center></td>
      <td><center><INPUT name="mark%40msapiro.net_plain"
type="CHECKBOX" value="off" ></center></td>
      <td><center>
<Select name="mark%40msapiro.net_language">
 <option value="ca"> Catalan </option>
 <option value="de"> German </option>
 <option value="el"> Greek </option>
 <option value="en" Selected> English (USA) </option>
 <option value="es"> Spanish (Spain) </option>
 <option value="fr"> French </option>
 <option value="he"> Hebrew </option>
 <option value="ja"> Japanese </option>
 <option value="pt_BR"> Portuguese (Brazil) </option>
 <option value="ru"> Russian </option>
</Select></center></td>
    </tr>


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From vlad at hollebcons.com  Fri Nov  9 22:08:26 2012
From: vlad at hollebcons.com (Vladimir Mikhelson)
Date: Fri, 09 Nov 2012 15:08:26 -0600
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <509D6DAA.3020204@hollebcons.com>
References: <PC195201211091237430544e318a64b@Notebook-09>
	<509D6DAA.3020204@hollebcons.com>
Message-ID: <509D70CA.1040808@hollebcons.com>


On 11/9/2012 2:55 PM, Vladimir Mikhelson wrote:
> On 11/9/2012 2:37 PM, Mark Sapiro wrote:
>> Vladimir Mikhelson wrote:
>>> The address was fine as far as I can tell, no special characters, no
>>> mixed case.
>>>
>>> The only relevant record in subscribe is as follows:
>>>
>>> Nov 09 11:48:05 2012 (14361) surcharge-letter: deleted user at domain.tld;
>>> admin mass unsub
>> And presumably that's the mass unsub you did after the checkbox unsub
>> failed.
>>
>> Can you set user options frem the membership list, e.g. set mod, ack,
>> hide, etc. on and off for a user?
>>
> Yes, I can.  I have tried "nomail" and "ack" back and forth with no
> problems.
>
I suspect the problem can be related to the initial setup
inconsistencies I observed working with this list. See my posting
"Cannot Modify MOD on the Membership List in the Mailman Admin WEB UI"
dated 09/08/2012.

I eventually followed the instructions published here:
http://wiki.list.org/pages/viewpage.action?pageId=4030592  After that
the Membership List page started working.

But I did not modify the Apache rewrite rules per
http://wiki.list.org/display/DOC/4.27+Securing+Mailman%27s+web+GUI+by+using+Secure+HTTP-SSL

Do you think it can be related?



From vlad at hollebcons.com  Fri Nov  9 22:24:03 2012
From: vlad at hollebcons.com (Vladimir Mikhelson)
Date: Fri, 09 Nov 2012 15:24:03 -0600
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <PC195201211091308160217efcf24e9@Notebook-09>
References: <PC195201211091308160217efcf24e9@Notebook-09>
Message-ID: <509D7473.70903@hollebcons.com>


On 11/9/2012 3:08 PM, Mark Sapiro wrote:
> Vladimir Mikhelson wrote:
>
>> Yes, I can.  I have tried "nomail" and "ack" back and forth with no
>> problems.
>
> Then I don't know what the issue might be. The unsub checkbox works for
> me. What is your Mailman version? Is there an appropriate POST
> transaction in your web server logs from when you did the 'unsub'?
>
> If you look at the source of a membership list page in your browser,
> does a typical row look as follows except for the actual user name and
> address and the available languages?
>
>     <tr>
>       <td><center><INPUT name="mark%40msapiro.net_unsub"
> type="CHECKBOX" value="off" ></center></td>
>       <td><a
> href="../../options/gpc-test/mark--at--msapiro.net">mark at msapiro.net</a><br><INPUT
> name="mark%40msapiro.net_realname" type="TEXT" value="Mark Sapiro"
> size="22" ><INPUT name="user" type="HIDDEN" value="mark%40msapiro.net"
>> </td>
>       <td><center><INPUT name="mark%40msapiro.net_mod" type="CHECKBOX"
> value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_hide" type="CHECKBOX"
> value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_nomail"
> type="CHECKBOX" value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_ack" type="CHECKBOX"
> value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_notmetoo"
> type="CHECKBOX" value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_nodupes"
> type="CHECKBOX" value="on" CHECKED ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_digest"
> type="CHECKBOX" value="off" ></center></td>
>       <td><center><INPUT name="mark%40msapiro.net_plain"
> type="CHECKBOX" value="off" ></center></td>
>       <td><center>
> <Select name="mark%40msapiro.net_language">
>  <option value="ca"> Catalan </option>
>  <option value="de"> German </option>
>  <option value="el"> Greek </option>
>  <option value="en" Selected> English (USA) </option>
>  <option value="es"> Spanish (Spain) </option>
>  <option value="fr"> French </option>
>  <option value="he"> Hebrew </option>
>  <option value="ja"> Japanese </option>
>  <option value="pt_BR"> Portuguese (Brazil) </option>
>  <option value="ru"> Russian </option>
> </Select></center></td>
>     </tr>
>
>

I hear you.  As I wrote in my followup e-mail I suspect initial list
setup inconsistencies.  I am still waiting to hear what you think about
this theory.
In a meantime I tried to add a new subscriber and was able to delete
them with no issue.

Nov 09 14:57:45 2012 (19561) surcharge-letter: new vlad at hollebcons.com,
admin mass sub
Nov 09 14:58:14 2012 (19663) surcharge-letter: deleted
vlad at hollebcons.com; member mgt page

Mailman version 2.1.15
Pyphon version: 2.7.3




From mark at msapiro.net  Fri Nov  9 22:30:41 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Nov 2012 13:30:41 -0800
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <509D70CA.1040808@hollebcons.com>
Message-ID: <PC195201211091330410022efcf243f@Notebook-09>

Vladimir Mikhelson wrote:
>
>But I did not modify the Apache rewrite rules per
>http://wiki.list.org/display/DOC/4.27+Securing+Mailman%27s+web+GUI+by+using+Secure+HTTP-SSL
>
>Do you think it can be related?


It's possible, but unlikely. If that were the issue, most likely you
wouldn't be able to change the user options either, but are you
redirecting/rewriting from http to https, and if so, if you view the
source of the membership list page, what is the value of the action=
string in the FORM tag?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From vlad at hollebcons.com  Fri Nov  9 22:35:38 2012
From: vlad at hollebcons.com (Vladimir Mikhelson)
Date: Fri, 09 Nov 2012 15:35:38 -0600
Subject: [Mailman-Users] Unsub in the Mmembership List Does Not Work
In-Reply-To: <PC195201211091330410022efcf243f@Notebook-09>
References: <PC195201211091330410022efcf243f@Notebook-09>
Message-ID: <509D772A.9090201@hollebcons.com>


On 11/9/2012 3:30 PM, Mark Sapiro wrote:
> Vladimir Mikhelson wrote:
>> But I did not modify the Apache rewrite rules per
>> http://wiki.list.org/display/DOC/4.27+Securing+Mailman%27s+web+GUI+by+using+Secure+HTTP-SSL
>>
>> Do you think it can be related?
>
> It's possible, but unlikely. If that were the issue, most likely you
> wouldn't be able to change the user options either, but are you
> redirecting/rewriting from http to https, and if so, if you view the
> source of the membership list page, what is the value of the action=
> string in the FORM tag?
>

<FORM action="https://host.domain.tld/mailman/admin/list/members?letter=v <view-source:https://cormonunx.corp.e-magellan.com/mailman/admin/surcharge-letter/members?letter=v>" method="POST" enctype="multipart/form-data">


From mark at msapiro.net  Fri Nov  9 22:49:42 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 9 Nov 2012 13:49:42 -0800
Subject: [Mailman-Users] {Disarmed} Re: Unsub in the Mmembership List
	Does Not Work
In-Reply-To: <509D772A.9090201@hollebcons.com>
Message-ID: <PC195201211091349420963efcf27ed@Notebook-09>

Vladimir Mikhelson wrote:
>
>On 11/9/2012 3:30 PM, Mark Sapiro wrote:
>> Vladimir Mikhelson wrote:
>>> But I did not modify the Apache rewrite rules per
>>> http://wiki.list.org/display/DOC/4.27+Securing+Mailman%27s+web+GUI+by+using+Secure+HTTP-SSL
>>>
>>> Do you think it can be related?
>>
>> It's possible, but unlikely. If that were the issue, most likely you
>> wouldn't be able to change the user options either, but are you
>> redirecting/rewriting from http to https, and if so, if you view the
>> source of the membership list page, what is the value of the action=
>> string in the FORM tag?
>>
>
><FORM action="https://host.domain.tld/mailman/admin/list/members?letter=v


There is a possible problem. It appears that you asked for the source
of an https URL. what do you get if you ask for an http URL?

I don't think it's likely, but if you go to an http URL and get a form
with an action= with an http URL and the web server redirects/rewrites
the POST url to https, it can lose the post data. This is the issue of
the FAQ at <http://wiki.list.org/x/ioA9>.

As I say. I don't think that's likely, but it could explain an
intermittent issue if sometimes you get the page via http and other
times via https.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From misterbhatt at gmail.com  Sat Nov 10 21:43:34 2012
From: misterbhatt at gmail.com (Amit Bhatt)
Date: Sun, 11 Nov 2012 02:13:34 +0530
Subject: [Mailman-Users] Spam mails being received on the mailing list,
	solution needed....
Message-ID: <9CFD012EED964EC4B21474880C67B069@amit>

dear all,

we are receiving around 1000 spam E-mails on a daily basis via one of our mailing lists. Most of these spams are received in form of subscription requests. The spam appearing in a huge scale is badly effecting the performance of the list.
Would appreciate if anyone can suggest us the proper remedies to resolve this issue.

    Regards,

Amit Bhatt



From mark at msapiro.net  Sun Nov 11 00:43:59 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 10 Nov 2012 15:43:59 -0800
Subject: [Mailman-Users] Spam mails being received on the mailing list,
	solution needed....
In-Reply-To: <9CFD012EED964EC4B21474880C67B069@amit>
Message-ID: <PC19520121110154359005004d6186d@Notebook-09>

Amit Bhatt wrote:
>
>we are receiving around 1000 spam E-mails on a daily basis via one of our mailing lists. Most of these spams are received in form of subscription requests. The spam appearing in a huge scale is badly effecting the performance of the list.
>Would appreciate if anyone can suggest us the proper remedies to resolve this issue.


You should use tools like spamassassin, clamav (with 3rd party
signatures such as those from <http://www.sanesecurity.com/>), etc. in
your incoming MTA to stop spam at the front door. Once it get's
delivered to a Mailman list-subscribe or list-join address, it's too
late.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From Ralf.Hildebrandt at charite.de  Sun Nov 11 11:29:58 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Sun, 11 Nov 2012 11:29:58 +0100
Subject: [Mailman-Users] Spam mails being received on the mailing list,
 solution needed....
In-Reply-To: <9CFD012EED964EC4B21474880C67B069@amit>
References: <9CFD012EED964EC4B21474880C67B069@amit>
Message-ID: <20121111102958.GA30392@charite.de>

* Amit Bhatt <misterbhatt at gmail.com>:

> we are receiving around 1000 spam E-mails on a daily basis via one of
> our mailing lists. Most of these spams are received in form of
> subscription requests. 

So only the list admin receives them (not everybody on the list), correct?
And why do they contain spam?

Could you show an example, please?

-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From Ralf.Hildebrandt at charite.de  Sun Nov 11 11:37:17 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Sun, 11 Nov 2012 11:37:17 +0100
Subject: [Mailman-Users] MM-2.1.15: Small bug regarding the "request forgery
	check"
Message-ID: <20121111103717.GB30392@charite.de>

Today I check the subscribers list ("Membership Management...
Section") on one of our lists and called up 
the legend using "Click here to include the legend for this table."

After clicking on that link, the page did indeed display the legend,
but the page now starts with:

Error: The form lifetime has expired. (request forgery check)

-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From odhiambo at gmail.com  Sun Nov 11 12:55:20 2012
From: odhiambo at gmail.com (Odhiambo Washington)
Date: Sun, 11 Nov 2012 14:55:20 +0300
Subject: [Mailman-Users] MM-2.1.15: Small bug regarding the "request
 forgery check"
In-Reply-To: <20121111103717.GB30392@charite.de>
References: <20121111103717.GB30392@charite.de>
Message-ID: <CAAdA2WNvDtcWaNZkRXU1m-6nUOy=yDsxEs4H6qL=E2+k5U+OLg@mail.gmail.com>

On Sun, Nov 11, 2012 at 1:37 PM, Ralf Hildebrandt <
Ralf.Hildebrandt at charite.de> wrote:

> Today I check the subscribers list ("Membership Management...
> Section") on one of our lists and called up
> the legend using "Click here to include the legend for this table."
>
> After clicking on that link, the page did indeed display the legend,
> but the page now starts with:
>
> Error: The form lifetime has expired. (request forgery check)
>
>
+1

Shows in mine too!

-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
I can't hear you -- I'm using the scrambler.

From mark at msapiro.net  Sun Nov 11 15:59:23 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 11 Nov 2012 06:59:23 -0800
Subject: [Mailman-Users] MM-2.1.15: Small bug regarding the "request
 forgery check"
In-Reply-To: <20121111103717.GB30392@charite.de>
References: <20121111103717.GB30392@charite.de>
Message-ID: <509FBD4B.2020408@msapiro.net>

On 11/11/2012 2:37 AM, Ralf Hildebrandt wrote:
> Today I check the subscribers list ("Membership Management...
> Section") on one of our lists and called up 
> the legend using "Click here to include the legend for this table."
> 
> After clicking on that link, the page did indeed display the legend,
> but the page now starts with:
> 
> Error: The form lifetime has expired. (request forgery check)


I found and fixed this several days ago. See
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1366>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From Ralf.Hildebrandt at charite.de  Sun Nov 11 18:38:12 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Sun, 11 Nov 2012 18:38:12 +0100
Subject: [Mailman-Users] MM-2.1.15: Small bug regarding the "request
 forgery check"
In-Reply-To: <509FBD4B.2020408@msapiro.net>
References: <20121111103717.GB30392@charite.de> <509FBD4B.2020408@msapiro.net>
Message-ID: <20121111173812.GF18233@charite.de>

* Mark Sapiro <mark at msapiro.net>:
> On 11/11/2012 2:37 AM, Ralf Hildebrandt wrote:
> > Today I check the subscribers list ("Membership Management...
> > Section") on one of our lists and called up 
> > the legend using "Click here to include the legend for this table."
> > 
> > After clicking on that link, the page did indeed display the legend,
> > but the page now starts with:
> > 
> > Error: The form lifetime has expired. (request forgery check)
> 
> 
> I found and fixed this several days ago. See
> <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1366>.

Thanks. WOrking OK for me :)

-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From odhiambo at gmail.com  Sun Nov 11 19:09:28 2012
From: odhiambo at gmail.com (Odhiambo Washington)
Date: Sun, 11 Nov 2012 21:09:28 +0300
Subject: [Mailman-Users] MM-2.1.15: Small bug regarding the "request
 forgery check"
In-Reply-To: <20121111173812.GF18233@charite.de>
References: <20121111103717.GB30392@charite.de> <509FBD4B.2020408@msapiro.net>
	<20121111173812.GF18233@charite.de>
Message-ID: <CAAdA2WOHiB5a3SAdmMfEGjWu4z-sb5susnPQVaORV9KiLsFX0Q@mail.gmail.com>

On Sun, Nov 11, 2012 at 8:38 PM, Ralf Hildebrandt <
Ralf.Hildebrandt at charite.de> wrote:

> * Mark Sapiro <mark at msapiro.net>:
> > On 11/11/2012 2:37 AM, Ralf Hildebrandt wrote:
> > > Today I check the subscribers list ("Membership Management...
> > > Section") on one of our lists and called up
> > > the legend using "Click here to include the legend for this table."
> > >
> > > After clicking on that link, the page did indeed display the legend,
> > > but the page now starts with:
> > >
> > > Error: The form lifetime has expired. (request forgery check)
> >
> >
> > I found and fixed this several days ago. See
> > <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1366>.
>
> Thanks. WOrking OK for me :)
>
>
> +1 :)

-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
I can't hear you -- I'm using the scrambler.

From rodrigoantunes at pelotas.ifsul.edu.br  Mon Nov 12 19:59:48 2012
From: rodrigoantunes at pelotas.ifsul.edu.br (Rodrigo Abrantes Antunes)
Date: Mon, 12 Nov 2012 16:59:48 -0200
Subject: [Mailman-Users] Non-member posting to the list
Message-ID: <20121112165948.Horde.dIWfEkv4Cn9QoUckjA2XXXA@webmail.pelotas.ifsul.edu.br>

Hi, I was configuring a server with Joomla and AcyMailing to send mass
emails and then I realized that if I configure the sender address as being
one of a non moderated user it could send a message to a list without the
non moderated user knows. How can I avoid that? Isn't there any
configuration in mailman that prevents messages with forged from headers to
be posted to the list?

  Thanks.

From mark at msapiro.net  Tue Nov 13 07:52:49 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 Nov 2012 22:52:49 -0800
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <20121112165948.Horde.dIWfEkv4Cn9QoUckjA2XXXA@webmail.pelotas.ifsul.edu.br>
Message-ID: <PC1952012111222524905931c23e8c2@MSAPIRO>

Rodrigo Abrantes Antunes wrote:

>Hi, I was configuring a server with Joomla and AcyMailing to send mass
>emails and then I realized that if I configure the sender address as being
>one of a non moderated user it could send a message to a list without the
>non moderated user knows. How can I avoid that?


See the FAQ at <http://wiki.list.org/x/XIA9>.


>Isn't there any
>configuration in mailman that prevents messages with forged from headers to
>be posted to the list?


If I knew how to tell if a header was spoofed, I could do that, but I
don't know how to tell; do you?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From tanstaafl at libertytrek.org  Tue Nov 13 13:22:34 2012
From: tanstaafl at libertytrek.org (Tanstaafl)
Date: Tue, 13 Nov 2012 07:22:34 -0500
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <PC1952012111222524905931c23e8c2@MSAPIRO>
References: <PC1952012111222524905931c23e8c2@MSAPIRO>
Message-ID: <50A23B8A.9050307@libertytrek.org>

On 2012-11-13 1:52 AM, Mark Sapiro <mark at msapiro.net> wrote:
> If I knew how to tell if a header was spoofed, I could do that, but I
> don't know how to tell; do you?

Maybe an alternative would be an option that for every message posted to 
the list, a confirmation email is sent to the members email address, 
that they then have to click a link to 'approve' sending the message, 
just like how subscribes/unsubscribes have to confirmed.

Maybe this could even be extended with some kind of way of cahing the 
source IP of approved messages, so when messages come in with the same 
sender and from the same IP that has already been approved, those 
messages go straight through without requiring confirmation?

Not a good option for really high volume lists with lots of members, but 
for smaller orgs, maybe a viable option?

Just thinking out loud, because this has definitely been a problem on 
our end (I've even had to set the emergency moderation bit a few times 
until these idiots stopped spamming the list).

I also just noticed the option under the Privacy > Spam controls in the 
GUI under 'Legacy anti-spam filters' where I can enter the listname 
itself, to prevent anyone sending spoofed messages from the list to the 
list.

From listeyon at metu.edu.tr  Tue Nov 13 14:44:25 2012
From: listeyon at metu.edu.tr (METU E-List Admin)
Date: Tue, 13 Nov 2012 15:44:25 +0200
Subject: [Mailman-Users] A question about IncomingRunner
Message-ID: <50A24EB9.6040708@metu.edu.tr>

Hello,

We are using Mailman 2.1.13 from Debian repositories (which is the 
latest version for squeeze). We are facing late mail delivery problems 
and after some research, we think we managed to overcome these problems 
by changing some configuration options. Although Mailman seems to 
running fine, we have observed an issue.

The issue is related with IncomingRunner. This qrunner process takes up 
approx. 1 GB RAM and sometimes causes high loads. We couldn't determine 
what causes high RAM and CPU usages.

The question is; this is an expected result of Mailman process? If not, 
how can we determine the root cause of this problem?

Thanks,

From mark at msapiro.net  Tue Nov 13 17:13:45 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 Nov 2012 08:13:45 -0800
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <50A23B8A.9050307@libertytrek.org>
Message-ID: <PC19520121113081345023135f7b114@Notebook-09>

Tanstaafl wrote:
>
>Maybe an alternative would be an option that for every message posted to 
>the list, a confirmation email is sent to the members email address, 
>that they then have to click a link to 'approve' sending the message, 
>just like how subscribes/unsubscribes have to confirmed.


That's an interesting idea. Personally, if I enabled that feature on
any of my lists, I'd be taken out and shot, and I'd want to do the
same to anyone who did it on any list I'm a member of, but if people
want such a list configuration option, we could consider it for MM 3.


>Maybe this could even be extended with some kind of way of cahing the 
>source IP of approved messages, so when messages come in with the same 
>sender and from the same IP that has already been approved, those 
>messages go straight through without requiring confirmation?


That would help some. Sort of like greylisting with a twist, but I
think it would still be unacceptable to many list members.


[...]
>
>I also just noticed the option under the Privacy > Spam controls in the 
>GUI under 'Legacy anti-spam filters' where I can enter the listname 
>itself, to prevent anyone sending spoofed messages from the list to the 
>list.


Yes, and you could also use header_filter_rules for this, but you have
bigger problems if the list posting address is a member of the list,
so ordinary non_member actions should handle this.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From rodrigoantunes at pelotas.ifsul.edu.br  Tue Nov 13 19:03:32 2012
From: rodrigoantunes at pelotas.ifsul.edu.br (Rodrigo Abrantes Antunes)
Date: Tue, 13 Nov 2012 16:03:32 -0200
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <PC19520121113081345023135f7b114@Notebook-09>
References: <PC19520121113081345023135f7b114@Notebook-09>
Message-ID: <20121113160332.Horde.42wJd0v4Cn9Qoot0k11A3yA@webmail.pelotas.ifsul.edu.br>

Citando Mark Sapiro <mark at msapiro.net>:
> Tanstaafl wrote:  > Maybe an alternative would be an option that for  
> every message posted to
>>    the list, a confirmation email is sent to the members email address,
>>    that they then have to click a link to 'approve' sending the message,
>>    just like how subscribes/unsubscribes have to confirmed.
>   That's an interesting idea. Personally, if I enabled that feature on
>   any of my lists, I'd be taken out and shot, and I'd want to do the
>   same to anyone who did it on any list I'm a member of, but if people
>   want such a list configuration option, we could consider it for MM 3.
>
>> Maybe this could even be extended with some kind of way of cahing the
>>    source IP of approved messages, so when messages come in with the same
>>    sender and from the same IP that has already been approved, those
>>    messages go straight through without requiring confirmation?
>   That would help some. Sort of like greylisting with a twist, but I
>   think it would still be unacceptable to many list members.
>
>
>   [...]  > I also just noticed the option under the Privacy > Spam  
> controls in the
>>    GUI under 'Legacy anti-spam filters' where I can enter the listname
>>    itself, to prevent anyone sending spoofed messages from the list to the
>>    list.
>   Yes, and you could also use header_filter_rules for this, but you have
>   bigger problems if the list posting address is a member of the list,
>   so ordinary non_member actions should handle this.
>
>   --
>   Mark Sapiro <mark at msapiro.net>? ? ? ? The highway is for gamblers,
>   San Francisco Bay Area, California? ? better use your sense - B. Dylan
>
>   ------------------------------------------------------
>   Mailman-Users mailing list Mailman-Users at python.org
> http://mail.python.org/mailman/listinfo/mailman-users
>   Mailman FAQ: http://wiki.list.org/x/AgA3
>   Security Policy: http://wiki.list.org/x/QIA9
>   Searchable Archives:  
> http://www.mail-archive.com/mailman-users%40python.org/Unsubscribe:  
> http://mail.python.org/mailman/options/mailman-users/rodrigoantunes%40pelotas.ifsul.edu.br

In my case, I found that the return-path header is the address of the
original sender, so how could I add a rule in mailman to deny posts with
return-path's address that are not members?

From mailman at veggiechinese.net  Tue Nov 13 23:23:02 2012
From: mailman at veggiechinese.net (Will Yardley)
Date: Tue, 13 Nov 2012 14:23:02 -0800
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <20121113160332.Horde.42wJd0v4Cn9Qoot0k11A3yA@webmail.pelotas.ifsul.edu.br>
References: <PC19520121113081345023135f7b114@Notebook-09>
	<20121113160332.Horde.42wJd0v4Cn9Qoot0k11A3yA@webmail.pelotas.ifsul.edu.br>
Message-ID: <20121113222302.GG12756@aura.veggiechinese.net>

On Tue, Nov 13, 2012 at 04:03:32PM -0200, Rodrigo Abrantes Antunes wrote:
> In my case, I found that the return-path header is the address of the
> original sender, so how could I add a rule in mailman to deny posts with
> return-path's address that are not members?

The envelope-sender can also be spoofed trivially.

If you want to prevent someone from sending email as someone who *is*
approved to post to the list, I think your safest bet is to require
approval for all posts to the list -- in other words, set the action for
posts by moderated members allowed to post to 'hold', and have the
moderate bit set even for users who are allowed to post.

w


From mark at msapiro.net  Wed Nov 14 02:32:15 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 Nov 2012 17:32:15 -0800
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <20121113222302.GG12756@aura.veggiechinese.net>
Message-ID: <PC19520121113173215069038a1b6c1@Notebook-09>

Will Yardley wrote:

>On Tue, Nov 13, 2012 at 04:03:32PM -0200, Rodrigo Abrantes Antunes wrote:
>> In my case, I found that the return-path header is the address of the
>> original sender, so how could I add a rule in mailman to deny posts with
>> return-path's address that are not members?
>
>The envelope-sender can also be spoofed trivially.
>
>If you want to prevent someone from sending email as someone who *is*
>approved to post to the list, I think your safest bet is to require
>approval for all posts to the list -- in other words, set the action for
>posts by moderated members allowed to post to 'hold', and have the
>moderate bit set even for users who are allowed to post.


Yes. As indicated in the FAQ I referred to in my original reply, the
safe way to do this is to moderate everyone or otherwise arrange for
all list posts to be held. Then authorized posts can be sent with an
Approved: <password> header or first body line pseudo-header to bypass
the hold. <password> is the list admin password, the moderator
password or, beginning in Mailman 2.1.15, the special list poster
password.

If you are the site admin, you can require that only the envelope
sender (the address reflected in Return-Path:) be recognized in
determining list membership by putting

SENDER_HEADERS = (None,)

in mm_cfg.py. See the documentation for this setting in Defaults.py.

It would probably also be possible to create a regexp for
header_filter_rules that would match only when the Return-Path:
address and the From: address were different and to use that to deal
with such posts. But, that wouldn't handle the case where both From:
and envelope sender were spoofed with the same address.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Wed Nov 14 02:58:32 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 Nov 2012 17:58:32 -0800
Subject: [Mailman-Users] A question about IncomingRunner
In-Reply-To: <50A24EB9.6040708@metu.edu.tr>
Message-ID: <PC19520121113175832026538a1b529@Notebook-09>

METU E-List Admin wrote:
>
>We are using Mailman 2.1.13 from Debian repositories (which is the 
>latest version for squeeze). We are facing late mail delivery problems 
>and after some research, we think we managed to overcome these problems 
>by changing some configuration options.


I don't know what you did or what the issue was, but delivery delays in
Mailman are usually caused by OutgoingRunner being backlogged and
fixed by tuning the MTA to speed up acceptance of list mail.


>Although Mailman seems to 
>running fine, we have observed an issue.
>
>The issue is related with IncomingRunner. This qrunner process takes up 
>approx. 1 GB RAM and sometimes causes high loads. We couldn't determine 
>what causes high RAM and CPU usages.
>
>The question is; this is an expected result of Mailman process? If not, 
>how can we determine the root cause of this problem?


I'd need more information to speak to high CPU usage, but there are a
couple of factors that cause runners to grow large

The basic issue is with Python's memory management. If a Python process
grows large because there is some object that requires a lot of
memory, when that object is no longer referenced, the memory is freed
within the Python process and is available for reuse, but unused
memory is not freed from the process and returned to the OS. Thus, a
Python process can grow to accommodate large objects, but even when
that memory is freed within the process, the process doesn't shrink.

Thus, if IncomingRunner processes a very large message, it will grow
large and remain so until restarted. This can happen with some spam,
even though the spam message is ultimately discarded.

Another issue is with Mailman prior to 2.1.15 and can affect all
runners.

In Mailman prior to 2.1.15, the runners kept an in memory list cache.
For various technical reasons, cached list objects were never freed
from the cache. Thus, runners would gradually grow until they held a
copy of every list object. If you have lots of lists, or even a few
lists with lots of members, runners could grow quite large for this
reason.

Note that in both these scenarios, even though the process is large, if
memory is scarce in the server, much of this memory will be only on
backing store and will not have a significant impact on server RAM
usage. See the FAQ at <http://wiki.list.org/x/94A9>.

As far as CPU is concerned, normally, OutgoingRunner uses the most (at
least in a VERPed environment) and ArchRunner uses some and the
others, not much.

Here's a `ps` showing CPU for a smallish installation that's been
running for 11 days.

UID        PID  PPID  C STIME TTY          TIME CMD
mailman  27691     1  0 Nov02 ?        00:00:00 /usr/bin/python2.6
/usr/local/mailman/bin/mailmanctl -s -q start
mailman  27692 27691  0 Nov02 ?        00:01:14 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
mailman  27693 27691  0 Nov02 ?        00:00:03 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
mailman  27694 27691  0 Nov02 ?        00:00:03 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
mailman  27695 27691  0 Nov02 ?        00:00:07 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
mailman  27696 27691  0 Nov02 ?        00:00:02 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
mailman  27697 27691  0 Nov02 ?        00:02:37 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
mailman  27698 27691  0 Nov02 ?        00:00:05 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
mailman  27699 27691  0 Nov02 ?        00:00:00 /usr/bin/python2.6
/usr/local/mailman/bin/qrunner --runner=RetryRunner:0:1 -s

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From stephen at xemacs.org  Wed Nov 14 04:58:28 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 14 Nov 2012 12:58:28 +0900
Subject: [Mailman-Users] Non-member posting to the list
In-Reply-To: <50A23B8A.9050307@libertytrek.org>
References: <PC1952012111222524905931c23e8c2@MSAPIRO>
	<50A23B8A.9050307@libertytrek.org>
Message-ID: <87a9uk96or.fsf@uwakimon.sk.tsukuba.ac.jp>

Tanstaafl writes:
 > On 2012-11-13 1:52 AM, Mark Sapiro <mark at msapiro.net> wrote:
 > > If I knew how to tell if a header was spoofed, I could do that, but I
 > > don't know how to tell; do you?
 > 
 > Maybe an alternative would be an option that for every message posted to 
 > the list, a confirmation email is sent to the members email
 > address, 

We already have various challenge-response mechanisms (eg TMDA).
They're widely hated even more than spam (eg, I simply drop those
correspondents on the floor and add their CR addresses to my killfile
in case of inadvertant CCs).

I think the best practical algorithm would look something like the
following:

1.  In the double opt-in process require an email confirmation (not by
    web).  This could also be delayed to first post, but they'll be
    dealing with the confirmation process in their MUA.  This could
    still be done with a link but it would be a mailto: rather than an
    http: link.
2.  Get originator information (From, Sender, envelope sender, and
    earliest Received, and SPF and DKIM information where available).
3.  Record the configuration.
4.  For every post from a member with new originator information,
    update the member information with a new originator record.
5.  If spam is received corresponding to an originator record, disable
    it.  This might be automated through the moderation process, or
    through milters (which one would hope catch most of the spam).
6.  Analyze originator information and issue a challenge whenever a
    post claiming to be from a member matches disabled originator
    information on file.  (Definition of "match" is non-trivial and
    probably necessarily heuristic.)  Otherwise approve the post.
7.  In case of challenge, if an approval response is received, warn
    the member that their address has been used to spam.

You could try reversing the polarity of step 5, and require
confirmation for every new originator record.  But that would probably
be too annoying.  Too many people have multiple locations they post
from, even if they use only one address.

 > that they then have to click a link to 'approve' sending the message, 
 > just like how subscribes/unsubscribes have to confirmed.
 > 
 > Maybe this could even be extended with some kind of way of cahing the 
 > source IP of approved messages,

I don't think this is an extension, I think it's absolutely necessary.

 > I also just noticed the option under the Privacy > Spam controls in
 > the GUI under 'Legacy anti-spam filters' where I can enter the
 > listname itself, to prevent anyone sending spoofed messages from
 > the list to the list.

Maybe this should be on by default.

Steve

From lucio at lambrate.inaf.it  Wed Nov 14 12:34:10 2012
From: lucio at lambrate.inaf.it (Lucio Chiappetti)
Date: Wed, 14 Nov 2012 12:34:10 +0100 (CET)
Subject: [Mailman-Users] mass edit on a mailing list
Message-ID: <alpine.LSU.2.00.1211141229070.3438@cbfrvqba.ynzoengr.vans.vg>

Let us assume that a substantial fraction of members of a nationwide 
mailing list (say about 110 on 1135) have changed domain, i.e.

   user at domain1.tld and user at domain2.tld

have now migrated to user at domain2.tld

Is there a way by which the *moderators* (which are sparse in different 
institutions and have access to the mailing list only via the web 
interface) can apply a mass edit (changing domain1 and domain2 into 
domain3) ?

Or can that be done via line-mode commands only by the *administrator*
(who has the list server in house) ?

-- 
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
------------------------------------------------------------------------
Italian Research STILL at risk.  La Ricerca italiana TUTTORA a rischio !
see http://sax.iasf-milano.inaf.it/~lucio/WWW/Opinions/nobrain3.html cfr

From mark at msapiro.net  Wed Nov 14 17:56:23 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 14 Nov 2012 08:56:23 -0800
Subject: [Mailman-Users] mass edit on a mailing list
In-Reply-To: <alpine.LSU.2.00.1211141229070.3438@cbfrvqba.ynzoengr.vans.vg>
Message-ID: <PC1952012111408562303373a75bd68@Notebook-09>

Lucio Chiappetti wrote:
>
>Is there a way by which the *moderators* (which are sparse in different 
>institutions and have access to the mailing list only via the web 
>interface) can apply a mass edit (changing domain1 and domain2 into 
>domain3) ?


Moderators, i.e. people who have only web access to the list and know
only the moderator password, cannot make any membership changes. They
can only handle held messages and requests via the admindb interface.

People who have only web access to the list and know the administrator
password can visit the admin pages and make changes, but even they
cannot directly change a members address without confirmation from the
member, and there is no function for mass change of domain.


>Or can that be done via line-mode commands only by the *administrator*
>(who has the list server in house) ?


The server doesn't have to be "in house", but one does need command
line access and write permission on the Mailman installation. If one
has such access, the script at
<http://www.msapiro.net/scripts/change_member_address.py> can do
exactly what you want.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From jdanield at free.fr  Wed Nov 14 18:11:17 2012
From: jdanield at free.fr (jdd)
Date: Wed, 14 Nov 2012 18:11:17 +0100
Subject: [Mailman-Users] mass edit on a mailing list
In-Reply-To: <PC1952012111408562303373a75bd68@Notebook-09>
References: <PC1952012111408562303373a75bd68@Notebook-09>
Message-ID: <50A3D0B5.3080501@free.fr>

Le 14/11/2012 17:56, Mark Sapiro a ?crit :

> People who have only web access to the list and know the administrator
> password can visit the admin pages and make changes, but even they
> cannot directly change a members address without confirmation from the
> member, and there is no function for mass change of domain.

can't you:

* extract the list
* edit it in any editor
* mass subscribe with the edited list
* eventually remove the old list (or wait to bounce mechanism to do it?

?

just an idea :-)

jdd


-- 
http://www.dodin.org
http://jddtube.dodin.org/20120616-52-highway_v1115

From mark at msapiro.net  Wed Nov 14 18:41:15 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 14 Nov 2012 09:41:15 -0800
Subject: [Mailman-Users] mass edit on a mailing list
In-Reply-To: <50A3D0B5.3080501@free.fr>
References: <PC1952012111408562303373a75bd68@Notebook-09>
	<50A3D0B5.3080501@free.fr>
Message-ID: <50A3D7BB.2090900@msapiro.net>

On 11/14/2012 9:11 AM, jdd wrote:
> Le 14/11/2012 17:56, Mark Sapiro a ?crit :
> 
>> People who have only web access to the list and know the administrator
>> password can visit the admin pages and make changes, but even they
>> cannot directly change a members address without confirmation from the
>> member, and there is no function for mass change of domain.
> 
> can't you:
> 
> * extract the list
> * edit it in any editor
> * mass subscribe with the edited list
> * eventually remove the old list (or wait to bounce mechanism to do it?


Yes, but this is not "directly" changing the member's addresses. It is
adding the new addresses and removing the old addresses and in the
process reverting all of the users' options to defaults, assigning them
a new, random password and possibly sending them goodbye and/or welcome
messages.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From karra.etc at gmail.com  Wed Nov 14 18:42:45 2012
From: karra.etc at gmail.com (Sriram Karra)
Date: Wed, 14 Nov 2012 23:12:45 +0530
Subject: [Mailman-Users] Different "output" between web interface and
	command line tool
Message-ID: <CAFkt3UP2g2pF1U5qjFdX3d7Y9q8PeGHNefUK49KoctD_YrXHew@mail.gmail.com>

I have a test installation of Mailman on a machine and created a list
(called mas) that I have marked Yes for "Advertise this list when people
ask what lists are on this machine?"  I can see the Yes option when I view
the listinfo as /mailman/listinfo/mas. However when this list is not listed
on /mailman/listinfo where I get a message saying there are no publicly
visible lists on this server. However running bin/list_lists -a does list
mas as one of the advertised lists.

No errors are logged in mailman/logs/error. Can someone help me understand
what is going on?

Regards,
Sriram

From mark at msapiro.net  Wed Nov 14 18:54:14 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 14 Nov 2012 09:54:14 -0800
Subject: [Mailman-Users] Different "output" between web interface
	andcommand line tool
In-Reply-To: <CAFkt3UP2g2pF1U5qjFdX3d7Y9q8PeGHNefUK49KoctD_YrXHew@mail.gmail.com>
Message-ID: <PC195201211140954140055ec60ac45@Notebook-09>

Sriram Karra wrote:

>I have a test installation of Mailman on a machine and created a list
>(called mas) that I have marked Yes for "Advertise this list when people
>ask what lists are on this machine?"  I can see the Yes option when I view
>the listinfo as /mailman/listinfo/mas. However when this list is not listed
>on /mailman/listinfo where I get a message saying there are no publicly
>visible lists on this server.


See the FAQ at <http://wiki.list.org/x/6oA9> and other FAQs linked
therefrom.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From listmail at b79.net  Wed Nov 14 21:38:37 2012
From: listmail at b79.net (John Magolske)
Date: Wed, 14 Nov 2012 12:38:37 -0800
Subject: [Mailman-Users] Download pipermail archives,
 convert to mbox file (script)
In-Reply-To: <PC1952012101714252604640dd569ea@MSAPIRO>
References: <20121017194504.GA8588@s70206.gridserver.com>
	<PC1952012101714252604640dd569ea@MSAPIRO>
Message-ID: <20121114203837.GA19191@s70206.gridserver.com>

* Mark Sapiro <mark at msapiro.net> [121017 16:57]:
> If the list's archive is public and you are not a subscriber, your
> script is probably fine (I didn't look in detail), but if you are
> willing to subscribe first, whether the archives are private or
> public, you can get the list's entire cummulative mbox archive with
> something like
> 
> wget
> 'http://www.example.com/mailman/private/LIST.mbox/LIST.mbox?username=U&password=P'
> 
> where LIST is the list name, U is a list member's address and P is that
> member's list password. This has the advantage of getting all the
> message's headers as processed by Mailman with the exception of those
> added by SMTPDirect.py (Sender: and Errors-To:), not just those few
> that are in the periodic .txt or .txt.gz files.

Thanks, this is great for catching up on subscribed-to lists. I just
used this to download the entire history of mailman-users into one
247MB mbox file. The only post-processing required involved removing
the first line (which was blank) of the file.

Question -- does that comprehensive mbox file exist on the server
somewhere (ie, not generated per request)? I'm wondering if it'd be
possible to set up rsync to do incremental updates and mirror backups
of an archive to other locations. I'm guessing rsync's delta-transfer
algorithm would use roughly the same amount of bandwidth as SMTP...
though it would re-write the entire mbox file at the destination with
each sync.

But also, I was thinking this could be used to fill gaps in list
traffic (when away from the net for extended periods of time & the
inbox exceeds number of allowed messages, mail server goes down for
some reason, etc.), offering a way to sync up without re-downloading
a potentially huge file. But maybe in this case a scheme for limiting
the download to a certain date range similar to how Gmane allows
setting a range of message numbers in a download URL [1] would make
more sense. Is there such a functionality in Mailman?

[1] http://gmane.org/export.php 

Regards,

John

-- 
John Magolske
http://B79.net/contact

From mark at msapiro.net  Wed Nov 14 22:51:53 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 14 Nov 2012 13:51:53 -0800
Subject: [Mailman-Users] Download pipermail archives,
	convert to mbox file (script)
In-Reply-To: <20121114203837.GA19191@s70206.gridserver.com>
Message-ID: <PC195201211141351530704572a5ef5@Notebook-09>

John Magolske wrote:
>
>Question -- does that comprehensive mbox file exist on the server
>somewhere (ie, not generated per request)?


Yes. It is archives/private/LISTNAME.mbox/LISTNAME.mbox.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From karra.etc at gmail.com  Thu Nov 15 03:43:23 2012
From: karra.etc at gmail.com (Sriram Karra)
Date: Thu, 15 Nov 2012 08:13:23 +0530
Subject: [Mailman-Users] Different "output" between web interface
 andcommand line tool
In-Reply-To: <PC195201211140954140055ec60ac45@Notebook-09>
References: <CAFkt3UP2g2pF1U5qjFdX3d7Y9q8PeGHNefUK49KoctD_YrXHew@mail.gmail.com>
	<PC195201211140954140055ec60ac45@Notebook-09>
Message-ID: <CAFkt3UN_cXg8pmkA9hgb6wtw8QBBhW9v6VgeeoC5pG+-qNkX2w@mail.gmail.com>

On Wed, Nov 14, 2012 at 11:24 PM, Mark Sapiro <mark at msapiro.net> wrote:

> Sriram Karra wrote:
>
> >I have a test installation of Mailman on a machine and created a list
> >(called mas) that I have marked Yes for "Advertise this list when people
> >ask what lists are on this machine?"  I can see the Yes option when I view
> >the listinfo as /mailman/listinfo/mas. However when this list is not
> listed
> >on /mailman/listinfo where I get a message saying there are no publicly
> >visible lists on this server.
>
>
> See the FAQ at <http://wiki.list.org/x/6oA9> and other FAQs linked
> therefrom.


Thank you. It was exactly what I was looking for.

Regards,
Sriram.

From rick at blaisedrake.com  Wed Nov 14 17:50:54 2012
From: rick at blaisedrake.com (Rick Drake)
Date: Wed, 14 Nov 2012 11:50:54 -0500
Subject: [Mailman-Users] Migrate from EZMLM to Mailman
Message-ID: <00ee01cdc288$48bd2ab0$da378010$@blaisedrake.com>

Hello,

Does anyone have experience migrating from EZMLM(Qmail) to Mailman?  We have
several lists
And all the notifications already setup and running on EZMLM.  Ideally we
would like to migrate
All the settings to Mailman.

If you can assist in this migration or recommend someone who can assist, it
would be 
greatly appreciated.

Thanks and if you have any questions, feel free to contact me directly at
(610) 685-9700, ext 112.

Thanks,

Rick

******************************************
Blaise Drake & Company
Phone:  (610) 685-9700, ext 112
Email:   rick at blaisedrake.com   



From hawat.thufir at gmail.com  Fri Nov 16 04:38:12 2012
From: hawat.thufir at gmail.com (Thufir)
Date: Thu, 15 Nov 2012 19:38:12 -0800
Subject: [Mailman-Users] what is a virtual domain?
Message-ID: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>

How does a virtual domain differ from a domain?

6.1.2 Virtual domains

Note: This section describes how to integrate Mailman with Postfix for
automatic generation of Postfix virtual_alias_maps for Mailman list
addresses. Mailman's support of virtual domains is limited in that list
names must be globally unique within a single Mailman instance, i.e., two
lists may not have the same name even if they are in different domains.

http://www.gnu.org/software/mailman/mailman-install/postfix-virtual.html


What I'm looking to do is to use postfix "vanity domains".  A vanity domain
is the same thing as a virtual domain?  These are syntactically valid
domains:

"Postfix on hosts without a real Internet hostname

This section is for hosts that don't have their own Internet hostname.
Typically these are systems that get a dynamic IP address via DHCP or via
dialup. "

http://www.postfix.org/SOHO_README.html#fantasy


Such as those available for free from no-ip.com?  On no-ip.com it's just
called a host, and you can add hosts from a drop down of domains which
no-ip.com owns.  I would call it a sub-domain.  The lingo is throwing me a
bit.

In this event, just enter the FQDN from no-ip.com into mailman and let
postfix handle the rest?

thanks,

Thufir

From mark at msapiro.net  Fri Nov 16 17:45:31 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 16 Nov 2012 08:45:31 -0800
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
Message-ID: <PC195201211160845310737199e6b00@Notebook-09>

Thufir wrote:

>How does a virtual domain differ from a domain?


As far as Mailman is concerned, there is no difference. GNU-Mailman
2.1.x can handle list addresses in multiple domains with the
restriction that the list names themselves must be globally unique. In
Mailman, in some contexts the various domains are referred to as
'virtual', but they can be thought of as simply domains or host names.

The difference comes about in the MTA and perhaps the web server. In
Postfix for example, some domains may be local and some virtual. The
local domain Mailman addresses can be delivered to Mailman via aliases
only. The virtual domain Mailman addresses need some kind of virtual
address mapping as well. Sometimes the choice of local vs virtual for
a domain in Postfix can be somewhat arbitrary. What it boils down to
is if delivery to local_part at domain_1.example.net should be handled
differently from delivery to local_part at local_domain.example.com, then
domain_1.example.net needs to be virtual.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From fmouse-mailman at fmp.com  Fri Nov 16 18:56:50 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Fri, 16 Nov 2012 11:56:50 -0600
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
References: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
Message-ID: <1353088610.5757.196.camel@pudina.fmp.com>

On Thu, 2012-11-15 at 19:38 -0800, Thufir wrote:
> How does a virtual domain differ from a domain?

To add to what Mark said, technically a "virtual domain" is a domain
name that resolves in the domain name system to the same IP address as
the primary ("real") domain name for your host.  A server host can have
many domain names, all pointing to the same IP address, and this is
common for web and mail servers these days.

If the protocols supporting the service to the host support it, the
particular service agent (mail server, web server, whatever) can
determine the name by which it was addressed and do something
intelligent with it.  This ability is built into the HTTP1.1 standard,
and also into the SMTP standard.  A list server sits behind a mail
server, so technically it _could_ do something with the virtual domain
name.  It all depends on the way the service agent handles this
information.

-- 
Lindsay Haisley       | "We have met the enemy and he is us."
FMP Computer Services |
512-259-1190          |          -- Pogo
http://www.fmp.com    |


From Kirsten.Petersen at oregonstate.edu  Sat Nov 17 01:01:57 2012
From: Kirsten.Petersen at oregonstate.edu (Petersen, Kirsten J - NET)
Date: Sat, 17 Nov 2012 00:01:57 +0000
Subject: [Mailman-Users] Automated Subscription Bots Inundating List
 Owners With Subscription Requests
Message-ID: <FFD6705BDF9330469C66AC6C3FAAC1290E8661EC@EX1.oregonstate.edu>

Gary, et al:

The Mailman lists at Oregon State University have been receiving excessive request for subscriptions since mid-October as well.  Our list administrators were suspicious because often the names on the requests did not match the email addresses.  Also, many lists that had been defunct for years were receiving requests, too.

I spent some time trying to figure out what the lists that were being hit had in common.  Not all of the lists receiving requests were advertised on the listinfo page.  Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve".  So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.

My folks are beating down my door for a solution, too, and I can't think of a good one.  We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use.  I am going to advise my list admins to enable confirmation, which should discourage these attempts.  It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.

If anyone else has any bright ideas about this problem, I would love to hear it.

-Kirsten Petersen
Network Services, Oregon State University
http://oregonstate.edu/is/services/network-services
net at oregonstate.edu<mailto:net at oregonstate.edu> (7-HELP, option 2)
itconsult at oregonstate.edu<mailto:itconsult at oregonstate.edu> (7-4710)

From mark at msapiro.net  Sat Nov 17 17:33:03 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 17 Nov 2012 08:33:03 -0800
Subject: [Mailman-Users] Automated Subscription Bots Inundating
	ListOwners With Subscription Requests
In-Reply-To: <FFD6705BDF9330469C66AC6C3FAAC1290E8661EC@EX1.oregonstate.edu>
Message-ID: <PC1952012111708330304940f1771f1@Notebook-09>

Petersen, Kirsten J - NET wrote:
>
>Today I realized that all of the lists involved in this attack have their subscribe_policy set to just "require approval" rather "confirm" or "confirm and approve".  So I think the theory that spammers were just trying to get on the lists to harvest member addresses is probably correct.


Most likely, they are hitting all your lists but not answering
confirmation requests because the bots don't know how or the
confirmation requests are going to invalid or spoofed addresses.


>My folks are beating down my door for a solution, too, and I can't think of a good one.  We host lists for the international community, so any measure I take that makes it harder for external people to subscribe will negatively impact intended use.  I am going to advise my list admins to enable confirmation, which should discourage these attempts.


It seems this is a solution.


>It also occurred to me that I could write a script to monitor the vette log and purge requests that look suspicious - mainly based on the same email address attempting to subscribe to multiple unrelated lists at the same time.
>
>If anyone else has any bright ideas about this problem, I would love to hear it.


For some time, there has been a withlist script, discard_address.py, at
<http://www.msapiro.net/scripts/> (mirrored at
<http://fog.ccsf.cc.ca.us/~msapiro/scripts/>) which would discard all
subscription requests and help posts from a specific address. While
this it probably not too useful here, I have just created a new
discard_subs.py script available at the same place which will discard
all held subscription requests older than N days (can be 0) for a list
or all lists.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From sourtooth at gmail.com  Sat Nov 17 18:49:03 2012
From: sourtooth at gmail.com (Ben Cooksley)
Date: Sun, 18 Nov 2012 06:49:03 +1300
Subject: [Mailman-Users] Automated Subscription Bots Inundating List
 Owners With Subscription Requests
In-Reply-To: <FFD6705BDF9330469C66AC6C3FAAC1290E8661EC@EX1.oregonstate.edu>
References: <FFD6705BDF9330469C66AC6C3FAAC1290E8661EC@EX1.oregonstate.edu>
Message-ID: <CA+XidOHuuXYUiriiYoeUjPpq2EBzg7TXijF1fDr3_gC+QzHm0g@mail.gmail.com>

[Post by list member from an unsubscribed address]

On Nov 18, 2012 4:07 AM, "Petersen, Kirsten J - NET" <
Kirsten.Petersen at oregonstate.edu> wrote:
>
> Gary, et al:
>
> The Mailman lists at Oregon State University have been receiving
excessive request for subscriptions since mid-October as well.  Our list
administrators were suspicious because often the names on the requests did
not match the email addresses.  Also, many lists that had been defunct for
years were receiving requests, too.
>
> I spent some time trying to figure out what the lists that were being hit
had in common.  Not all of the lists receiving requests were advertised on
the listinfo page.  Today I realized that all of the lists involved in this
attack have their subscribe_policy set to just "require approval" rather
"confirm" or "confirm and approve".  So I think the theory that spammers
were just trying to get on the lists to harvest member addresses is
probably correct.
>
> My folks are beating down my door for a solution, too, and I can't think
of a good one.  We host lists for the international community, so any
measure I take that makes it harder for external people to subscribe will
negatively impact intended use.  I am going to advise my list admins to
enable confirmation, which should discourage these attempts.  It also
occurred to me that I could write a script to monitor the vette log and
purge requests that look suspicious - mainly based on the same email
address attempting to subscribe to multiple unrelated lists at the same
time.

At KDE we took the semi drastic measure of allowing the commencement of
mailing list subscription by email only as the attackers use HTTP POST to
perform their attacks.

If Mailman were to implement basic CSRF protection for all POST requests
that would also slow the attackers down I suspect (as they would have to
make a GET request first and parse it).

One thing I do know is that at least for us the attacks all appeared to be
coming from Tor endpoints or open web proxies.

Regards,
Ben

[Quoted footers removed by moderator]

From mark at msapiro.net  Mon Nov 19 02:09:08 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 18 Nov 2012 17:09:08 -0800
Subject: [Mailman-Users] Automated Subscription Bots Inundating
	ListOwners With Subscription Requests
In-Reply-To: <CA+XidOHuuXYUiriiYoeUjPpq2EBzg7TXijF1fDr3_gC+QzHm0g@mail.gmail.com>
Message-ID: <PC1952012111817090808902f516942@MSAPIRO>

Ben Cooksley wrote:
>
>If Mailman were to implement basic CSRF protection for all POST requests
>that would also slow the attackers down I suspect (as they would have to
>make a GET request first and parse it).


I have implemented a simple version of what I think you requested in
your post at
<http://mail.python.org/pipermail/mailman-users/2012-October/074287.html>.

It is implemented by the attached patch against Mailman 2.1.15. The
patch will apply to versions 2.1.12 and later with at most line number
changes, For older versions, the hashing function
Mailman.Utils.sha_new doesn't exist and will need to be changed in the
patch to something else. Note that the patch only enables configuring
the token for the listinfo subscribe form. To actually enable
placement and checking of the token, one must assign a non-empty
string value to SUBSCRIBE_FORM_SECRET in mm_cfg.py. I.e.,

SUBSCRIBE_FORM_SECRET = 'Some site specific string'

The actual token is a hex digest of a sha hash of this string plus the
list's internal name plus the IP address of the caller.

A more secure token would include something more random such as the
time of day, but would be a bit more cumbersome to implement -
volunteers are welcome.

Let us know if this helps.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: patch.txt
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20121118/01da155a/attachment.txt>

From stephen at xemacs.org  Mon Nov 19 05:17:44 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 19 Nov 2012 13:17:44 +0900
Subject: [Mailman-Users] Automated Subscription Bots Inundating List
 Owners With Subscription Requests
In-Reply-To: <CA+XidOHuuXYUiriiYoeUjPpq2EBzg7TXijF1fDr3_gC+QzHm0g@mail.gmail.com>
References: <FFD6705BDF9330469C66AC6C3FAAC1290E8661EC@EX1.oregonstate.edu>
	<CA+XidOHuuXYUiriiYoeUjPpq2EBzg7TXijF1fDr3_gC+QzHm0g@mail.gmail.com>
Message-ID: <87d2za6xav.fsf@uwakimon.sk.tsukuba.ac.jp>

Ben Cooksley writes:

 > If Mailman were to implement basic CSRF protection for all POST requests
 > that would also slow the attackers down I suspect (as they would have to
 > make a GET request first and parse it).

It might slow a human down, but as soon as it becomes a feature of
Mailman, the attackers will implement the necessary countermeasure (if
it isn't already implemented because they use libcurl or so in their
program!)  Bandwidth?  CPU?  These guys have no such constraints.

 > One thing I do know is that at least for us the attacks all
 > appeared to be coming from Tor endpoints or open web proxies.

Big surprise.  Not to mention demonstrating that CSRF protection won't
help, because you're dealing with real players, not junior high school
students from a fishing village in western Japan.

The problem here is that you cannot authenticate users you don't
already know.  So CSRF just adds a "get a free token" step to the
automated process.  I'm sure all the major libraries already implement
this, so unless the attackers are remarkably stupid, undoubtedly the
needed code is immediately to hand.

One partial solution would be to allow OpenID logins to the website to
use it to register subscriptions.  Of course you probably can't trust
Google or Yahoo accounts. ;-)



From hawat.thufir at gmail.com  Mon Nov 19 07:41:32 2012
From: hawat.thufir at gmail.com (Thufir)
Date: Sun, 18 Nov 2012 22:41:32 -0800
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
References: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
Message-ID: <50A9D49C.6010100@gmail.com>

On Fri, 16 Nov 2012 11:56:50 -0600, Lindsay Haisley wrote:

> On Thu, 2012-11-15 at 19:38 -0800, Thufir wrote:
>> How does a virtual domain differ from a domain?
>
> To add to what Mark said, technically a "virtual domain" is a domain
> name that resolves in the domain name system to the same IP address as
> the primary ("real") domain name for your host.  A server host can have
> many domain names, all pointing to the same IP address, and this is
> common for web and mail servers these days.


It's not totally clear.  I suppose it's a somewhat unusual situation 
because I'm only using my computer and not an intranet or anything along 
those lines.  This configuration is down to postfix?

I don't think I'm using a virtual domain but only a local domain.  (And 
only a local domain, nothing should go out to the internet.)

I'm going with using my vanity domain:

root at dur:~#
root at dur:~# newlist mailman
Enter the email of the person running the list: thufir at dur.bounceme.net
Initial mailman password:
To finish creating your mailing list, you must edit your /etc/aliases (or
equivalent) file by adding the following lines, and possibly running the
`newaliases' program:

...

dur.bounceme.net is a local domain.  It doesn't exist in any DNS I'm
aware of.  It's just a FQDN which I got from no-ip.com as a free "host". 
  It's unique, but I don't own the bounceme.net domain as a whole, only 
have access to that sub-domain.

I have postfix configured, hopefully, to only send mail to this machine
and not to try to send anything to the internet.

I'm not on the wrong track with mailman?  Assuming Postfix and my hosts 
and so forth are configured correctly, of course...


thanks,

Thufir

From hawat.thufir at gmail.com  Mon Nov 19 11:27:05 2012
From: hawat.thufir at gmail.com (Thufir)
Date: Mon, 19 Nov 2012 02:27:05 -0800
Subject: [Mailman-Users] monthly archive
Message-ID: <50AA0979.8090004@gmail.com>

When I visit the archive for my first list, alpha, it says:


  No messages have been posted to this list yet, so the archives are 
currently empty. You can get more information about this list.


However, I subscribed thufir at dur.bounceme.com, receive the e-mail 
locally, clicked the link, sent a first post, received a copy to 
thufir at dur.bounceme.net.  The list itself seems to work fine.

I've reviewed the list settings through the web UI and it says to 
archive monthly.  Do I need to wait a month before that first post shows 
in the archive?

Also, where are the settings for a list stored?  I browse to something like:

http://dur.bounceme.net/cgi-bin/mailman/admin/alpha/?VARHELP=archive/archive_volume_frequency

but I'm not sure where the config file for that setting for this mailing 
list actually resides.

Do I need to "run" mailman to create the archive?  Can I directly access 
the archive files and list config files without the web UI?



thanks,

Thufir

From fmouse-mailman at fmp.com  Mon Nov 19 15:40:52 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Mon, 19 Nov 2012 08:40:52 -0600
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <50A9D49C.6010100@gmail.com>
References: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
	<50A9D49C.6010100@gmail.com>
Message-ID: <1353336052.46493.17.camel@pudina.fmp.com>

On Sun, 2012-11-18 at 22:41 -0800, Thufir wrote:
> On Fri, 16 Nov 2012 11:56:50 -0600, Lindsay Haisley wrote:
> 
> > On Thu, 2012-11-15 at 19:38 -0800, Thufir wrote:
> >> How does a virtual domain differ from a domain?
> >
> > To add to what Mark said, technically a "virtual domain" is a domain
> > name that resolves in the domain name system to the same IP address as
> > the primary ("real") domain name for your host.  A server host can have
> > many domain names, all pointing to the same IP address, and this is
> > common for web and mail servers these days.
>
> It's not totally clear.  I suppose it's a somewhat unusual situation 
> because I'm only using my computer and not an intranet or anything along 
> those lines.  This configuration is down to postfix?

It's not unusual at all.  From the point of view of DNS, there's no
difference between a virtual domain and a real one.  They're just
different names which resolve to the same IP address.  My server has
dozens of them.

> I don't think I'm using a virtual domain but only a local domain.  (And 
> only a local domain, nothing should go out to the internet.)
> 
> I'm going with using my vanity domain:

If you're using a name which is resolved by a local name server, or from
a hosts file, you can use any names you want, and of course they don't
have to be registered.  They don't even have to comply with standard
naming conventions, although using names such as "my.mailserver.local"
may confuse some software.  I'm not sure what's meant by the term
"vanity domain" but I'm sure it'll do just fine, as long as it resolves
to a proper local IP address and if necessary there's a MX (mail
exchange) record associated with it.

-- 
Lindsay Haisley       | "Never expect the people who caused a problem
FMP Computer Services |  to solve it." - Albert Einstein
512-259-1190          |        
http://www.fmp.com    |


From stephen at xemacs.org  Mon Nov 19 16:53:52 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 20 Nov 2012 00:53:52 +0900
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <1353336052.46493.17.camel@pudina.fmp.com>
References: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
	<50A9D49C.6010100@gmail.com>
	<1353336052.46493.17.camel@pudina.fmp.com>
Message-ID: <874nkl7fn3.fsf@uwakimon.sk.tsukuba.ac.jp>

Lindsay Haisley writes:

 > It's not unusual at all.  From the point of view of DNS, there's no
 > difference between a virtual domain and a real one.

Actually, that's not true.  In the context of Mailman, the most
important one is that an MX record must point to a real domain (ie,
one with an A record) and you probably even want that A record to be
invertible (ie, the PTR for that IP address points back to the same
domain).  A virtual domain also is not 100% reliable for SSL/TLS
services because basic TLS does its certificate exchange at a level
"below" the DNS, so deciding which virtual domain's certificate to
present is problematic (there is an extension to the protocol which
fixes this, but it's not 100% implemented, in particular IE on XP
still can't do it according to Wikipedia, which will kill you in Japan
where about 1/3 of business systems are still XP-based).

This isn't particularly relevant to people who are just plain users of
the system, and I imagine to you it's all second-nature now, but the
OP sounds like he's a bit into do-it-yourself so he should be aware of
the limitations on doing tricky stuff based on a virtual domain.

Steve

From fmouse-mailman at fmp.com  Mon Nov 19 17:11:37 2012
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Mon, 19 Nov 2012 10:11:37 -0600
Subject: [Mailman-Users] what is a virtual domain?
In-Reply-To: <874nkl7fn3.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAOwGw7Nf+oEXCH3w7X=e_0J_6h=Y6oV9X+-J=ho252ui-M_y_g@mail.gmail.com>
	<50A9D49C.6010100@gmail.com> <1353336052.46493.17.camel@pudina.fmp.com>
	<874nkl7fn3.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1353341497.46493.41.camel@pudina.fmp.com>

On Tue, 2012-11-20 at 00:53 +0900, Stephen J. Turnbull wrote:
> Lindsay Haisley writes:
> 
>  > It's not unusual at all.  From the point of view of DNS, there's no
>  > difference between a virtual domain and a real one.
> 
> Actually, that's not true.

I re-read Thufir's question and realized that I misunderstood it.  Yes,
what he's trying to do is decidedly unusual.

> A virtual domain also is not 100% reliable for SSL/TLS
> services because basic TLS does its certificate exchange at a level
> "below" the DNS, so deciding which virtual domain's certificate to
> present is problematic (there is an extension to the protocol which
> fixes this, but it's not 100% implemented, in particular IE on XP
> still can't do it according to Wikipedia, which will kill you in Japan
> where about 1/3 of business systems are still XP-based).

Being a natural-born cheapskate, and running a _very_ small business, I
don't even have a wildcard SSL cert signing for FMP's SSL web presence.
Certificates for email SSL/TLS are self-signed by scripts which came
with the mail server (Courier-MTA).  Customers who want SSL pages get a
URL under secure.fmp.com with a directory/symlink to their home
directory, and a PHP snippet in the page to deflect non-SSL accesses to
the secure URL.

> This isn't particularly relevant to people who are just plain users of
> the system, and I imagine to you it's all second-nature now, but the
> OP sounds like he's a bit into do-it-yourself so he should be aware of
> the limitations on doing tricky stuff based on a virtual domain.

I've always been a bit non-conformist in my system administration
practices, which hasn't always made things easy, but I've learned a lot.
I've never tried anything such as it seems that Thufir is working with,
though.

-- 
Lindsay Haisley       | "Fighting against human creativity is like 
FMP Computer Services |   trying to eradicate dandelions"
512-259-1190          |
http://www.fmp.com    |       -- Pamela Jones


From hawat.thufir at gmail.com  Mon Nov 19 18:44:59 2012
From: hawat.thufir at gmail.com (thufir)
Date: Mon, 19 Nov 2012 09:44:59 -0800
Subject: [Mailman-Users] NNTP Mass catch up
Message-ID: <50AA701B.3090609@localhost>

In the Mail<->News gateways Section for my "alpha" list I clicked the 
box for a mass catchup.  The news server is the same machine, so I 
entered localhost.  (I'm running leafnode.)

I do see NNTP in the cron script for mailman, so I uncommented that:

root at dur:~#
root at dur:~# cat /etc/cron.d/mailman | grep gate_news
  */5 * * * * list [ -x /usr/lib/mailman/cron/gate_news ] && 
/usr/lib/mailman/cron/gate_news
root at dur:~#


How do I know the results of the mass catch up?  It seems to have worked:

root at dur:~#
root at dur:~# nl /var/log/mailman/fromusenet
      1	Nov 19 09:40:02 2012 (3035) alpha: [2..8718]
      2	Nov 19 09:40:02 2012 (3035) alpha caught up to article 8718
      3	Nov 19 09:40:02 2012 (3035) alpha watermark: 8718
root at dur:~#


only I don't see any such articles actually in the list.  Mails to the 
list (it's only me subscribed) seem to work fine.

root at dur:~#
root at dur:~# nl /var/log/mailman/post
      1	Nov 19 01:30:04 2012 (13366) post to alpha from 
alpha-request at dur.bounceme.net, size=1646, 
message-id=<mailman.0.1353317401.13682.alpha at dur.bounceme.net>, 1 failures
      2	Nov 19 02:13:29 2012 (13366) post to alpha from 
thufir at dur.bounceme.net, size=1499, message-id=<1353320001.13556.0 at dur>, 
success
      3	Nov 19 02:24:18 2012 (13366) post to alpha from 
thufir at dur.bounceme.net, size=1850, message-id=<1353320657.13556.2 at dur>, 
success
      4	Nov 19 09:31:54 2012 (1413) post to alpha from 
thufir at dur.bounceme.net, size=1523, message-id=<1353346312.2541.0 at dur>, 
success
root at dur:~#


and come through fine.  I can read and post to the list, but the 
archives are empty and the NNTP messages are in limbo.




thanks,

Thufir

From mark at msapiro.net  Mon Nov 19 19:44:57 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 19 Nov 2012 10:44:57 -0800
Subject: [Mailman-Users] monthly archive
In-Reply-To: <50AA0979.8090004@gmail.com>
Message-ID: <PC195201211191044570000fa330739@MSAPIRO>

Thufir wrote:

>When I visit the archive for my first list, alpha, it says:
>
>
>  No messages have been posted to this list yet, so the archives are 
>currently empty. You can get more information about this list.
>
>
>However, I subscribed thufir at dur.bounceme.com, receive the e-mail 
>locally, clicked the link, sent a first post, received a copy to 
>thufir at dur.bounceme.net.  The list itself seems to work fine.


Then the post should have been archived assuming ArchRunner is running,
no errors are reported in Mailman's logs/error log and the list's
Archiving options -> archive is set to Yes.


>I've reviewed the list settings through the web UI and it says to 
>archive monthly.  Do I need to wait a month before that first post shows 
>in the archive?


No


>Also, where are the settings for a list stored?  I browse to something like:
>
>http://dur.bounceme.net/cgi-bin/mailman/admin/alpha/?VARHELP=archive/archive_volume_frequency
>
>but I'm not sure where the config file for that setting for this mailing 
>list actually resides.


Mailman's lists/LISTNAME/config.pck


>Do I need to "run" mailman to create the archive?


Mailman (ArchRunner) must be running to archive messages, but then if
Mailman isn't running, posts will not be processed and sent to list
members.


>Can I directly access 
>the archive files and list config files without the web UI?



Yes. The archive files are in archives/private/LISTNAME*

Tools for examining/changing the list configuration are in Mailman's
bin/ directory.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Mon Nov 19 20:01:45 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 19 Nov 2012 11:01:45 -0800
Subject: [Mailman-Users] NNTP Mass catch up
In-Reply-To: <50AA701B.3090609@localhost>
Message-ID: <PC195201211191101450531d059e800@MSAPIRO>

thufir wrote:
>
>How do I know the results of the mass catch up?  It seems to have worked:
>
>root at dur:~#
>root at dur:~# nl /var/log/mailman/fromusenet
>      1	Nov 19 09:40:02 2012 (3035) alpha: [2..8718]
>      2	Nov 19 09:40:02 2012 (3035) alpha caught up to article 8718
>      3	Nov 19 09:40:02 2012 (3035) alpha watermark: 8718
>root at dur:~#
>
>
>only I don't see any such articles actually in the list.



You have it backwards. Mass catchup means ignore all messages currently
on the news server and start gatwaying to the list with the next news
message to arrive.

If you wanted to gateway all the existing news server messages to the
list, you should not have done a mass catchup and if everything was
configured OK, the messages would have been gated to the list the
first time cron/gate_news ran.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From hawat.thufir at gmail.com  Mon Nov 19 23:52:15 2012
From: hawat.thufir at gmail.com (Thufir)
Date: Mon, 19 Nov 2012 14:52:15 -0800
Subject: [Mailman-Users] NNTP Mass catch up
In-Reply-To: <50AA701B.3090609@localhost>
References: <50AA701B.3090609@localhost>
Message-ID: <50AAB81F.8000204@gmail.com>

On Mon, 19 Nov 2012 11:01:45 -0800, Mark Sapiro wrote:

[..]
>>only I don't see any such articles actually in the list.
>
>
>
> You have it backwards. Mass catchup means ignore all messages currently
> on the news server and start gatwaying to the list with the next news
> message to arrive.
>
> If you wanted to gateway all the existing news server messages to the
> list, you should not have done a mass catchup and if everything was
> configured OK, the messages would have been gated to the list the first
> time cron/gate_news ran.

Ok, the messages are there now.  I don't know why it was such a struggle 
to setup for me, in retrospect that's not too bad.

thanks for the help :)


-Thufir

From mangoo at wpkg.org  Wed Nov 21 23:01:58 2012
From: mangoo at wpkg.org (Tomasz Chmielewski)
Date: Thu, 22 Nov 2012 00:01:58 +0200
Subject: [Mailman-Users] sitemap.xml for Mailman
Message-ID: <50AD4F56.9090001@wpkg.org>

Hi,

I wrote a simple bash script which generates a sitemap.xml file (i.e. to 
be submitted to Google):

http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/


My rationale was: why the heck is my archive lacking so many posts in 
google, and is generally "behind" all other archives.


-- 
Tomasz Chmielewski

From mark at msapiro.net  Thu Nov 22 19:36:47 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 22 Nov 2012 10:36:47 -0800
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <50AD4F56.9090001@wpkg.org>
Message-ID: <PC1952012112210364709373f530dd8@MSAPIRO>

Tomasz Chmielewski wrote:
>
>I wrote a simple bash script which generates a sitemap.xml file (i.e. to 
>be submitted to Google):
>
>http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/



There is an issue with the above script. Namely, the XML generated only
contains data for the last list in $LISTS. This will fix that

--- sitemap.old 2012-11-22 10:02:51.000000000 -0800
+++ sitemap.new 2012-11-22 10:27:37.000000000 -0800
@@ -18,8 +18,10 @@
 set -u

 # find html files with their dates
+URLS=""
 for LIST in $LISTS; do
-    URLS=$(find $MAILMANPATH/$LIST/ -type f -name \*html | xargs ls
--time-style=long-iso -l | awk '{print $6"T"$7":00+00:00 "$8}' | grep
-v attachments)
+    URLS="$URLS
+$(find $MAILMANPATH/$LIST/ -type f -name \*html | xargs ls
--time-style=long-iso -l | awk '{print $6"T"$7":00+00:00 "$8}' | grep
-v attachments)"
 done

 # if the article is crawled once a month, it should be enough

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mangoo at wpkg.org  Thu Nov 22 20:37:29 2012
From: mangoo at wpkg.org (Tomasz Chmielewski)
Date: Thu, 22 Nov 2012 21:37:29 +0200
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <PC1952012112210364709373f530dd8@MSAPIRO>
References: <PC1952012112210364709373f530dd8@MSAPIRO>
Message-ID: <50AE7EF9.1030109@wpkg.org>

On 11/22/2012 08:36 PM, Mark Sapiro wrote:
> Tomasz Chmielewski wrote:
>>
>> I wrote a simple bash script which generates a sitemap.xml file (i.e. to
>> be submitted to Google):
>>
>> http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/
>
>
>
> There is an issue with the above script. Namely, the XML generated only
> contains data for the last list in $LISTS. This will fix that

> +    URLS="$URLS
> +$(find $MAILMANPATH/$LIST/ -type f -name \*html | xargs ls
> --time-style=long-iso -l | awk '{print $6"T"$7":00+00:00 "$8}' | grep
> -v attachments)"

Indeed - thanks for pointing out!


-- 
Tomasz Chmielewski
http://blog.wpkg.org


From Ralf.Hildebrandt at charite.de  Thu Nov 22 21:16:13 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Thu, 22 Nov 2012 21:16:13 +0100
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <50AE7EF9.1030109@wpkg.org>
References: <PC1952012112210364709373f530dd8@MSAPIRO>
	<50AE7EF9.1030109@wpkg.org>
Message-ID: <20121122201613.GB5330@charite.de>

* Tomasz Chmielewski <mangoo at wpkg.org>:
> On 11/22/2012 08:36 PM, Mark Sapiro wrote:
> >Tomasz Chmielewski wrote:
> >>
> >>I wrote a simple bash script which generates a sitemap.xml file (i.e. to
> >>be submitted to Google):
> >>
> >>http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/
> >
> >
> >
> >There is an issue with the above script. Namely, the XML generated only
> >contains data for the last list in $LISTS. This will fix that
> 
> >+    URLS="$URLS
> >+$(find $MAILMANPATH/$LIST/ -type f -name \*html | xargs ls
> >--time-style=long-iso -l | awk '{print $6"T"$7":00+00:00 "$8}' | grep
> >-v attachments)"
> 
> Indeed - thanks for pointing out!

Did you update your patch already? If so, I'd apply it to our very own
mailman installation here at python.org :)

-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From mangoo at wpkg.org  Thu Nov 22 21:21:19 2012
From: mangoo at wpkg.org (Tomasz Chmielewski)
Date: Thu, 22 Nov 2012 22:21:19 +0200
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <20121122201613.GB5330@charite.de>
References: <PC1952012112210364709373f530dd8@MSAPIRO>
	<50AE7EF9.1030109@wpkg.org> <20121122201613.GB5330@charite.de>
Message-ID: <50AE893F.2030306@wpkg.org>

On 11/22/2012 10:16 PM, Ralf Hildebrandt wrote:

>> Indeed - thanks for pointing out!
>
> Did you update your patch already? If so, I'd apply it to our very own
> mailman installation here at python.org :)

Yes I did (with a small change).


-- 
Tomasz Chmielewski
http://blog.wpkg.org



From mark at msapiro.net  Sun Nov 25 00:11:19 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 24 Nov 2012 15:11:19 -0800
Subject: [Mailman-Users] Automated Subscription Bots Inundating
 ListOwners With Subscription Requests
In-Reply-To: <PC1952012111817090808902f516942@MSAPIRO>
References: <PC1952012111817090808902f516942@MSAPIRO>
Message-ID: <50B15417.2080607@msapiro.net>

Mark Sapiro wrote at
<http://mail.python.org/pipermail/mailman-users/2012-November/074415.html>:

> I have implemented a simple version of what I think you requested in
> your post at
> <http://mail.python.org/pipermail/mailman-users/2012-October/074287.html>.
> 
> It is implemented by the attached patch against Mailman 2.1.15.


I have augmented that patch with a timestamp and it now also checks that
the hash is no older than mm_cfg.FORM_LIFETIME. See
<https://launchpad.net/bugs/1082746> and
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371>
for a bug report and the patch which will be released with Mailman 2.1.16.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Sun Nov 25 00:25:59 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 24 Nov 2012 15:25:59 -0800
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <50AD4F56.9090001@wpkg.org>
References: <50AD4F56.9090001@wpkg.org>
Message-ID: <50B15787.3030104@msapiro.net>

Tomasz Chmielewski wrote:

> I wrote a simple bash script which generates a sitemap.xml file (i.e. to 
> be submitted to Google):
> 
> http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/

I was inspired by your script to add a -p / --public-archive option to
Mailman's bin/list_lists so that if one wanted to process just those
lists with public archives, one can put

LISTS=`/path/to/mailman/bin/list_lists --bare --public-archive`

in the appropriate place to do that. Of course, one could have done

LISTS=`ls $MAILMANPATH/../public|grep -v "\\.mbox$"`

to accomplish essentially the same thing, but since list_lists already
had a -a / --advertised option, this seemed a good addition.

See <https://launchpad.net/bugs/1082711> and
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1369>
for a bug report and patch.

Also, I think the bash script would make a good addition to the contrib
directory for Mailman 2.1.16. May I add it there?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mangoo at wpkg.org  Sun Nov 25 01:16:12 2012
From: mangoo at wpkg.org (Tomasz Chmielewski)
Date: Sun, 25 Nov 2012 02:16:12 +0200
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <50B15787.3030104@msapiro.net>
References: <50AD4F56.9090001@wpkg.org> <50B15787.3030104@msapiro.net>
Message-ID: <50B1634C.5090200@wpkg.org>

On 11/25/2012 01:25 AM, Mark Sapiro wrote:
> Tomasz Chmielewski wrote:
>
>> I wrote a simple bash script which generates a sitemap.xml file (i.e. to
>> be submitted to Google):
>>
>> http://blog.wpkg.org/2012/11/21/sitemap-xml-for-mailman/
>
> I was inspired by your script to add a -p / --public-archive option to
> Mailman's bin/list_lists so that if one wanted to process just those
> lists with public archives, one can put
>
> LISTS=`/path/to/mailman/bin/list_lists --bare --public-archive`
>
> in the appropriate place to do that. Of course, one could have done
>
> LISTS=`ls $MAILMANPATH/../public|grep -v "\\.mbox$"`
>
> to accomplish essentially the same thing, but since list_lists already
> had a -a / --advertised option, this seemed a good addition.
>
> See <https://launchpad.net/bugs/1082711> and
> <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1369>
> for a bug report and patch.
>
> Also, I think the bash script would make a good addition to the contrib
> directory for Mailman 2.1.16. May I add it there?

Sure, it would be great.


-- 
Tomasz Chmielewski
http://blog.wpkg.org



From Ralf.Hildebrandt at charite.de  Sun Nov 25 10:14:22 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Sun, 25 Nov 2012 10:14:22 +0100
Subject: [Mailman-Users] Automated Subscription Bots Inundating
 ListOwners With Subscription Requests
In-Reply-To: <50B15417.2080607@msapiro.net>
References: <PC1952012111817090808902f516942@MSAPIRO>
	<50B15417.2080607@msapiro.net>
Message-ID: <20121125091422.GD25058@charite.de>

* Mark Sapiro <mark at msapiro.net>:

> I have augmented that patch with a timestamp and it now also checks that
> the hash is no older than mm_cfg.FORM_LIFETIME. See
> <https://launchpad.net/bugs/1082746> and
> <http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1371>
> for a bug report and the patch which will be released with Mailman 2.1.16.

Excellent!!!
-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From Ralf.Hildebrandt at charite.de  Sun Nov 25 10:15:40 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Sun, 25 Nov 2012 10:15:40 +0100
Subject: [Mailman-Users] sitemap.xml for Mailman
In-Reply-To: <50B15787.3030104@msapiro.net>
References: <50AD4F56.9090001@wpkg.org>
 <50B15787.3030104@msapiro.net>
Message-ID: <20121125091540.GE25058@charite.de>

> Also, I think the bash script would make a good addition to the contrib
> directory for Mailman 2.1.16. May I add it there?

Personally, I'd say yes. It adds and doesn't change any
existing functionality for the benefit of all. So why not add it?
-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From roger.richcorp at btconnect.com  Mon Nov 26 11:37:34 2012
From: roger.richcorp at btconnect.com (Roger Richmond)
Date: Mon, 26 Nov 2012 10:37:34 +0000
Subject: [Mailman-Users] Content Filtering
Message-ID: <005401cdcbc2$0de9ea20$29bdbe60$@richcorp@btconnect.com>

I have had a problem with Mailman  stripping out MS Word documents, and have
partially solved it by adding "application/msword" to the pass mime types
section.  However, Mailman still strips out documents created in Office 2007
and later, with the .docx extension.

 

Any ideas how I can allow these please?

 

My pass mime types are:

multipart

text

text/html

text/plain

image

application/pdf

application/msword

 

My filter filename extensions are :

exe

bat

cmd

com

pif

scr

vbs

cpl

 

Collapse alternatives and convert html to plain text are both set to yes.

 

Many thanks,

 

Roger Richmond


From stephen at xemacs.org  Mon Nov 26 14:40:56 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 26 Nov 2012 22:40:56 +0900
Subject: [Mailman-Users]  Content Filtering
In-Reply-To: <005401cdcbc2$0de9ea20$29bdbe60$@richcorp@btconnect.com>
References: <005401cdcbc2$0de9ea20$29bdbe60$@richcorp@btconnect.com>
Message-ID: <87boek5vo7.fsf@uwakimon.sk.tsukuba.ac.jp>

Roger Richmond writes:

 > I have had a problem with Mailman  stripping out MS Word documents, and have
 > partially solved it by adding "application/msword" to the pass mime types
 > section.  However, Mailman still strips out documents created in Office 2007
 > and later, with the .docx extension.

The general approach is to look at the original mail containing the
documents in a text editor (or Gmail's "Show Original") and search for
the Content-Type headers.  They'll look something like this:

Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document; name="some junk from the boss"

Most likely that's the one you want most.  Add

application/vnd.openxmlformats-officedocument.wordprocessingml.document

to Pass Mime Types and you'll probably get most of them.  However, not
all mail programs conform to the registered types.  There may be
others you want to catch, but you'll have to investigate them
yourself I suspect.

From karsten.becker at ecologic.eu  Tue Nov 27 13:53:13 2012
From: karsten.becker at ecologic.eu (Karsten Becker)
Date: Tue, 27 Nov 2012 13:53:13 +0100
Subject: [Mailman-Users] Mailman needs 20 seconds to complete per email
Message-ID: <50B4B7B9.7050303@ecologic.eu>

Hi there,

I'm running an Ubuntu Server 10.04 LTS (Lucid Lynx) with the repository
mailman in version
> ii  mailman                         1:2.1.13-1ubuntu0.2
Powerful, web-based mailing list manager

Everything is running fine, so emails arriving at postfix get transfered
nearby immediate towards mailman and vice-versa. I'm running Mailman in
a multi-domain-MySQL-based setup, as we have to run a lot of different
domains in our company. But as said above, this all works.

The problem is, that if Mailman gets an email from Postfix, it sits down
and does "something" for roundabout 20 seconds. Then it sends out the
emails towards Postfix, and Postfix delivers.

For normal mailing list operation this is not a problem as it sends out
one bulk email out to Postfix - but when I do a mass subscription of for
example 100 people, Mailman will start to send out the personalized
welcome emails one after another - and will also need 20 seconds per
email. So it takes 100x 20 seconds = 2000 seconds = 33 minutes till
everybody gets his/her welcome email.

Is this usual behaviour of Mailman to do "something" that long? The
server is a powerful one with plenty of GHz and RAM. What the hell could
this "something" be?  :-)

Following you will find my configuration file and the smtp log of mailman:

/etc/mailman/mm_cfg.py:

> from Defaults import *
> MAILMAN_SITE_LIST = 'mailman'
> DEFAULT_URL_PATTERN = 'http://%s/mailman/'
> PRIVATE_ARCHIVE_URL = '/mailman/private'
> IMAGE_LOGOS         = '/mailman/images/mailman/'
> SHORTCUT_ICON       = 'favicon.ico'
> DEFAULT_EMAIL_HOST = 'lists.mycompany.eu'
> DEFAULT_URL_HOST   = 'lists.mycompany.eu'
> add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> DEFAULT_SERVER_LANGUAGE = 'en'
> USE_ENVELOPE_SENDER    = 0              # Still used?
> DEFAULT_SEND_REMINDERS = 0
> MTA='Postfix'
> SMTPHOST = 'localhost'
> SMTPPORT = 10030
> POSTFIX_STYLE_VIRTUAL_DOMAINS = ['lists.mycompany.eu', 'lists.myothercompany.org', 'lists.mythirdcompany.eu']
> add_virtualhost('lists.myothercompany.org', 'lists.myothercompany.org')
> add_virtualhost('lists.mythirdcompany.eu', 'lists.mythirdcompany.eu')
> DEB_LISTMASTER = 'postmaster at mycompany.eu'
> DEFAULT_ARCHIVE_PRIVATE = 1
> MEMBER_PASSWORD_LENGTH = 15
> ADMIN_PASSWORD_LENGTH = 20
> DEFAULT_LIST_ADVERTISED = No
> DEFAULT_MAX_NUM_RECIPIENTS = 10
> DEFAULT_MAX_MESSAGE_SIZE = 250           # KB
> DEFAULT_MAX_DAYS_TO_HOLD = 7
> DEFAULT_GENERIC_NONMEMBER_ACTION = 2
> DEFAULT_SUBSCRIBE_POLICY = 3

/var/log/mailman/smtp

> Nov 27 08:00:22 2012 (10393) <mailman.0.1353999601.22399.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.021 seconds
> Nov 27 08:00:43 2012 (10393) <mailman.1.1353999601.22399.agriculture at lists.mycompany.eu> smtp to agriculture for 1 recips, completed in 21.013 seconds
> Nov 27 08:01:04 2012 (10393) <mailman.2.1353999601.22399.wimix at lists.mycompany.eu> smtp to wimix for 1 recips, completed in 21.007 seconds
> Nov 27 08:01:25 2012 (10393) <mailman.3.1353999601.22399.out-of-pfalzburger at lists.mycompany.eu> smtp to out-of-pfalzburger for 1 recips, completed in 21.006 seconds
> Nov 27 08:01:46 2012 (10393) <mailman.4.1353999601.22399.confix at lists.mycompany.eu> smtp to confix for 1 recips, completed in 21.018 seconds
> Nov 27 08:02:07 2012 (10393) <mailman.5.1353999601.22399.energy at lists.mycompany.eu> smtp to energy for 1 recips, completed in 21.006 seconds
> Nov 27 08:02:28 2012 (10393) <mailman.6.1353999601.22399.china at lists.mycompany.eu> smtp to china for 1 recips, completed in 21.008 seconds
> Nov 27 08:02:49 2012 (10393) <mailman.0.1353999601.22399.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 2 recips, completed in 21.007 seconds
> Nov 27 08:03:10 2012 (10393) <mailman.1.1353999601.22399.agriculture at lists.mycompany.eu> smtp to agriculture for 3 recips, completed in 21.005 seconds
> Nov 27 08:03:31 2012 (10393) <mailman.2.1353999601.22399.wimix at lists.mycompany.eu> smtp to wimix for 2 recips, completed in 21.007 seconds
> Nov 27 08:04:13 2012 (10393) <mailman.4.1353999601.22399.confix at lists.mycompany.eu> smtp to confix for 2 recips, completed in 21.011 seconds
> Nov 27 08:04:34 2012 (10393) <mailman.5.1353999601.22399.energy at lists.mycompany.eu> smtp to energy for 2 recips, completed in 21.007 seconds
> Nov 27 08:04:55 2012 (10393) <mailman.6.1353999601.22399.china at lists.mycompany.eu> smtp to china for 2 recips, completed in 21.006 seconds
> Nov 27 09:01:20 2012 (10393) <50B47339.6040405 at mycompany.eu> smtp to energy for 61 recips, completed in 21.124 seconds
> Nov 27 09:18:43 2012 (10393) <50B4774C.1030707 at mycompany.eu> smtp to berlinix for 101 recips, completed in 21.126 seconds
> Nov 27 09:51:29 2012 (10393) <mailman.0.1354006267.31674.china at lists.mycompany.eu> smtp to china for 1 recips, completed in 21.022 seconds
> Nov 27 10:27:04 2012 (10393) <mailman.0.1354008402.6430.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.029 seconds
> Nov 27 10:27:25 2012 (10393) <mailman.1.1354008402.6430.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.004 seconds
> Nov 27 10:33:09 2012 (10393) <mailman.0.1354008767.9391.energy at lists.mycompany.eu> smtp to energy for 1 recips, completed in 21.022 seconds
> Nov 27 11:45:19 2012 (10393) <mailman.0.1354013097.25318.confix at lists.mycompany.eu> smtp to confix for 1 recips, completed in 21.023 seconds
> Nov 27 11:49:32 2012 (10393) <mailman.14.1354013350.10391.confix at lists.mycompany.eu> smtp to confix for 1 recips, completed in 21.018 seconds
> Nov 27 11:49:54 2012 (10393) <mailman.16.1354013350.10391.confix at lists.mycompany.eu> smtp to confix for 1 recips, completed in 21.815 seconds
> Nov 27 11:50:18 2012 (10393) <mailman.16.1354013350.10391.confix at lists.mycompany.eu> smtp to confix for 2 recips, completed in 21.391 seconds
> Nov 27 12:20:17 2012 (10393) <mailman.0.1354015196.1021.confix at lists.mycompany.eu> smtp to confix for 1 recips, completed in 21.015 seconds
> Nov 27 12:21:05 2012 (10393) <mailman.0.1354015243.1152.wimix at lists.mycompany.eu> smtp to wimix for 1 recips, completed in 21.001 seconds
> Nov 27 12:40:52 2012 (10393) <50B4A6AD.40705 at mycompany.eu> smtp to ecologix for 123 recips, completed in 21.235 seconds


Regards
Karsten

From mark at msapiro.net  Tue Nov 27 22:42:01 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 27 Nov 2012 13:42:01 -0800
Subject: [Mailman-Users] Mailman needs 20 seconds to complete per email
In-Reply-To: <50B4B7B9.7050303@ecologic.eu>
Message-ID: <PC19932012112713420101872d0be6bc@MSAPIRO>

Karsten Becker wrote:
>
>I'm running an Ubuntu Server 10.04 LTS (Lucid Lynx) with the repository
>mailman in version
>> ii  mailman                         1:2.1.13-1ubuntu0.2
>Powerful, web-based mailing list manager


Please see the FAQ at <http://wiki.list.org/x/OIDD>.


[...]
>/var/log/mailman/smtp
>
>> Nov 27 08:00:22 2012 (10393) <mailman.0.1353999601.22399.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.021 seconds
>> Nov 27 08:00:43 2012 (10393) <mailman.1.1353999601.22399.agriculture at lists.mycompany.eu> smtp to agriculture for 1 recips, completed in 21.013 seconds
[...]


It is extreme and highly unusual for a single Mailman-Postfix SMTP
transaction to take 20 seconds, but it is almost certainly due to
something in your Postfix configuration.

See the FAQs at <http://wiki.list.org/x/roA9> and
<http://wiki.list.org/x/rIA9> for some hints that may help.

Also see the FAQ at <http://wiki.list.org/x/-IA9> for a way to enable
logging from Python's smtplib which may help you determine in exactly
what part of the SMTP transaction the delay occurs in the response
from Postfix.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From Ralf.Hildebrandt at charite.de  Tue Nov 27 22:56:42 2012
From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt)
Date: Tue, 27 Nov 2012 22:56:42 +0100
Subject: [Mailman-Users] Mailman needs 20 seconds to complete per email
In-Reply-To: <PC19932012112713420101872d0be6bc@MSAPIRO>
References: <50B4B7B9.7050303@ecologic.eu>
	<PC19932012112713420101872d0be6bc@MSAPIRO>
Message-ID: <20121127215641.GC7365@charite.de>

* Mark Sapiro <mark at msapiro.net>:

> >> Nov 27 08:00:22 2012 (10393) <mailman.0.1353999601.22399.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.021 seconds
> >> Nov 27 08:00:43 2012 (10393) <mailman.1.1353999601.22399.agriculture at lists.mycompany.eu> smtp to agriculture for 1 recips, completed in 21.013 seconds
> [...]
> 
> 
> It is extreme and highly unusual for a single Mailman-Postfix SMTP
> transaction to take 20 seconds, but it is almost certainly due to
> something in your Postfix configuration.

It's probably some DNS resolution issue (I'd say). Or a really slow
smtpd_proxy_filter.

-- 
Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
ralf.hildebrandt at charite.de        Campus Benjamin Franklin
http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155

From bcooksley at kde.org  Tue Nov 27 23:02:10 2012
From: bcooksley at kde.org (Ben Cooksley)
Date: Wed, 28 Nov 2012 11:02:10 +1300
Subject: [Mailman-Users] Mailman needs 20 seconds to complete per email
In-Reply-To: <20121127215641.GC7365@charite.de>
References: <50B4B7B9.7050303@ecologic.eu>
	<PC19932012112713420101872d0be6bc@MSAPIRO>
	<20121127215641.GC7365@charite.de>
Message-ID: <CA+XidOFtkSjCgw_M5eApivmxXcU6x3mPOV5-CiVPUQkFQ0ywiw@mail.gmail.com>

On Wed, Nov 28, 2012 at 10:56 AM, Ralf Hildebrandt
<Ralf.Hildebrandt at charite.de> wrote:
> * Mark Sapiro <mark at msapiro.net>:
>
>> >> Nov 27 08:00:22 2012 (10393) <mailman.0.1353999601.22399.carbonmarkets at lists.mycompany.eu> smtp to carbonmarkets for 1 recips, completed in 21.021 seconds
>> >> Nov 27 08:00:43 2012 (10393) <mailman.1.1353999601.22399.agriculture at lists.mycompany.eu> smtp to agriculture for 1 recips, completed in 21.013 seconds
>> [...]
>>
>>
>> It is extreme and highly unusual for a single Mailman-Postfix SMTP
>> transaction to take 20 seconds, but it is almost certainly due to
>> something in your Postfix configuration.
>
> It's probably some DNS resolution issue (I'd say). Or a really slow
> smtpd_proxy_filter.

It could also be a sleep command, in one of the various
smtpd_client_restrictions rules.
Usually one puts permit_mynetworks (which contains
localhost/127.0.0.1) before those though.

>
> --
> Ralf Hildebrandt                   Charite Universit?tsmedizin Berlin
> ralf.hildebrandt at charite.de        Campus Benjamin Franklin
> http://www.charite.de              Hindenburgdamm 30, 12203 Berlin
> Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> http://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: http://mail.python.org/mailman/options/mailman-users/bcooksley%40kde.org

From dries.annaert at gmail.com  Wed Nov 28 10:40:19 2012
From: dries.annaert at gmail.com (dries annaert)
Date: Wed, 28 Nov 2012 10:40:19 +0100
Subject: [Mailman-Users] Reply-to and subscribing
Message-ID: <CAOTiLEGV--i2A2P8Z0JP7MTp_4A3TgN864oi5_nHJ=S3uPFi9w@mail.gmail.com>

I've setup the dreamhost implementation of mailman succesfully but I'm
facing a problem with the subscriptions.

What I want to try is to manage (un-)subscriptions through a drupal website
where (if the name of the list is checked) a mail is being sent to the
list-join address of the list. However, to avoid being marked as spam by
dreamhost, I have to sent the message with the actual subscribing address
in the reply-to field.

However, I can't find if it's possible (and if so, how?) to look into the
reply-to field for (un-)subscriptions.

Anyone has an idea if and how this can be done?

Best regards,
Dries

From dries.annaert at gmail.com  Wed Nov 28 11:26:21 2012
From: dries.annaert at gmail.com (dries annaert)
Date: Wed, 28 Nov 2012 11:26:21 +0100
Subject: [Mailman-Users] Reply-to and subscribing
In-Reply-To: <CAOTiLEGV--i2A2P8Z0JP7MTp_4A3TgN864oi5_nHJ=S3uPFi9w@mail.gmail.com>
References: <CAOTiLEGV--i2A2P8Z0JP7MTp_4A3TgN864oi5_nHJ=S3uPFi9w@mail.gmail.com>
Message-ID: <CAOTiLEGoGsrc8cgyX5aLjWvJCHXe1NdTzek+TVZp3z53opw++A@mail.gmail.com>

Nevermind,

I think I can get out of this problem with the list-request and sending the
address as a command :)

Sorry for the inconvenience.

Best regards,
Dries


2012/11/28 dries annaert <dries.annaert at gmail.com>

> I've setup the dreamhost implementation of mailman succesfully but I'm
> facing a problem with the subscriptions.
>
> What I want to try is to manage (un-)subscriptions through a drupal
> website where (if the name of the list is checked) a mail is being sent to
> the list-join address of the list. However, to avoid being marked as spam
> by dreamhost, I have to sent the message with the actual subscribing
> address in the reply-to field.
>
> However, I can't find if it's possible (and if so, how?) to look into the
> reply-to field for (un-)subscriptions.
>
> Anyone has an idea if and how this can be done?
>
> Best regards,
> Dries
>

From cstalberg at web-analysts.net  Thu Nov 29 01:34:44 2012
From: cstalberg at web-analysts.net (Web Analysts)
Date: Wed, 28 Nov 2012 19:34:44 -0500 (EST)
Subject: [Mailman-Users] error when adding an unsubscribe link to email
	footer
In-Reply-To: <CAOTiLEGV--i2A2P8Z0JP7MTp_4A3TgN864oi5_nHJ=S3uPFi9w@mail.gmail.com>
Message-ID: <1354149284.411@web-analysts.net>

   When I try and add

Unsubscribe: %(user_optionsurl)s?password=%(user_password)s&unsub=1&unsubconfirm
=1
to Footer added to mail sent to regular list member
I get the error

  Warning: The following illegal substitution variables were found in the
  msg_footer string: user_optionsurl, user_password

   Your list may not operate properly until you correct this problem.

From mark at msapiro.net  Thu Nov 29 01:52:08 2012
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 28 Nov 2012 16:52:08 -0800
Subject: [Mailman-Users] error when adding an unsubscribe link to
	emailfooter
In-Reply-To: <1354149284.411@web-analysts.net>
Message-ID: <PC1993201211281652080140f89fe694@MSAPIRO>

Web Analysts wrote:
>
>  Warning: The following illegal substitution variables were found in the
>  msg_footer string: user_optionsurl, user_password


You must enable personalization to use those replacements. See the FAQ
at <http://wiki.list.org/x/UYA9>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan