[Greg Stark <gsstark@mit.edu>] Re: Bounce removal parameters default values
So the problem I described last January and again mentioned last September is still happening to me, and now to a lot more people. It will only become more and more prevalent as viruses become more common and sites that filter them become more common.
Perhaps I should restate the problem more simply. Mailman is committing the basic sin of network security -- receiving data from the network and trusting it for purposes other than as opaque data.
It is using messages posted to the list -- the content and format of which it does not control -- to detect bouncing email addresses. Because of this it cannot tell if the bounces it's receiving are caused by a broken email address or caused by some particularity of the posted message.
Virus scans are only one type of bounce that could cause someone to be unsubscribed spuriously. For example, most mail servers have a maximum message size for example. Consider the security implications: all I have to do to mass unsubscribe many people--even everyone--on a list is send a message over 50k. Everyone using old versions of sendmail will be unsubscribed. A larger message will unsubscribe anyone using most modern MTAs. Nor do the tests that require multiple bounces protect anything; I just have to send my attack a few times quickly.
Really Mailman should simply not trust outside data for any purpose. It should treat the bounces received from mailing list messages purely as hints. It should then send its *own* message with content not subject to any control from outside to the user. Only if that known inoffensive message bounces should it consider removing the user.
This is really a DOS security issue, though the worst case attack is unsubscribing many users of a list. That it gets triggered normally even when not specifically under attack only makes the problem apparent.
-- greg
On Thu, 2004-06-17 at 14:36, Greg Stark wrote:
It is using messages posted to the list -- the content and format of which it does not control -- to detect bouncing email addresses. Because of this it cannot tell if the bounces it's receiving are caused by a broken email address or caused by some particularity of the posted message.
Really Mailman should simply not trust outside data for any purpose. It should treat the bounces received from mailing list messages purely as hints. It should then send its *own* message with content not subject to any control from outside to the user. Only if that known inoffensive message bounces should it consider removing the user.
Upgrade to Mailman 2.1.5, which sends out probe messages after the bounce threshold is reached. Members will only get disabled if the probe message bounces, it should be computationally infeasible to forge a probe bounce, and bogus probes bounces are simply ignored. When a probe is sent, the member's bounce score is reset to zero, since it's impossible to tell whether the probe actually reached its destination -- all you know is that it hasn't bounced... yet.
-Barry
On Thu, 2004-06-17 at 21:19, Barry Warsaw wrote:
Upgrade to Mailman 2.1.5, which sends out probe messages after the bounce threshold is reached. Members will only get disabled if the probe message bounces, it should be computationally infeasible to forge a probe bounce, and bogus probes bounces are simply ignored. When a probe is sent, the member's bounce score is reset to zero, since it's impossible to tell whether the probe actually reached its destination -- all you know is that it hasn't bounced... yet.
It will be interesting to see how this works in practice against one pathological bouncer I have on the exim list... Approximately one message in 10 (by volume - I can't match the bounces to any particular message) generates a bounce which has no useful information in it at all. The exim.org box is still running Mailman 2.0.x (very old box - about to be replaced with something with a better than clockwork CPU) so I can't do generic VERP , but VERP probes have failed to establish the link to the original address on the list... I am now wondering if someone is just sending me random bounces for the hell of it....
Nigel.-- [ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ]
This new VERP probe is ridiculous. One of my customer on a cpanel system had thousands of unsubscribes because of it. Why? Cpanel does not support in its current default exim setting VERP and there was no need for it because it is not on by default in mailman. Now out of the blue it is used even though VERP is turned off in mm_mfg.py And since this mailman VERP probe is unforgiving it unsubscribes right away! Not a good thing at all and the complaints from customers are coming in big time........
-----Original Message----- From: Nigel Metheringham [mailto:Nigel.Metheringham@dev.intechnology.co.uk] Sent: Monday, June 21, 2004 1:21 AM To: Barry Warsaw Cc: mailman-developers Subject: Re: [Mailman-Developers] [Greg Stark <gsstark@mit.edu>] Re: Bounceremoval parameters default values
On Thu, 2004-06-17 at 21:19, Barry Warsaw wrote:
Upgrade to Mailman 2.1.5, which sends out probe messages after the bounce threshold is reached. Members will only get disabled if the probe message bounces, it should be computationally infeasible to forge a probe bounce, and bogus probes bounces are simply ignored.
When a probe is sent, the member's bounce score is reset to zero, since it's impossible to tell whether the probe actually reached its destination -- all you know is that it hasn't bounced... yet.It will be interesting to see how this works in practice against one pathological bouncer I have on the exim list... Approximately one message in 10 (by volume - I can't match the bounces to any particular message) generates a bounce which has no useful information in it at all. The exim.org box is still running Mailman 2.0.x (very old box - about to be replaced with something with a better than clockwork CPU) so I can't do generic VERP , but VERP probes have failed to establish the link to the original address on the list... I am now wondering if someone is just sending me random bounces for the hell of it....
Nigel.
[ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ]
Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailma> n-developers
Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/somu chfun%40atlantismail.com
At 3:44 AM -0700 2004-06-24, Somuchfun wrote:
This new VERP probe is ridiculous.
[ ... deletia ... ]Not a good thing at all and the complaints from customers are coming in big time........
As JC correctly said (mea culpa), it does require support from the MTA. If your MTA doesn't support plus detail, then that would be a problem.
-- Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.
SAGE member since 1995. See <http://www.sage.org/> for more info.
At 2:36 PM -0400 2004-06-17, Greg Stark wrote:
Virus scans are only one type of bounce that could cause someone to be unsubscribed spuriously. For example, most mail servers have a maximum message size for example. Consider the security implications: all I have to do to mass unsubscribe many people--even everyone--on a list is send a message over 50k. Everyone using old versions of sendmail will be unsubscribed. A larger message will unsubscribe anyone using most modern MTAs. Nor do the tests that require multiple bounces protect anything; I just have to send my attack a few times quickly.
50k?!? Where are you getting this number? Maximum message size on most MTAs is usually a default of something like 1-10MB, or even unlimited. In more than ten years of specializing in running mail systems, I don't think I have *once* seen an MTA that was default configured to a maximum message size of just 50k.
Really Mailman should simply not trust outside data for any purpose. It should treat the bounces received from mailing list messages purely as hints. It should then send its *own* message with content not subject to any control from outside to the user. Only if that known inoffensive message bounces should it consider removing the user.
This is really a DOS security issue, though the worst case attack is unsubscribing many users of a list. That it gets triggered normally even when not specifically under attack only makes the problem apparent.
This is basically what Mailman is now doing. From the Mailman-2.1.5/NEWS file:
- The bounce processor has been redesigned so that now when an address's
bounce score reaches the threshold, that address will be sent a probe
message. Only if the probe bounces will the address be disabled. The
score is reset to zero when the probe is sent. Also, bounce events are
now kept in an event file instead of in memory. This should help
contain the bloat of the BounceRunner.
New supporting variables in Defaults.py: VERP_PROBE_FORMAT,
VERP_PROBE_REGEXP
REGISTER_BOUNCES_EVERY is promoted to a Defaults.py variable.-- Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.
SAGE member since 1995. See <http://www.sage.org/> for more info.
Brad Knowles <brad.knowles@skynet.be> writes:
At 2:36 PM -0400 2004-06-17, Greg Stark wrote:
Virus scans are only one type of bounce that could cause someone to be unsubscribed spuriously. For example, most mail servers have a maximum message size for example. Consider the security implications: all I have to do to mass unsubscribe many people--even everyone--on a list is send a message over 50k. Everyone using old versions of sendmail will be unsubscribed. A larger message will unsubscribe anyone using most modern MTAs. Nor do the tests that require multiple bounces protect anything; I just have to send my attack a few times quickly.
That's one whacky line-wrap algorithm your MUA uses.
50k?!? Where are you getting this number? Maximum message size on most MTAs is usually a default of something like 1-10MB, or even unlimited. In more than ten years of specializing in running mail systems, I don't think I have *once* seen an MTA that was default configured to a maximum message size of just 50k.
Well I said what I meant, "old version of sendmail". 50k was indeed the standard maximum size for sendmail installs prior to MIME attachments and html-mail and all these new-fangled gadgets.
I'm subscribed to plenty of mailing lists where attachments and html mail are severely discouraged so even today it wouldn't be out of place to refuse mail from these lists over 50k.
This is basically what Mailman is now doing. From the Mailman-2.1.5/NEWS file:
- The bounce processor has been redesigned so that now when an address's bounce score reaches the threshold, that address will be sent a probe message. Only if the probe bounces will the address be disabled. The score is reset to zero when the probe is sent. Also, bounce events are now kept in an event file instead of in memory. This should help contain the bloat of the BounceRunner.
Hum. Well, then, uh, great! :) Thanks.
Now I just have to figure out why I'm still being dropped. I guess some of these mailing lists are old versions of mailman.
This is version skew issue is going to be a big issue going forward. Users of mailing lists and ultimately any network service are at the mercy of the admins of every list they're on to upgrade.
-- greg
At 5:28 PM -0400 2004-06-17, Greg Stark wrote:
Well I said what I meant, "old version of sendmail". 50k was indeed the standard maximum size for sendmail installs prior to MIME attachments and html-mail and all these new-fangled gadgets.
Just how old are we talking here? Going back to the very early 90's (before version 8), there was no default maximum message size. I'd have to check, but I'm not even sure that there was an option to set the maximum message size at that time -- I don't think there were enough single-letter options left.
I'm subscribed to plenty of mailing lists where attachments and html mail are severely discouraged so even today it wouldn't be out of place to refuse mail from these lists over 50k.
Ahh, now that's a different matter. That's the setting on the mailing list, and most likely not all mailing lists on that server have the same setting. Most likely, the MTA is not set to the same value as the mailing list, and if it was then you'd be able to get in and back out cleanly. Once out, you'd only have problems if your message size was below the maximum set by the list but above the maximum allowed by most recipients, which is highly unlikely.
Now I just have to figure out why I'm still being dropped. I guess some of these mailing lists are old versions of mailman.
Could be. What's in the logs?This is version skew issue is going to be a big issue going forward. Users of mailing lists and ultimately any network service are at the mercy of the admins of every list they're on to upgrade.
Yup. Not much we can do about that.-- Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.
SAGE member since 1995. See <http://www.sage.org/> for more info.
Greg Stark wrote:
This is version skew issue is going to be a big issue going forward. Users of mailing lists and ultimately any network service are at the mercy of the admins of every list they're on to upgrade.
Short of some elaborate Microsoft/Semantic-scale auto-upgrade machinery, I don't know how to solve that particular problem. ;)
-Barry
Greg Stark wrote:
This is version skew issue is going to be a big issue going forward. Users of mailing lists and ultimately any network service are at the mercy of the admins of every list they're on to upgrade.
Short of some elaborate Microsoft/Semantic-scale auto-upgrade machinery, I have no idea how to solve that particular problem. ;)
-Barry
participants (5)
-
Barry Warsaw -
Brad Knowles -
Greg Stark -
Nigel Metheringham -
Somuchfun