Mailman 3 December 2000 - Mailman-Developers

Cookie error?
by John A. Martin 11 Dec '00

11 Dec '00

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Is there something to be done for this? - -------------- cut here ---->8 ---< head Dec 11 06:59:25 2000 admin(31509): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ admin(31509): [----- Mailman Version: 2.0 -----] admin(31509): [----- Traceback ------] admin(31509): Traceback (innermost last): admin(31509): File "/home/mailman/scripts/driver", line 96, in run_main admin(31509): main() admin(31509): File "/home/mailman/Mailman/Cgi/private.py", line 157, in main admin(31509): cookie='archive') admin(31509): File "/home/mailman/Mailman/SecurityManager.py", line 68, in WebAuthenticate admin(31509): return self.CheckCookie(key) admin(31509): File "/home/mailman/Mailman/SecurityManager.py", line 117, in CheckCookie admin(31509): c = Cookie.Cookie(cookiedata) admin(31509): File "/home/mailman/Mailman/Cookie.py", line 509, in __init__ admin(31509): if input: self.load(input) admin(31509): File "/home/mailman/Mailman/Cookie.py", line 546, in load admin(31509): self.__ParseString(rawdata) admin(31509): File "/home/mailman/Mailman/Cookie.py", line 573, in __ParseString admin(31509): M.set(K, apply(self.net_setfunc, (V,)), V) admin(31509): File "/home/mailman/Mailman/Cookie.py", line 421, in set admin(31509): raise CookieError("Attempt to set a reserved key: %s" % key) admin(31509): CookieError: Attempt to set a reserved key: expires admin(31509): [----- Python Information -----] admin(31509): sys.version = 1.5.2 (#1, Feb 1 2000, 16:32:16) [GCC egcs-2.91.66 19990314/Linux (egcs- admin(31509): sys.executable = /usr/bin/python admin(31509): sys.prefix = /usr admin(31509): sys.exec_prefix= /usr admin(31509): sys.path = /usr admin(31509): sys.platform = linux-i386 admin(31509): [----- Environment Variables -----] admin(31509): DOCUMENT_ROOT: /home/listproc/httpd/html/ admin(31509): SERVER_ADDR: 216.0.124.17 admin(31509): SERVER_PORT: 80 admin(31509): PATH_TRANSLATED: /home/listproc/httpd/html/child-proofing/ admin(31509): GATEWAY_INTERFACE: CGI/1.1 admin(31509): HTTP_USER_AGENT: EmailSiphon admin(31509): REMOTE_ADDR: 63.203.128.204 admin(31509): SERVER_NAME: lists.essential.org admin(31509): SCRIPT_FILENAME: /home/mailman/cgi-bin/private admin(31509): HTTP_ACCEPT: www/source, text/html, video/mpeg, image/jpeg, image/x-tiff, image/x-rgb, image/x-xbm, image/gif, */*, application/postscript admin(31509): REQUEST_URI: /mailman/private/child-proofing/ admin(31509): QUERY_STRING: admin(31509): SERVER_PROTOCOL: HTTP/1.0 admin(31509): PATH_INFO: /child-proofing/ admin(31509): HTTP_HOST: lists.essential.org admin(31509): REQUEST_METHOD: GET admin(31509): SERVER_SIGNATURE: <ADDRESS>Apache/1.3.14 Server at lists.essential.org Port 80</ADDRESS> admin(31509): SCRIPT_NAME: /mailman/private admin(31509): SERVER_ADMIN: root@localhost admin(31509): SERVER_SOFTWARE: Apache/1.3.14 (Unix) (Red-Hat/Linux) mod_perl/1.23 admin(31509): PYTHONPATH: /home/mailman admin(31509): HTTP_COOKIE: browser=3FCB80CC3A34BF2E; expires=Tue, 31-Dec-2002 05:00:00 GMT; domain=.zdnet.com; path=/ admin(31509): REMOTE_PORT: 4868 - ---- 8<------- cut here ----------> tail jam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See <http://www.gnupg.org/> iEYEARECAAYFAjo00AkACgkQUEvv1b/iXy+e2ACghQLA7YjnTJd694ZkvNaC/0kC JAAAoJxIM+E3kMreQPtzqtKenh+rPTi4 =Uzey -----END PGP SIGNATURE-----

1 0

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by J C Lawrence 10 Dec '00

10 Dec '00

On Sat, 9 Dec 2000 03:09:26 -0600 Christopher Lindsey <lindsey(a)ncsa.uiuc.edu> wrote: >> I'm passing this along mostly as a FYI, but also as a sanity >> check. I sent this out to list-managers tonight, to bring up an >> issue that sort of crystalized this afternoon and made me realize >> that I think we have the beginnings of a problem in mail list >> land. Your thoughts are welcome....If I'm right, well, oh, >> boy. If I'm wrong -- I'd love to find out my idea won't work, but >> I think it's not only possible, but fairly easy. > Hi Chuq, > Yes, this has definitely been troublesome. I've blocked many > commercial sites like findmail.com (egroups) and remarq.com from > my lists because of their secret archiving that displays email > addresses to the public, but at least they don't spam the lists > back. But of course anyone can browse these sites and get > addresses to their heart's content, then forge MAIL FROM: to sneak > mail into the lists. > I'm not sure what the right thing is to do. MLMs like sympa ( > http://listes.cru.fr/sympa/ > ) are definitely moving in the right direction with S/MIME > signatures/encryption and X509 user certs, but that still doesn't > stop someone from using throwaway certs to spam several lists or > from harvesting addresses. You are failing to distinguish between two problems: 1) Is this post from someone I know? 2) Is this post from who I think it is from? #1 is handled by any form of digital signature, and is handled especially well by non-centrally signed/verified forms (eg PGP, GPG, etc). All that's needed is for the list member to convey or have conveyed to you their public key. #2 is handled by having a public key for a member. It doesn't matter if it is signed or if Verisign or some other set of pains vouch for it. It merely matters that a current post from them cross-checks with their signature. So what problem does that leave? Is this post, which comes from a member for whom I have a key, and which ckecks against his key, SPAM? It doesn't matter where the keys or certs come from. It doesn't matter if a trusted authority is involved or not. It is a human question. I'm sure that spammers are just as capable as we are at getting demo certs from Thawte, or in cooking up new GPG keys willy nilly. Again, this is a question of trust networks, and is a subset of the problem of reputational systems. Unfortunately, unlike the systems I normally needing reputational systems that I normally spend my time looking at, this is not an area which panders to central databases, and over-arching solutions. SPAM detection at the individual membership level is a question of individual evaluation. > The problem is that when these methods are used for authentication > they just prove that the email address sending the stuff is who we > think he or she is. But at least you can't forge the source email > address to look like it's coming from a list member who is allowed > to post (well, it's harder :) It raises the bar to requiring that SPAMers compromise members keys in a wholesale fashion. While certainly possible (I spent an hour last weekend seeing how many exploitable Windows systems I could find within that hour. I gave up after about 20 minutes when the count passed treble digits), the barrier to entry is much larger and the ROI is much smaller (a compromised key only gains access to a few lists and posting venues). > I think that there's an implicit level of trust that has to be > honored in mailing list management. Even SASL-based SMTP > authentication from ISPs isn't going to prevent throw-away > accounts from being used. Until we can get a fingerprint or > cornea scan (or even a driver's license) with each mailing list > subscription and compare it against a master database (which I'm > not advocating), you can't be 100% sure of the users. No. Think in terms of trust, not identity. Do you really care that you can track that particularl membership back to one identifiable human body? Really? Over the entire planet? Or do you really just care that JoeBlow has posted signal in the past and you feel that you can trust him to post signal in the future? Or more simply, if you are not going to operate on past posting behaviour as your trust metric: Why do you trust member X to not post spam? What are the criteria you use for making that decision? Why are those criteria trustable? What are the risks? I like past behaviour as it is simple, non-invasive (I don't need to know who anybody is), and it fits transparently in as an invisible extention of traditional list moderation models. > For now I'd say that the best method is a social one; require > references when people want to subscribe to your list. Ask them > which lists they participate on, an example post from another > list, etc. But ultimately it becomes a judgement call by the > listowner either way. For a few years I ran a list where membersip was by invitation only -- a current member had to invite you. It worked well. Membership grew steadily, more than 70% of the members regularly posted to the list, and signal was high. Later I moved the list to free subscription with posting authority granted on application only, with applications needing to be accompanied by a proposed first post. This too worked well with minor caveats. List membership grew roughly twice as fast, but the poting percentage fell to around 40%. The caveats were in maintaining the approved poster list, and in particular determinging and enforcing policy for removing posting authority was painful. I currently run an open subscription model with hand moderation of all posts. Again, this has worked well. Subscription rates are roughly 4 times higher than ever before, but posting percentages are down in the <10% range. It is of course labour intensive. Recently I've moved to not only hand moderating the list, but hand editing posts (eg to remove HTML, over quoting, inflammatory content, etc) marking each post I edit with a comment as to the changes. While I haven't been doing this for long, subscription rates have noticably increased by perhaps 20% (tho the sample time is small). The work load on me is non-trivial. It moved me from the position of gatekeeper to editor, a position I'm willing to occupy, but am not keen on. In the end you are trading editorial control (your trust model and signal definition) for work. Ultimately you are attempting to automate the process of determining signal. The initial approximation is by determining signal sources. The approximation we are discussing above is in determining if signal sources really are signal sources. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

2 2

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by Darrell Fuhriman 10 Dec '00

10 Dec '00

Chuq Von Rospach <chuqui(a)plaidworks.com> writes: > But Murr Rhame on list-managers said something that made me think of a > possible answer -- new subscribers automatically go into "hold for > approval" mode. it'd be another flag in the user record (like digest Yes, Lyris does this. It's a feature I've wanted to add to mailman, but haven't had the bandwidth to do it. It would be even nicer if it were smart enough to set a threshold and as soon as they've had that many posts approved, the restriction is automatically lifted. There's a couple other things that Lyris does that would be cool, but I can't remember what they are right now. :) Darrell

5 5

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by J C Lawrence 10 Dec '00

10 Dec '00

On Sat, 9 Dec 2000 20:51:07 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > At 8:47 PM -0800 12/9/00, J C Lawrence wrote: >> The fact that the posts happened to be on-topic is slightly droll >> and beyond my ability to explain. > I've had that happen, and it was finally tracked to another list > member who was pissed at the first, and attempting to ruin their > reputation on the list. Don't downplay the underlying personal > interactions and politics of a list, especially one iwth strong > emotions, where otherwise mature people act like three year olds > over a stale donut. Yeah, that was my first suspicion and it really fit all the way up to the fact that two of them were actually pretty good posts that sparked useful and deep running threads, and the third was non-commital to the point of being ho-hum (technically on-topic, simply re-iterated prior traffic). <shrug> The members and I in each case just agreed to >boggle< and keep running without comment, under the suspicion that comment would only encourage the activity. Problem is, I know how trivially simple it is: $ dig python.org MX ... python.org. 23h47m13s IN MX 50 dinsdale.python.org. ... $ telnet dinsdale.python.org smtp HELO kanga.nu MAIL FROM: <insert member address here> RCPT TO: mailman-developers(a)python.org DATA ... . QUIT (which in one case appears to be exactly what happened) -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by J C Lawrence 09 Dec '00

09 Dec '00

On Sat, 9 Dec 2000 15:20:08 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > In practice, for many places, that's simply not an option until > SOMEONE figures out how to get AOL to support it, because without > AOL, a significant chunk of the audience can't do it, making it > worthless (and while individual list admins can tell AOL to take a > flying leap, the typical one won't, and many of us can't. So any > MLM that comes up with a solution that effectively locks out AOL > is a MLM that dies in the marketplace...) Does anyone here have contacts at AOL? I used to know people in their NOC and in their IS team (which doesn't really apply to this end). I'll see if I can't dig them up to see what might be needed to get some basic things done (kinda difficult at this time of year). -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by J C Lawrence 09 Dec '00

09 Dec '00

On Sat, 9 Dec 2000 17:32:53 -0500 Vince Sabio <vince(a)vjs.org> wrote: > Like Chuq, I shudder at the thought of someone forging subscriber > addresses to spam mailing lists. I've already had posts (three times now) to my lists purportedly from members of that list (From: header etc), where the members in question deny ever having written those posts, and careful examination of the Received headers supports their claim that it wasn't them. The simple transaltion is that it sure as heck *looks* like people are already forging mail as members of lists. The fact that the posts happened to be on-topic is slightly droll and beyond my ability to explain. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

2 1

FYI -- mailback validations no longer safe?
by Chuq Von Rospach 09 Dec '00

09 Dec '00

I'm passing this along mostly as a FYI, but also as a sanity check. I sent this out to list-managers tonight, to bring up an issue that sort of crystalized this afternoon and made me realize that I think we have the beginnings of a problem in mail list land. Your thoughts are welcome....If I'm right, well, oh, boy. If I'm wrong -- I'd love to find out my idea won't work, but I think it's not only possible, but fairly easy. ---- I somewhat hesitate to bring this up, but I heard of another situation today that seems to fit in, and I think it's time to raise the issue. I'm beginning to think that mailback validation as an anti-spam technique has been beaten. Worse, I think there are now spam systems written that will beat them in an automated way. I will say up front I don't have a smoking gun. If and when I find one, I'll say so. But I'm now beginning to think the spammers have figured out how to beat mailbacks. Someone we know runs a list on egroups. Twice today he was spammed by the porn spammers -- from subscribed accounts. This isn't the first time I've heard of this in the last few weeks, but he's someone I know runs a pretty clean ship. to get hit by two separate porn spammers on the same day, in independent attacks, that raises a real warning flag, because where the porn spammers innovate, everyone else follows. In the last few years, there have been some significant, fundamental changes in the internet (duh). Now that I've spent a few hours thinking like a spammer, I realize these changes make it trivial for a *smart* spammer with some basic resources to circumvent mailbacks. Here's how: First, you get access to some domains -- the key ot mailbacks is that you have to have physical access to the mailback address to finish the confirmation. n today's internet, however -- that isn't a big deal. you register one for yourself, hook yourself up using dynamic DNS while attached via PPP to UUnet or one of the ISPs, and you have a fully functional mailserver. Or if you prefer, simply break into some lameoid's home machine sitting on a cable modem and borrow imstupid.org while he's not paying attention. Either way, you now have a spammer with a set of available domains, which he's either bought, borrowed or stolen, and access to the return mail sent to those domains. this spammer's built a validation-bot. It's fed a list of mailing lists, and it spends all of its time figuring out what MLM it uses (not hard), and subscribing accounts to them. it can send the appropriate subscribe messages, read the confirmations, and send appropriate confirmations. Even better, if the MLM supports nomail, you turn off deliveries, so you don't run the risk of inbound e-mail alerting anyway on imstupid.org (if you think about it, the only thing that has to be on imstupid.org is a set of aliases forwarding to your real machine, and only for the period of time you're setting up the subscriptions. If you're real lucky, you find out you can hack their DNS and set up really.imstupid.org, and send EVERYHTING offsite). The spammer lets his bot run for a while, and tracks the database with which address is subscribed to which list. He can even subscribe multiples from multiple domains if he wants, and let them lie fallow. When you block off one, it falls back and sends from the next. he now owns your list, at least until you figure out what's going on and nuke the subscribed address. But if you think about it, once that validation handshake is complete, there's never ANY further validation. so he can set up temporary shop, validate to his heart's content, and then later on, after all the temporary stuff is safely hidden away, spam from anywhere, safely. Because he knows the address that will get him on the list. If this is true, and it's beginning to look like egroups is a target of one attack, and I've heard rumors of some mailman lists being hit as well, then lists that depend on mailback validation have a problem. And I think there's been a feeling that mailbacks are the one true way of validation to the point where there hasn't been much (if any) thought about improved techniques or alternatives. And if I, having spent four hours on the "how would I do this?" train of thought can find a fairly easy to implement design, so can those that aren't so pure of heart and don't say their prayers at night. This isn't something the "buy a CD for $200" lameoid spammers can do (but I'll bet a really good spammer could build a system to do it taht's turnkey. there's enough wide open hardware out on the net, especially overseas, that you could get a good 6 month run before neough stuff shut you down to make it not worth it...), but the port spammers and gambling spammers and the spammers for hire? it's perfect for them. I've felt for a while that the list community was way too comfortable with mailbacks as "safe and unbeatable". I'm now seeing what I think is evidence that this is no longer true. And I'm afraid that because we have sat back adn not innovated here, we're going to end up behind the eight ball. and I don't see any easy answers if I'm right -- only that if I am wrong, I won't be wrong forever. So I'm throwing it to the list, to see if there's information others have that might corroborate what I think I'm seeing (that you may not have realized for waht it might be), or t poke holes in my analysis, or to start thinking of how to deal with it. There I go, being a troublemaker again... (grin, sort of) thoughts? chuq -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) We're visiting the relatives. Cover us.

3 4

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by J C Lawrence 09 Dec '00

09 Dec '00

On Sat, 9 Dec 2000 00:22:37 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > I'm beginning to think that mailback validation as an anti-spam > technique has been beaten. Worse, I think there are now spam > systems written that will beat them in an automated way. I've written on this before to the Mailman lists. I have similar suspicions. Like you I have no smoking guns, but I have a suggestive evidence. > I will say up front I don't have a smoking gun. If and when I find > one, I'll say so. But I'm now beginning to think the spammers have > figured out how to beat mailbacks. Its hardly complex -- just look for key strings in messages coming to an account, and then bounce back messages accordingly. Given someone with minimal scripting knowledge, what, 30 minutes? Four simple patterns will cover 95% of the lists out there. > Someone we know runs a list on egroups. Twice today he was spammed > by the porn spammers -- from subscribed accounts. This isn't the > first time I've heard of this in the last few weeks, but he's > someone I know runs a pretty clean ship. to get hit by two > separate porn spammers on the same day, in independent attacks, > that raises a real warning flag, because where the porn spammers > innovate, everyone else follows. Occam's razor indicates that this could be done equally well thru mail forgery of a blameless member. > he now owns your list, at least until you figure out what's going > on and nuke the subscribed address. But if you think about it, > once that validation handshake is complete, there's never ANY > further validation. so he can set up temporary shop, validate to > his heart's content, and then later on, after all the temporary > stuff is safely hidden away, spam from anywhere, safely. Because > he knows the address that will get him on the list. Bingo. This is one of the base reasons I now hand moderate my main lists. I'm looking hard at going back to a posting_authority setup (members prove themselves worthy of automatic posting (no moderator overview)), but Mailman does not currently lend itself to that model. Yet. (Using approved posted in Mailman is not sufficiently maintainable) > If this is true, and it's beginning to look like egroups is a > target of one attack, and I've heard rumors of some mailman lists > being hit as well, then lists that depend on mailback validation > have a problem. And I think there's been a feeling that mailbacks > are the one true way of validation to the point where there hasn't > been much (if any) thought about improved techniques or > alternatives. When you get down to it this is a question of trust models, and is a susbset of the problem of reputational systems. Its a non-trivial problem. > I've felt for a while that the list community was way too > comfortable with mailbacks as "safe and unbeatable". I'm now > seeing what I think is evidence that this is no longer true. And > I'm afraid that because we have sat back adn not innovated here, > we're going to end up behind the eight ball. and I don't see any > easy answers if I'm right -- only that if I am wrong, I won't be > wrong forever. I'm at the point where I'm willing to lay money on your being not only right, but being visibily demonstrated as right within the next calendar year. We have two problems: 1) Determining that a given member of a list is not a spammer. 2) Determining that a given post is not a SPAM The first can be largely addressed via putting in mechanisms where N moderator approved posts are required before being granted posting authority. Its a barrier to entry technique -- not secure, but certainly not profitable for the spammer in terms of ROI. As a side comment, this is one of the features I'd like to see rolled in the next Mailman design we're discussing (given the model I'm musing, it should be trivial). The second is a horrible nasty problem in this age of mail forgery and the ease of harvesting member addresses from lists (especially once you are a subscriber). Given that a spammer can susbcribe and can then harvest addresses with (presumably) posting authority with no more than a couple hours worth of scripting and a little time waiting while his bot runs, the simple MESSAGE_FROM_XXX_IS_OKAY metric is likely to last no longer. So what's the final solution? I don't think there is an elegant solution without involving presumed non-forgeable proofs of identity (ie public key crypto). Doing that requires a broadscale PKI structure (a horrible problem in and of itself), severe changes in user habits, and a host of other invasive non-trivial changes. Its going to happen tho. TLS/SMTP is just not enough. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=-- On Sat, 9 Dec 2000 00:22:37 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > I'm passing this along mostly as a FYI, but also as a sanity > check. I sent this out to list-managers tonight, to bring up an > issue that sort of crystalized this afternoon and made me realize > that I think we have the beginnings of a problem in mail list > land. Your thoughts are welcome....If I'm right, well, oh, boy. If > I'm wrong -- I'd love to find out my idea won't work, but I think > it's not only possible, but fairly easy. > ---- > I somewhat hesitate to bring this up, but I heard of another > situation today that seems to fit in, and I think it's time to > raise the issue. > I'm beginning to think that mailback validation as an anti-spam > technique has been beaten. Worse, I think there are now spam > systems written that will beat them in an automated way. > I will say up front I don't have a smoking gun. If and when I find > one, I'll say so. But I'm now beginning to think the spammers have > figured out how to beat mailbacks. > Someone we know runs a list on egroups. Twice today he was spammed > by the porn spammers -- from subscribed accounts. This isn't the > first time I've heard of this in the last few weeks, but he's > someone I know runs a pretty clean ship. to get hit by two > separate porn spammers on the same day, in independent attacks, > that raises a real warning flag, because where the porn spammers > innovate, everyone else follows. > In the last few years, there have been some significant, > fundamental changes in the internet (duh). Now that I've spent a > few hours thinking like a spammer, I realize these changes make it > trivial for a *smart* spammer with some basic resources to > circumvent mailbacks. Here's how: > First, you get access to some domains -- the key ot mailbacks is > that you have to have physical access to the mailback address to > finish the confirmation. n today's internet, however -- that isn't > a big deal. you register one for yourself, hook yourself up using > dynamic DNS while attached via PPP to UUnet or one of the ISPs, > and you have a fully functional mailserver. Or if you prefer, > simply break into some lameoid's home machine sitting on a cable > modem and borrow imstupid.org while he's not paying > attention. Either way, you now have a spammer with a set of > available domains, which he's either bought, borrowed or stolen, > and access to the return mail sent to those domains. > this spammer's built a validation-bot. It's fed a list of mailing > lists, and it spends all of its time figuring out what MLM it uses > (not hard), and subscribing accounts to them. it can send the > appropriate subscribe messages, read the confirmations, and send > appropriate confirmations. Even better, if the MLM supports > nomail, you turn off deliveries, so you don't run the risk of > inbound e-mail alerting anyway on imstupid.org (if you think about > it, the only thing that has to be on imstupid.org is a set of > aliases forwarding to your real machine, and only for the period > of time you're setting up the subscriptions. If you're real lucky, > you find out you can hack their DNS and set up > really.imstupid.org, and send EVERYHTING offsite). > The spammer lets his bot run for a while, and tracks the database > with which address is subscribed to which list. He can even > subscribe multiples from multiple domains if he wants, and let > them lie fallow. When you block off one, it falls back and sends > from the next. > he now owns your list, at least until you figure out what's going > on and nuke the subscribed address. But if you think about it, > once that validation handshake is complete, there's never ANY > further validation. so he can set up temporary shop, validate to > his heart's content, and then later on, after all the temporary > stuff is safely hidden away, spam from anywhere, safely. Because > he knows the address that will get him on the list. > If this is true, and it's beginning to look like egroups is a > target of one attack, and I've heard rumors of some mailman lists > being hit as well, then lists that depend on mailback validation > have a problem. And I think there's been a feeling that mailbacks > are the one true way of validation to the point where there hasn't > been much (if any) thought about improved techniques or > alternatives. > And if I, having spent four hours on the "how would I do this?" > train of thought can find a fairly easy to implement design, so > can those that aren't so pure of heart and don't say their prayers > at night. This isn't something the "buy a CD for $200" lameoid > spammers can do (but I'll bet a really good spammer could build a > system to do it taht's turnkey. there's enough wide open hardware > out on the net, especially overseas, that you could get a good 6 > month run before neough stuff shut you down to make it not worth > it...), but the port spammers and gambling spammers and the > spammers for hire? it's perfect for them. > I've felt for a while that the list community was way too > comfortable with mailbacks as "safe and unbeatable". I'm now > seeing what I think is evidence that this is no longer true. And > I'm afraid that because we have sat back adn not innovated here, > we're going to end up behind the eight ball. and I don't see any > easy answers if I'm right -- only that if I am wrong, I won't be > wrong forever. > So I'm throwing it to the list, to see if there's information > others have that might corroborate what I think I'm seeing (that > you may not have realized for waht it might be), or t poke holes > in my analysis, or to start thinking of how to deal with it. > There I go, being a troublemaker again... (grin, sort of) > thoughts? > chuq > -- Chuq Von Rospach - Plaidworks Consulting > (mailto:chuqui@plaidworks.com) Apple Mail List Gnome > (mailto:chuq@apple.com) > We're visiting the relatives. Cover us. > _______________________________________________ Mailman-Developers > mailing list Mailman-Developers(a)python.org > http://www.python.org/mailman/listinfo/mailman-developers -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by Landon Curt Noll 09 Dec '00

09 Dec '00

> Is it possible to put a list into a mode where it is partially moderated and > partially unmoderated? Where moderated applies not only to posting but to > all other operations as well, such as querying the list members. You raise a good point here. In my system that is under development, I use the number of postings and the posting rate as part of the credit checks. You could say, for example, the first 3 EMails fail the ``credit check'' and must be processed by hand. Again failing the ``credit check'' does not mean rejection, just that the message must go to a human for social checks and their best judgement. Another thing I have thought of is to check on the 'days since last message in relation to the average member' to watch for sleeper EMail addresses being enabled by a spammer. chongo <> /\oo/\

1 0

Re: [Mailman-Developers] FYI -- mailback validations no longer safe?
by Landon Curt Noll 09 Dec '00

09 Dec '00

> I think that there's an implicit level of trust that has to be honored > in mailing list management. Even SASL-based SMTP authentication from > ISPs isn't going to prevent throw-away accounts from being used. > Until we can get a fingerprint or cornea scan (or even a driver's > license) with each mailing list subscription and compare it against > a master database (which I'm not advocating), you can't be 100% > sure of the users. > > For now I'd say that the best method is a social one; require > references when people want to subscribe to your list. Ask them which > lists they participate on, an example post from another list, etc. > But ultimately it becomes a judgement call by the listowner either way. I think Chris is right on target here. Judgement and non-automated checks are one good way of reducing the number of spammers. For one list, I depend on the person telling me something or asking me something about topic. It all I get in a join request, I write back asking them if they really wanted to join the list and "BTW: which version of XXX are you using?" or "What would you like to see improved in XXX?" or "In what ways are you using XXX?" or something like that. Yea, I suppose a spammer could go through the effort to learn about XXX to get on the list ... that would be a lot of special work just to target a small list. =-= Processing requests ``by hand'' is important in order to use ``Judgement and non-automated checks''. Clearly this does not work for huge lists or where the list owner is too busy to process each and every request. An alternative to perform a ``credit check'' first: *) Is the request from a recently created domain? *) Did the DNS lookup of the IP address of the sender match the envelope? *) Did the request come from an area flagged by the RBL, DUL, RSS or dorkslayers? *) Did the request come from a known spam haven or place from which you had problem before? None of these checks are perfect, but that is OK. The false negatives of things like the RSS are not a problem either. Anything that can pass all of the credit checks gets processed automatically. Anything else you process ``by hand'' using ``Judgement and non-automated checks''. =-= Prevention helps. Avoid placing your list or list-request address on the web (or on Usenet) in text form. Put your instructions / list information as text in the form of a jpg. So far spammers are not scanning jpg images for text and groking them ... I hope! =-= Do not use a single EMail address for the list. Make use of sub-domains. For example instead of: XXX-list(a)foo.org Use: XXX-list(a)Jrandom-text.XXX.m.foo.org where `Jrandom-text' is different for each and every list member. Here `Jrandom-text' is could be random collection of chars produced by your friendly neighborhood Lavarand server. :-) Perform automated in-bound checks on EMail going into 'Jrandom-text.XXX.m.foo.org'. Use the 'credit checks' listed above. Look for the EMail being relayed thru the subscription address. Look for procmail-like spam triggers. If the message passes the sanity check, allow it to go thru the system in an automated fashion. If a flag is raised, process it by hand. When EMail it sent out to the list, send it out with each message being from XXX-list(a)Jrandom-text.XXX.m.foo.org style addresses. All list users know (or care) is that they have / use / receive EMail from a single address. They need not know / or care that the single address is different from person to person. Another advantage to the ``non-single EMail address for the list'' is that if an address becomes a problem, you can turn it off without impacting the entire list. Say the user who is assigned 'XXX-list(a)table.XXX.m.foo.org' gets hit by a virus that spams their address book. Say this virus is able to duck under the ``credit checks''. Say despite all of your efforts the spam goes onto the list. Well you: *) disable 'XXX-list(a)atlbe.XXX.m.foo.org' *) apologize to the XXX-list Later on if the member cleans up their mess, allow them to re-join by assigning them a new address: 'XXX-list(a)lztyg.XXX.m.foo.org'. If people object to Jrandom-text, one can always use a randomly selected word-number combination. chongo <> /\oo/\

2 1

Re: [Mailman-Developers] about qrunner and locking
by Ralph Corderoy 09 Dec '00

09 Dec '00

Hi, > > Sorry if this is wandering a little off topic, but what's a `beat'? > > What stops > > > > A writes `A' > > A waits a beat > > A short wait. Just long enough to let things settle so you get > reliable data. It's really a musical term that's I've absconded > with... This won't work. A doesn't know how long to wait because it doesn't know how quickly B might get around to trampling the `A'. Ralph.

1 0

Re: [Mailman-Developers] about qrunner and locking
by Ralph Corderoy 09 Dec '00

09 Dec '00

Hi Chuq, > here's another way that should work: each record has a locking field > in it. When qrunner wants to execute that item, it reads the field. > If the field is NULL, it writes its ID (hwatever it is, guaranteed > unique) into that locking field. It then waits a beat, and reads it > back. if it reads back its own ID, it knows it owns the record and > can execute it. If it reads back someone else's ID, it lost the lock, > but someone else owns the record so it can skip it and move on. Sorry if this is wandering a little off topic, but what's a `beat'? What stops A reads, gets NULL B reads, gets NULL A writes `A' A waits a beat A reads `A' and has the lock B held up for a little writes `B' B waits a beat B reads `B' and also has the lock I think I'm missing something and that's annoying :-) Ralph.

2 1

Thoughts on splitting qrunner
by barry＠digicool.com 08 Dec '00

08 Dec '00

http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/SplittingQrunner Comments please, either here or in the Wiki! -Barry

3 6

Re: [Mailman-Developers] about qrunner and locking
by J C Lawrence 08 Dec '00

08 Dec '00

On Fri, 8 Dec 2000 09:36:11 +0100 Thomas Wouters <thomas(a)xs4all.net> wrote: > Over NFS, almost nothing is atomic. And even if you do grab an > atomic operation, you can get .... The only operation which I've found guaranteed atomic across all NFS implementations is creat(). > I'm afraid that there isn't a good solution to your problem, right > now. In all honesty, and I say this with all my professional years > of experience in this area, NFS sucks large granite elephant > testicles through a very thin straw. (To butcher a Pratchett > quote, "NFS is like a vampire; it bites, it sucks, and it leaves > you lifeless") Yuo're beginning to sound a bit like me, only you're more polite. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: Huge qrunner process
by sigma＠pair.com 08 Dec '00

08 Dec '00

Following up on my own message... qrunner isn't the only culprit. senddigests is worse, since it doesn't stop itself from running indefinitely. There is a definite memory leak or inefficiency somewhere. Just the following fragment of code, edited from senddigests, is enough to send the memory usage sky-high: def main(): for listname in Utils.list_names(): mlist = MailList.MailList(listname, lock=0) del mlist The "del mlist" doesn't help. I've noticed that one pathological list has a 12MB config.db file. If loading config.db is inefficient by a factor of eleven, then that could explain the swelling to 135MB. With that list removed, the memory usage peak is just 89MB. Is there a way to tell if Python is deleting the MailList object when mlist gets reassigned, so I can find out if there is a leak each time a list is loaded? Otherwise it's just inefficient memory usage in proportion to the size of config.db Thanks, Kevin ----- Forwarded message from sigma(a)pair.com ----- >From mailman-developers-admin(a)python.org Thu Dec 07 12:00:16 2000 Delivered-To: sigma(a)smx.pair.com Delivered-To: sigma(a)pair.com Delivered-To: mailman-developers(a)new.python.org Delivered-To: mailman-developers(a)python.org Message-ID: <20001207115954.1364.qmail(a)smx.pair.com> From: sigma(a)pair.com To: mailman-developers(a)python.org X-Mailer: ELM [version 2.4ME+ PL40 (25)] Subject: [Mailman-Developers] Huge qrunner process Sender: mailman-developers-admin(a)python.org Errors-To: mailman-developers-admin(a)python.org X-BeenThere: mailman-developers(a)python.org X-Mailman-Version: 2.0 Precedence: bulk List-Help: <mailto:mailman-developers-request@python.org?subject=help> List-Post: <mailto:mailman-developers@python.org> List-Subscribe: <http://www.python.org/mailman/listinfo/mailman-developers>, <mailto:mailman-developers-request@python.org?subject=subscribe> List-Id: Mailman mailing list developers <mailman-developers.python.org> List-Unsubscribe: <http://www.python.org/mailman/listinfo/mailman-developers>, <mailto:mailman-developers-request@python.org?subject=unsubscribe> List-Archive: <http://www.python.org/pipermail/mailman-developers/> Date: Thu, 7 Dec 2000 06:59:54 -0500 (EST) Can anyone enlighten me about why the qrunner process might need a tremendous amount of memory? Running Mailman 2.0 on a FreeBSD 4.1.1-STABLE server with Python 2.0. There are about 1500 lists, but the qfiles directory only has 32 files in it. Nonetheless, each minute when qrunner runs, it looks like this in top: 59606 mailman -2 20 135M 74456K getblk 0:01 8.93% 3.52% python 135 MB? It seems excessive. Any insight would be appreciated :) Thanks, Kevin _______________________________________________ Mailman-Developers mailing list Mailman-Developers(a)python.org http://www.python.org/mailman/listinfo/mailman-developers ----- End of forwarded message from sigma(a)pair.com -----

3 7

glitch in 2.0 check_perms
by Chuq Von Rospach 08 Dec '00

08 Dec '00

Justfound a glitch in 2.0 check_perms. I was getting a failure on one of my lists when trying to process an admin request, because of a permission problem. I found that the listname/request.db file was mode 644 instead of 664, so that anything trying to open it for writing failed. I don't know how it got that way, none of the other lists have had this problem. But when I found the problem, I ran check_perms, and it reported no problem, which is clearly wrong... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) We're visiting the relatives. Cover us.

2 1

giving mailman egroups like functionality?
by Rajat 08 Dec '00

08 Dec '00

hello friends, am new to this list ... but have been using mailman for quite some time. am also aware of howto program in python .... but am no big expert in python! one of my implementation projets requires what egroups gives .... Now mailman is an excellent tool for mailing list management with excellent web based GUI interface .... what i want is howto - 1.) a web page thru which users can login ( that i have taken care of - assumed that users are already in my ldap database and they get into their user area say like a mailbox ) and can create their own mailing lists rather than someone makes for them from a shell! 2.) then everytime they login they see what lists they are moderator are for plus also thise lists which they have subscribed too. 3.) when they click on the list they are moderator are for then it should give the link to http://hostname/mailman/admin/listname 4.) and when they click the link to the list they have been subscribed they go the link http://hostnanme/mailman/listindo/listname 5.) also how can i execute the commands present in my ~Mailman/bin directory .. eg: list_lists i would really appreciate if you could help me out in doing this ... plus if these functional are, if u think not required in mailman or do not confer to mailman standards, then please at least do help me out howto go about it .. either, a) changing the python code of mailman b) any way i can achieve this my executing any existing binary commands which comes with Mailman thru perl or PHP. The requirements are on a urgent basis ... so would appreciate all your help Thnx Regards Rajat P.S. : i have told that s/w like lyris list manager does this .. but am a hardcore mailman fan .. and i would never like to use any other mailing list manager :) __________________________________________________ Do You Yahoo!? Yahoo! Shopping - Thousands of Stores. Millions of Products. http://shopping.yahoo.com/

1 0

Re: [Mailman-Developers] msg_header: insertion of sender name/address
by J C Lawrence 07 Dec '00

07 Dec '00

On Thu, 7 Dec 2000 15:07:36 -0800 (PST) Dan Mick <Dan.Mick(a)West.Sun.COM> wrote: > Well, it looks like the header and footer get the entire mlist's > dictionary, plus mm_cfg.CGIEXT, to interpolate with, but nothing > per-message. I suppose the right answer would be to change > Decorate.py (or add a new module parallel to it) to add in > selected fields from the message dictionary (assuming there are no > name clashes between the mlist dictionary and the message > dictionary, anyway). Simpler would be to prefix Mailman with a procmail/maildrop/whatwever script that copied the From: header into the body of the message. The handling of MIME problems of course is a different matter. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] msg_header: insertion of sender name/address
by Dan Mick 07 Dec '00

07 Dec '00

Well, it looks like the header and footer get the entire mlist's dictionary, plus mm_cfg.CGIEXT, to interpolate with, but nothing per-message. I suppose the right answer would be to change Decorate.py (or add a new module parallel to it) to add in selected fields from the message dictionary (assuming there are no name clashes between the mlist dictionary and the message dictionary, anyway). d = Utils.SafeDict(mlist.__dict__) d['cgiext'] = mm_cfg.CGIEXT # interpolate into the header try: header = string.replace(mlist.msg_header % d, '\r\n', '\n') I don't see a way it can be done without hackery. > I had to read it twice, but this is not another question about whether user > names can be merged into a message for custom stuff. Any idea if there is > a way to grab the Sender/From lines and put them into the message body? I > think that the problem is that the mail systems that she is using munge up > the display of senders. > > Cheers. > > --chris > > > >From: Patrick_Healy(a)nywd.uscourts.gov > >X-Mailer: ccMail Link to SMTP R8.30.00.7 > >Date: Thu, 07 Dec 2000 16:39:40 -0500 > >To: <ckolar(a)admin.aurora.edu> > >Subject: Mailman > > > > > >Mr. Kolar, > > > >Sorry to bother you with this question, but I've not received any responses to > >my question on the list. I really appreciate any help you could give me > >on this > >problem. > > > >In a nutshell - The federal judiciary uses Cc:Mail as it's enterprise email > >system (soon to be replaced by Notes). The Mailman lists that I've > >created and > >administer work great to those clients. However, they have no indication > >in the > >body of the message of who sent the message. > > > >This means that unless the sender remembers to sign there note, it essentially > >becomes anonymous. > > > >Is there an environment variable that I can insert into the custom header that > >will cause the sender's name to be displayed? > > > >Thanks! > > > >Pat Healy > >U.S. District Court, Western District of New York > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://www.python.org/mailman/listinfo/mailman-developers

1 0

msg_header: insertion of sender name/address
by Christopher Kolar 07 Dec '00

07 Dec '00

I had to read it twice, but this is not another question about whether user names can be merged into a message for custom stuff. Any idea if there is a way to grab the Sender/From lines and put them into the message body? I think that the problem is that the mail systems that she is using munge up the display of senders. Cheers. --chris >From: Patrick_Healy(a)nywd.uscourts.gov >X-Mailer: ccMail Link to SMTP R8.30.00.7 >Date: Thu, 07 Dec 2000 16:39:40 -0500 >To: <ckolar(a)admin.aurora.edu> >Subject: Mailman > > >Mr. Kolar, > >Sorry to bother you with this question, but I've not received any responses to >my question on the list. I really appreciate any help you could give me >on this >problem. > >In a nutshell - The federal judiciary uses Cc:Mail as it's enterprise email >system (soon to be replaced by Notes). The Mailman lists that I've >created and >administer work great to those clients. However, they have no indication >in the >body of the message of who sent the message. > >This means that unless the sender remembers to sign there note, it essentially >becomes anonymous. > >Is there an environment variable that I can insert into the custom header that >will cause the sender's name to be displayed? > >Thanks! > >Pat Healy >U.S. District Court, Western District of New York

1 0

New checkins coming
by barry＠digicool.com 07 Dec '00

07 Dec '00

I'm going to start checking in a bunch of new changes, first to clean up some of the code by using Python 2.0 features (most notably string methods), and then by integrating Juan Carlos's i18n patches. This means that if you plan on using the cvs snapshot you will have to be running Python 2.0. Second, there are no guarantees of stability for a little while until the i18n changes stabilize. For now, stick with Mailman 2.0. If necessary we'll create a cvs branch for 2.0.1, but I only want to do that if we have to. -Barry

1 0

Huge qrunner process
by sigma＠pair.com 07 Dec '00

07 Dec '00

Can anyone enlighten me about why the qrunner process might need a tremendous amount of memory? Running Mailman 2.0 on a FreeBSD 4.1.1-STABLE server with Python 2.0. There are about 1500 lists, but the qfiles directory only has 32 files in it. Nonetheless, each minute when qrunner runs, it looks like this in top: 59606 mailman -2 20 135M 74456K getblk 0:01 8.93% 3.52% python 135 MB? It seems excessive. Any insight would be appreciated :) Thanks, Kevin

1 0

Monthly reminder Errors-To etc
by John A. Martin 01 Dec '00

01 Dec '00

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It must be my mistake because otherwise there would be a hue and cry by now if all Mailmans (Mailmen? :-)) were doing the same. My Mailman-1.1 sent monthly reminders thus - -------------- cut here ---->8 ---< head Return-Path: <mailman-owner(a)venice.essential.org> Delivered-To: jam(a)jamux.com Received: from milan.essential.org (milan.essential.org [216.0.124.12]) by athene.jamux.com (Postfix) with ESMTP id 1AD3448031 for <jam(a)jamux.com>; Wed, 1 Nov 2000 05:01:39 -0500 (EST) Received: from venice.essential.org (venice.essential.org [216.0.124.17]) by milan.essential.org (8.9.3/8.9.3) with ESMTP id FAA06070 for <jam(a)essential.org>; Wed, 1 Nov 2000 05:01:38 -0500 Received: from venice.essential.org (localhost [127.0.0.1]) by venice.essential.org (Postfix) with ESMTP id DC46429B60 for <jam(a)essential.org>; Wed, 1 Nov 2000 05:01:37 -0500 (EST) From: mailman-owner(a)venice.essential.org Subject: lists.essential.org mailing list memberships reminder To: jam(a)essential.org X-No-Archive: yes Precedence: bulk X-Mailman-Version: 1.1 Precedence: bulk List-Id: Promoting new uses of agricultural residue... <ag-res.lists.essential.org> Message-Id: <20001101100137.DC46429B60(a)venice.essential.org> Date: Wed, 1 Nov 2000 05:01:37 -0500 (EST) - ---- 8<------- cut here ----------> tail the bogus "List-Id being a known problem creating a minor annoiance. Now my shinny new Mailman-2.0 sent them thus[1] - -------------- cut here ---->8 ---< head Return-Path: <ababa-admin(a)venice.essential.org> Delivered-To: jam(a)jamux.com Received: from milan.essential.org (milan.essential.org [216.0.124.12]) by athene.jamux.com (Postfix) with ESMTP id 728B14800B for <jam(a)jamux.com>; Fri, 1 Dec 2000 05:02:06 -0500 (EST) Received: from venice.essential.org (venice.essential.org [216.0.124.17]) by milan.essential.org (8.9.3/8.9.3) with ESMTP id FAA29820 for <jam(a)essential.org>; Fri, 1 Dec 2000 05:02:05 -0500 Received: from venice.essential.org (localhost [127.0.0.1]) by venice.essential.org (Postfix) with ESMTP id 9418729BA8 for <jam(a)essential.org>; Fri, 1 Dec 2000 05:02:05 -0500 (EST) Subject: lists.essential.org mailing list memberships reminder From: mailman-owner(a)venice.essential.org To: jam(a)essential.org X-No-Archive: yes X-Ack: no Sender: ababa-admin(a)venice.essential.org Errors-To: ababa-admin(a)venice.essential.org X-BeenThere: ababa(a)lists.essential.org Precedence: bulk Message-Id: <20001201100205.9418729BA8(a)venice.essential.org> Date: Fri, 1 Dec 2000 05:02:05 -0500 (EST) - ---- 8<------- cut here ----------> tail with the bogus envelope sender, "Errors-To" and header-sender presumably causing <ababa-admin(a)venice.essential.org> to be inundated with delivery error notices. Is this an error that has crept into my Mailman configuration, or is this what Mailman will do until the item on the To Do list is resolved? I am making an unadvertised aaaaa-list hoping to shield <ababa-admin(a)venice.essential.org> from a repeat performance. Will this work? ALSO, when the problem of the monthly reminders is worked upon I urge again for a X-Addressed-To or some such header field to help understand those delivery error notifications from AOL and many universities that, correctly IMHO, attempt to reduce the exposure of potentially private information by excluding the original message body from delivery error notices. Add recipient address munging and without some counter measure such a delivery error notice is useless. jam Footnotes: [1] To protect the innocent, I have replaced the name of the alphabetically first mailing list on this Mailman with a fictions list name. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: OpenPGP encrypted mail preferred. See <http://www.gnupg.org/> iEYEARECAAYFAjonsgAACgkQUEvv1b/iXy8c0ACfZNqc1MiPgJdMkMoZM3fRRLM1 iZ0AniKuBuz+GswVatO8iz6vi9+eqx2n =uD/d -----END PGP SIGNATURE-----

1 0