Mailman 3 July 2002 - Mailman-Developers

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Wed, 17 Jul 2002 11:27:27 -0400 Jay R Ashworth <jra(a)baylink.com> wrote: > On Tue, Jul 16, 2002 at 10:47:55PM -0700, J C Lawrence wrote: >> On Tue, 16 Jul 2002 22:34:51 -0700 Chuq Von Rospach >> Is that really a valid and interesting case that can't be (usually) >> handled by posting directly to the list? > Absolutely, positively, 100% guaranteed. > Think Google. Think 5-year old message with *almost* exactly the > answer you need. Think "hasn't been on that mailing list in 2 years, > since a bogus bounce dropped me and I didn't bother re-upping." Yup, which is a small enough probability of happening, with a small enough population who might even/ever want to do that, with a small enough probability of the original address even remaining valid over that time...etc that I don't see it as much of an interesting case. The (nicer) thing is that it also handles given a CGI which does reply-to-list from the web archives. ObNote: I have that feature on the archives at Kanga.Nu. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 10:58:00 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > First, a minor announcement. I'm no longer in charge of the mailing > lists at apple, sort of. We've hired a person full-time, and he's been > taking over the lists server as his full-time responsibility, allowing > me to go off and work on other projects. I'm still in the loop, just > not "it". I'm still going to be heavily involved as we move that box > to Mailman 2.1, and after that, probably fade a bit more into the > woodwork (I still run my Mailman box at home, however, so I'm not > going away. JC, quite jeering) <sob> > ... to do what we can to make sure people who post to the mailing > lists don't get harvested. You realise that subscribed robots on lists are going to be the next SPAMmer trick? Of course they're easily defeated with an enforced moderated first post, but 'net-wide that's going to be a painful evolution. > A secondary issue here is the problem of disclosing admins and admin > addresses. I know we've hashed that through once, but we've come to > the (somewhat reluctant) decision to whitelist all public, > non-personal email addresses. We're going to be implementing TMDA to > do this... Actually it got very little discussion beyond the commentary that many/most of the valid mail to -owner and -admin is from member's non-subscribed addresses (even more likely given plus addressing). TMDA seems to work well tho for an -owner and -admin filter. > ... and will be switching all admin to generic addresses that filter > through TMDA, as well as things like postmaster@ and the like. Good idea. I've not moved over postmaster and webmaster. > I'm going to look and see if I can interface TMDA to the subscriber > databases so that subscribers are by definition whitelisted... It already can plug into Mailman's member roster in config.db. Generically extending that to plug, say, into an externally abstracted interface shouldn't be too difficult. What I've been poking at here is having TMDA call an external tool, passing it the address to be verified along with the To: address. The external tool would then do whatever (eg SQL query, LDAP lookup, poke into config.db, etc) and exit with an appropriate return code. Should work fairly well, especially for those cases where the authentication data is not local. > ... but we've hit the poiint where we have to do this. I'm not happy > about it, but the war is lost, I think. Or even close to being over. > And speaking of privacy, harvesting and spamming, a new and disturbing > thing happened this weekend that I want to bring up -- one for which I > have lots of questions, but no real answers. ... > So what he did was open up his address book and send his message to > everyone in it. And he's running one of these new e-mail clients that > happily caches addresses it sees in case you want them again. So all > of the addresses of people posting to the mailing lists he subscribed > to were in his address book cache, so when he grabbed his address > book, he grabbed all of those addresses, too. Yeah, I've seen this happen a couple times now. Typically it gets made into a public decapitation and all onlookers learn the DO NOT DO THAT lesson (or at least seem to). > So we have a clear violation of our anti-harvesting rules -- yet he > didn't overtly harvest. He just grabbed what was in his address book > at the time. Note: Many MUAs do this sort of address collection outside of Outlook. Heck, even the exmh I use can do collect addresses that way (and does by default IIRC) > This creates a major privacy quagmire. How do you set up rules for > something like that? You don't. You can't stop it technically so you have to rely on social engineering and a public pillory with frequent enough public humiliations that it starts entering the public unconscious. > Where does ownership and protection end? I draw the line at the edge of my MTA. Once you send a message to one my lists and it gets broadcast all rules and bets are off. I'll take reasonable care and due diligence within my server, but outside of that I don't attempt to control any more than I attempt to control the persistently rude (whom I normally unsubscribe, but that's another matter). > I just don't know how to deal with the issues this address caching > causes. I'm firmly of the mind that we can't. Its intrinsically an education issue. Now we can be effective in helping educate, but until is enters the social consciousness we'll be fighting up hill. > Ultimately, we're going to have to rethink our "no harvesting" rules, > and likely also write disclaimers explaining what our limits are. <nod> One possible technical approach is the mask all From: and CC: addresses on messages broadcast from a list with date limited plus addresses which reverse map on the list server back to the original address. Its no sort of permanent solution, it doesn't handle addresses quoted in .sigs, message bodies, or other odd/arbitrary headers, but it can do a whole lot to cripple address harvesting. <ponder> Actually, I rather like this idea for certain lists. > We've actually considered switching our lists to obscured addresses, > turned that down as being worse than the disease (for now). You mean like the above? > But now we're wondering if we have to go to some sort of address > cloaking ON lists, maybe some kind of address remapping through the > server for replies, something. And I'm gritting my teeth at the > developers who created those @#$@$#@$#23 caches (which are nice in > some ways) for not also creating some way to flag addresses as not > cacheable. Because, IMHO, that'd solve this problem. Note that making a fixed and persistent address mapping really doesn't handle anything. You've just created another alias which works just as well for the SPAMmers or address harvesters and you can't (server side) distinguish between valid or SPAM mail sent to it any more successfully than you can for normal mail. Well, unless you setup TMDA-style filters for every such mapped plus address, which, asides from the cache expense really isn't such a bad idea. Hurm. I kinda like it actually. Date limiting the validity of the mapped address has two pleasant effects: It limits the size of your database of addresses, and it limits the window of opportunity for abuse of the address. > But they didn't. Grumble. Note that such address collection has been around a long time. exmh is by no means a particularly new or bleeding edge MUA and its had this sort of address collection feature for as long as I've used it. > I think it's an issue we have to come to grips with. I only see two addresses: 1) A pillory 2) Dynamic and/or filtered address mapping as above. Neither are particularly pleasant. Address mapping breaks as soon as Outlook (for instance) starts scanning message bodies for addresses to cache. > Are we hitting a point where mail list servers have to act as blind > front ends for all of the subscribers, where replies are processed by > those servers, and the server then takes on the job of acting as a > troll-exterminator and spam blocker? Yes, not yet, but soon. > And what does that really mean for things like Mailman? It means that Mailman will have to have a plug-in layer to do the appropriate processing. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

3 2

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 23:15:24 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 10:59 PM, "J C Lawrence" <claw(a)kanga.nu> wrote: >> How about the CGI business as a solution? You can do that without >> exposing the address and while doing due diligence on constraining >> abuse. > Once you build the infrastructure to support cloaked-but-forwarding > addresses, it's definitely possible. But not 1st generation stuff, I > don't think. True. Its a pretty clean v2 tho, and with a short runway between v1 and v2. >> ObNote: Apple runs a large enough mail ship that its an attractive >> target in and of itself should you do an automated system like this. > Shh. If you don't tell anyone... (oh, wait, security through obscurity > is a bad idea...) There are many problems which become clearly non-linear purely due to the Big Number effect. SPAM is one of those problems on all three ends of the equation. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 22:50:25 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: >>> What happens when someone sees an address in the archive six months >>> from now and needs to contact that author? >> Is that really a valid and interesting case that can't be (usually) >> handled by posting directly to the list? > No, but it's something that happens often enough that I have to worry > about it, and not often enough that I consider it a showstopper. <nod> Different loads. How about the CGI business as a solution? You can do that without exposing the address and while doing due diligence on constraining abuse. ObNote: Apple runs a large enough mail ship that its an attractive target in and of itself should you do an automated system like this. *THAT* is painful as you can't insert enough entropy into the UI behavior without becoming either mappable or unusable -- which basically means you end up doing something like a TMDA filter which holds the message for the final recipient for a week or so...which makes havoc of your storage and bandwidth budgets... -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

2 1

Re: [Mailman-Developers] Re: multiple addresses in (was AOL and odd) Reply-to
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 22:57:44 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 10:36 PM, "J C Lawrence" <claw(a)kanga.nu> wrote: >> Nick Simicich <njs(a)scifi.squawk.com> asserts that AOL will silently >> discard mail which lists multiple addresses in Reply-To:, one of >> which matches From: and is an AOL address. Later in that thread: > AOL silently blackholes stuff that trips its spam filters. > I can easily see, given the recent spate of spam with forged headers > to look like it's coming FROM someone (or you) @ your domain to > someone (or you) @ your domain, that comes from off-site, that AOL has > decided that anything that looks to be from an AOL account but coming > from outside the AOL universe is probably spam. Because, probably, it > is. If, as has been suggested, AOL users can't normally set Reply-To:, such a check on Reply-To is also quite valid (or damned close enough for AOL's uses). > It's basically hit a point where it's unsafe to accept mail "from" > your domain unless it's explicitly coming from a trusted SMTP > source. And yes, that screws over someone (like me) who owns a domain > on a box somewhere, and who goes on the road and uses a dialup like > earthlink and wants to continue using his real domain in his > email. Do *NOT* talk to me about POP before SMTP. I repeat, DO NOT... <Atrociously horrible hack> > But the spammers are now exploiting that, so you're going to see > everyone lock it down tight, both on the receiving side, and (with > responsible ISPs) the sending side (by limiting what domain names > they'll propogate outward) Bingo. I've already started seeing this. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 22:35:50 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 10:31 PM, "J C Lawrence" <claw(a)kanga.nu> wrote: >>> Hmmm. Can you predict TMDA's response for scripting? >> Yes, currently there is precious little entropy insertion. > (insert filthy expletive involving sheep and the second amendment) Ahem. Yeah, but. TMDA not only gets most of the way there, it also builds and provides most of the framework needed to get there. That's not a small thing, and shouldn't be an ignored thing. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 17:07:48 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 2:37 PM, "Barry A. Warsaw" <barry(a)zope.com> wrote: > One of the big things I dislike about Mhonarc is that archives are a > rather low-usage system, but maintaining the Mhonarc index pages is > rather intensive use of system resources. Rather. I've been messing with my current archive setup having the current PHP-based pages of variable assignments post-processed into SQL INSERT statements. A CGI then reads the SQL DB and builds the thread tree dynamically on display. It works fairly well. Not great, but it works. Handling the thread tree elegantly when In-Reply-To and References are bad is non-trivial. ObNote: cacheing the thread tree would be smart but is not that easy. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

2 1

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 22:18:11 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 9:36 PM, "J C Lawrence" <claw(a)kanga.nu> wrote: >> Should work fairly well, especially for those cases where the >> authentication data is not local. > Which would, actually, allow a user to generate an authentication via > a web site that stores into a MySQL database, allowing you to extend > TMDA to use a clickurl instead of a reply token thingie. Damned good idea. > I wonder when the spambots will start scripting TMDA? Once it hits a thousandth of a percentage point. > It'd require bulletproof or temporary/disposable return addresses, > but... An easy hack. > Hmmm. Can you predict TMDA's response for scripting? Yes, currently there is precious little entropy insertion. The more simple problem (and easy to fix) is that TMDA and like systems rely on plus addressing and the base form of the address continues to deliver directly... -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

2 1

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 21:26:57 -0400 bob <Puff@NLE" <bob(a)nleaudio.com>> wrote: > Exactly. I didn't state that, but you put it well. But who cares if > it's the primary MX or not? If it hits any of my MXs, it will be > delivered. You can configure your MXes to require that HELO and From: must agree (at least within the same AS range), and to auto-reject/bounce messages that don't. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0

Re: [Mailman-Developers] Opening up a few can o' worms here...
by J C Lawrence 17 Jul '02

17 Jul '02

On Tue, 16 Jul 2002 17:28:07 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 7/16/02 5:35 PM, "Bob Puff@NLE" <bob(a)nleaudio.com> wrote: >> I've seen the next generation of spammer software at work recently. >> Spammer's machine makes direct SMTP connection to my box > Actually, the REAL state of the art is that they look up your MX > records, and do this to the HIGHEST ranked one (not the lowest). Yup. I first started seeing this around March of last year and started mentioning that fact here and on the various list-manager lists at that time. > This is on the (it turns out, quite valid) assumption that it won't be > spamblocked as well as the main MX relay is, but will be validated to > forward stuff in to you. And where they're trying that, we're finding > it works (grumble grr) damn well. You're not alone. Currently some 30% of my SPAM comes thru backup MXes. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw(a)kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live.

1 0