Mailman 3 April 2000 - Mailman-Developers

Re: [Mailman-Developers] security szenario possible?
by Harald Meland 09 Apr '00

09 Apr '00

[Adrian Letzner - Sun Germany Berlin SE] > i would like to know, if mailman can handle the following szenario: > > 1. automatic subscribing to a fixed list (eg. via cgi script). that > means: > a program (eg. cgi-script) should handle the subscribing/unsubscribing > mechanism by sending a static mail to the *-request address WITHOUT (!!) > using the password mechanism (new privacy option: *not confirming). In fact, that does not constitute a new privacy option -- if you put ALLOW_OPEN_SUBSCRIBE = 1 in your ~mailman/Mailman/mm_cfg.py, a fourth option "none" should magically appear for "What steps are required for subscription?" on all your lists Privacy Options pages. Note that this allows _any_ of the lists in your Mailman installation to use the open subscribes option, and that is not necessarily a good thing (in that it allows anyone to subscribe unsuspecting others to your lists against their will). > 2. deleting the mail-header to anonymisize the mails which will be > posted. Doesn't the Hide the sender of a message, replacing it with the list address (Removes From, Sender and Reply-To fields) option on the bottom of the Privacy Options page do this? -- Harald

1 0

Re: [Mailman-Developers] Answering my own question
by Harald Meland 09 Apr '00

09 Apr '00

[Dan Mick] > Adding a "save_list=0" (or just a fourth argument '0') to the two > calls to SetUserOption in admin.py speeds up that "set one option" > flag a *whole* lot. I've got to stop answering emails sequentially ;) > It might be that similar "optimizations" to the other forms > processing (DeleteMember, primarily) are a bit unsafe, but it seems > like a huge win for the SetUserOption calls, and not very unsafe, > since the list will be saved after processing all users anyway. The SetUserOption() calls are being made even if there have been no changes, while the other MailList methods are only called if something actually has changed. I think that fact justifies putting off optimizing this further, at least for now. -- Harald

1 0

Re: [Mailman-Developers] Speaking of pathological behavior...
by Harald Meland 09 Apr '00

09 Apr '00

[Dan Mick] > 1) the archive links are remade, even though they exist (probably not a big deal) > 2) config.db.tmp is written, and then .db becomes .last and .tmp becomes .db. Thanks for reporting, I think the change I just checked into CVS should fix this (admin.py was calling MailList.SetUserOption() without the keyword argument "save_list=0", causing the list to be Save()d at least admin_user_chunksize * len(("hide", "nomail", "ack", "notmetoo", "plain")) times(!) every time the list's Membership Management page was visited). > Probably batching up all the "changes" into one config.db write is a > big big win. The bigger a admin_user_chunksize you're using, the bigger this win will be :) -- Harald

1 0

Re: [Mailman-Developers] membership managent web interface
by Harald Meland 09 Apr '00

09 Apr '00

[Ricardo Kustner] > But most of the times you know the exact address you want to unsubscribe > or view/edit, so why not add a search box on top? Can't this already be done by using the list member's "options" page (using the user's address in combination with your list admin password)? > I even think that only a searchbox on the initial page should be > enough, and only show the paged list of members when the user > specifically asks for it... saves a bit of bandwidth / response > time. any thoughts about this? I disagree. The list admin interface is cryptic enough as it is, IMO. Further obscuring the overview of list members wouldn't be nice. -- Harald

2 1

observations from new user
by Brian Grossman 09 Apr '00

09 Apr '00

I'm evaluating mailman to replace my majordomo v1 installation. Looks good so far. Nice work. One general observation: Why does DEFAULT_PRIVATE_ROSTER default to 0? I would think that 2 would be a better choice. I'm using this with qmail. 2.0beta2 seems to work well with qmail 1.03. Where should I send a script that eliminates the need for five .qmail files per list? This one is called from .qmail-default. I'm not subscribed to the list, so please cc me on replies. Brian

2 1

Re: [Mailman-Developers] Listing Lists Faster in 2.0?
by Harald Meland 08 Apr '00

08 Apr '00

[Roberto Ullfig] > Roberto Ullfig wrote: > > > > So, in 1.0rc2, displaying the list of lists for 529 lists > > requires 529**2 = 279841 system stat calls and takes over one > > and a half minutes on our Ultra-2 2x296 processor system! Is > > this because of Python, Mailman, or both? Has this been "fixed" > > in 2.0? You really should only need to make one stat call per > > list. > > > > Whatever the answer is, we'd like to be able to generate the > lists of lists once a day; Just a quick note: In the (severely hacked-up) Mailman installation we have here at uio.no (with ~3500 lists, although these are spread out over ~150 virtual domains), I've implemented caching of the data Mailman needs to generate the "list overview" pages, thereby avoiding excessive stat()ing. This is one of the features I'm holding off committing until after 2.0 is out. -- Harald

1 0

Re: [Mailman-Developers] User list in mysql
by Ricardo Kustner 07 Apr '00

07 Apr '00

On Thu, Apr 06, 2000 at 06:25:18PM -0700, Dan Chen wrote: > I've taken a look at MailList.py and understand that the subscriber list and > all other data is stored as a marshal in config.db. And this is stored in > the methods Load and Save. > I'm proposing that when Save is called, it also makes changes to the mysql > db. And after a Load, it queries the db. This sounds very expensive, and yes I'd love to see mailman support mysql too...but I think the best thing to do is moving the userlist to a seperate marshal db and abstracting the access code to make it easy to replace it with your own favorite sql server. from what i've seen in the mailman code is that there have to be made a lot of changes to support something like this... so maybe we'll have to wait till 2.0 gets released and work on it from there. in short, these are my thoughts on how this could be handled: 1) rewrite mailman to modify/read the user list through single function calls 2) create a module with those functions 3) create a module that is used to write the userlist to a marshal db 4) create a module to write the userlist to a mysql db 5) create a module to write the userlist to ... etc Ricardo. --

1 0

One more thing about bin/update...
by Barry A. Warsaw 07 Apr '00

07 Apr '00

... What do you think about running it by default on a "make install"? With the gate_news changes, you'll really want to be sure to run it. I guess this is another case of helping lazy users who don't read or follow directions. Am I trying to pamper them too much? -Barry

1 0

Re: [Mailman-Developers] race condition in locking ?
by Harald Meland 07 Apr '00

07 Apr '00

[Thomas Wouters] > On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote: > > > My current CVS checkout is available ("live" :) under <URL: > > http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with <URL: > > http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py> > > being of special interest in this discussion... > > Ah, very cool. You'll find my current version of LockFile.py > attached, for reference, as you asked. Thanks, I've skimmed through most of it now. > I think the link() solutions, as opposed to the rename() solution, > is a better one, if you'll premit me. Sure :) > For one, you can check on lock ownership by statting your own > lockfile, which should only be touched by you, instead of reading a > possibly highly volatile lockfile. You can also doublecheck the > lockfile inode, see if it is equal to your own lockfiles' inode, to > confirm that you have the lock. ... the downside being that if, e.g., anyone tries to read the info in the lockfile in the middle of a refresh(), they'll get a ValueError -- which your module seems to take care of, but it all looks a bit convoluted to my eyes. I especially dislike that lock() has to call locked() _after_ the check on ST_NLINK, because further down in lock() there's code that might break a lock it really shouldn't (if it weren't for the nonatomic-write-of-lock-contents issue). I also see some bugs that are fixed in my module but still present in yours, but those are rather irrelevant to this discussion (as they can easily be ported to your module if that's the one we'll go for). Does anyone but Thomas and me have any opinions on which locking scheme is preferrable? I guess I don't have any very strong feelings about the issue ;), so any sagely advise on the matter would be welcome. [ I've tried to model my version of the module as closely as possible after the way Exim does its lockfile locking, the idea being that Exim really should have been around long enough to have solved these things correctly. Reading the longish comment from ca. line 1206 and onwards in src/transports/appendfile.c of the Exim distribution proved very helpful to me. Of course, when it comes to lock stealing and other ugly (but needed, I guess) stuff, Exim couldn't provide much help. ] > Ah, well, I couldn't find any parts of Mailman in the CVS tree that > acually _called_ steal, so I couldn't figure out what it should do > in borderline cases. The only place I know of is cron/gate_news. > However, reading your LockFile.py made me realise how to steal the > lock. LockFile.steal() now does its utmost best to steal the lock, > only failing if someone else is busy stealing the lock from us. It > returns 1 for success, 0 for failure, mostly because I wanted it to > signify that in some way. ... but the caller should note that steal() returning 1 does not necessarily imply that it is the owner of the lock (anymore). The steal() stuff is bound to be full of race conditions, AFAICT -- so whether it returns anything doesn't really matter (IMHO), as the return code truly can't be trusted anyway. Cheers, -- Harald

2 1

Re: [Mailman-Developers] cron/gate_news watermark == 0
by Harald Meland 07 Apr '00

07 Apr '00

[Jim Tittsler] > A watermark value of 0 is used for two different purposes in cron/gate_news: > 1. the group has never been gatewayed, so "catch up" to the highest article > number > 2. the highest article number seen for the group is 0 (which can be a valid > "high article number" before the first posting in a newly created > newsgroup) > The result of this is the first article posted to the newly created newsgroup > fails to get gatewayed to the mailing list because the watermark for that > group is (still) 0 which is used as a flag to "catch up" so all that happens > is the watermark gets set to 1. > > I believe I have fixed that in my copy by using the value None for the first > time a group is gatewayed, allowing proper processing when the watermark is > truly article number 0. Hmmmm... When checking the contents of my "data/gate_watermarks" marshal, I find several old mailing lists with an entry stating that their watermark is 0 -- even though these lists aren't actually gating any newsgroups. As all these lists are rather old, I guess this could be how things were stored in some early Mailman version -- back when gate_news were forking processes to gate _all_ lists, regardless of whether the list in question had requested that any gating should be done, and the watermark file was written to by each and every child... On checking this with CVS, it appears that revision 1.7 of gate_news indeed has the behaviour I suspected, while 1.8, which was checked in 1998/12/18 00:22:23, seems to do it "right". The upshot of this is that any list touch by rev. 1.7 (and maybe earlier) which do not do any gating would possibly get *all* the messages present in the group it's gating from posted to it if it started gating after your patch was applied. A possible way to approach all this would be to have "make update" replace those 0 values in gate_watermarks with None values iff there are no other None values (i.e. only change things on the first time "make update" is run). Or, even better, move the per-list watermark info from gate_watermarks into the list's config.db, replacing any 0 values in gate_watermarks with None values in config.db. This has the advantage of reducing the number of different files that would have to be changed when cloning/renaming a list. -- Harald

3 2