[Adrian Letzner - Sun Germany Berlin SE]
> i would like to know, if mailman can handle the following szenario:
> 1. automatic subscribing to a fixed list (eg. via cgi script). that
> a program (eg. cgi-script) should handle the subscribing/unsubscribing
> mechanism by sending a static mail to the *-request address WITHOUT (!!)
> using the password mechanism (new privacy option: *not confirming).
In fact, that does not constitute a new privacy option -- if you put
ALLOW_OPEN_SUBSCRIBE = 1
in your ~mailman/Mailman/mm_cfg.py, a fourth option "none" should
magically appear for "What steps are required for subscription?" on
all your lists Privacy Options pages.
Note that this allows _any_ of the lists in your Mailman installation
to use the open subscribes option, and that is not necessarily a good
thing (in that it allows anyone to subscribe unsuspecting others to
your lists against their will).
> 2. deleting the mail-header to anonymisize the mails which will be
Hide the sender of a message, replacing it with the list address
(Removes From, Sender and Reply-To fields)
option on the bottom of the Privacy Options page do this?
> Adding a "save_list=0" (or just a fourth argument '0') to the two
> calls to SetUserOption in admin.py speeds up that "set one option"
> flag a *whole* lot.
I've got to stop answering emails sequentially ;)
> It might be that similar "optimizations" to the other forms
> processing (DeleteMember, primarily) are a bit unsafe, but it seems
> like a huge win for the SetUserOption calls, and not very unsafe,
> since the list will be saved after processing all users anyway.
The SetUserOption() calls are being made even if there have been no
changes, while the other MailList methods are only called if something
actually has changed. I think that fact justifies putting off
optimizing this further, at least for now.
> 1) the archive links are remade, even though they exist (probably not a big deal)
> 2) config.db.tmp is written, and then .db becomes .last and .tmp becomes .db.
Thanks for reporting, I think the change I just checked into CVS
should fix this (admin.py was calling MailList.SetUserOption() without
the keyword argument "save_list=0", causing the list to be Save()d at
admin_user_chunksize * len(("hide", "nomail", "ack", "notmetoo", "plain"))
times(!) every time the list's Membership Management page was
> Probably batching up all the "changes" into one config.db write is a
> big big win.
The bigger a admin_user_chunksize you're using, the bigger this win
will be :)
> But most of the times you know the exact address you want to unsubscribe
> or view/edit, so why not add a search box on top?
Can't this already be done by using the list member's "options" page
(using the user's address in combination with your list admin
> I even think that only a searchbox on the initial page should be
> enough, and only show the paged list of members when the user
> specifically asks for it... saves a bit of bandwidth / response
> time. any thoughts about this?
I disagree. The list admin interface is cryptic enough as it is, IMO.
Further obscuring the overview of list members wouldn't be nice.
I'm evaluating mailman to replace my majordomo v1 installation.
Looks good so far. Nice work.
One general observation: Why does DEFAULT_PRIVATE_ROSTER default to 0? I
would think that 2 would be a better choice.
I'm using this with qmail. 2.0beta2 seems to work well with qmail 1.03.
Where should I send a script that eliminates the need for five .qmail
files per list? This one is called from .qmail-default.
I'm not subscribed to the list, so please cc me on replies.
> Roberto Ullfig wrote:
> > So, in 1.0rc2, displaying the list of lists for 529 lists
> > requires 529**2 = 279841 system stat calls and takes over one
> > and a half minutes on our Ultra-2 2x296 processor system! Is
> > this because of Python, Mailman, or both? Has this been "fixed"
> > in 2.0? You really should only need to make one stat call per
> > list.
> Whatever the answer is, we'd like to be able to generate the
> lists of lists once a day;
Just a quick note: In the (severely hacked-up) Mailman installation we
have here at uio.no (with ~3500 lists, although these are spread out
over ~150 virtual domains), I've implemented caching of the data
Mailman needs to generate the "list overview" pages, thereby avoiding
This is one of the features I'm holding off committing until after 2.0
On Thu, Apr 06, 2000 at 06:25:18PM -0700, Dan Chen wrote:
> I've taken a look at MailList.py and understand that the subscriber list and
> all other data is stored as a marshal in config.db. And this is stored in
> the methods Load and Save.
> I'm proposing that when Save is called, it also makes changes to the mysql
> db. And after a Load, it queries the db. This sounds very expensive, and
yes I'd love to see mailman support mysql too...but I think the best thing
to do is moving the userlist to a seperate marshal db and abstracting the
access code to make it easy to replace it with your own favorite sql server.
from what i've seen in the mailman code is that there have to be made a lot
of changes to support something like this... so maybe we'll have to wait till
2.0 gets released and work on it from there.
in short, these are my thoughts on how this could be handled:
1) rewrite mailman to modify/read the user list through single function calls
2) create a module with those functions
3) create a module that is used to write the userlist to a marshal db
4) create a module to write the userlist to a mysql db
5) create a module to write the userlist to ... etc
... What do you think about running it by default on a "make install"?
With the gate_news changes, you'll really want to be sure to run it.
I guess this is another case of helping lazy users who don't read or
follow directions. Am I trying to pamper them too much?
> On Mon, Feb 07, 2000 at 10:01:36PM +0100, Harald Meland wrote:
> > My current CVS checkout is available ("live" :) under <URL:
> > http://www.uio.no/~hmeland/tmp/mailman-userdb/>, with <URL:
> > http://www.uio.no/~hmeland/tmp/mailman-userdb/Mailman/LockFile.py>
> > being of special interest in this discussion...
> Ah, very cool. You'll find my current version of LockFile.py
> attached, for reference, as you asked.
Thanks, I've skimmed through most of it now.
> I think the link() solutions, as opposed to the rename() solution,
> is a better one, if you'll premit me.
> For one, you can check on lock ownership by statting your own
> lockfile, which should only be touched by you, instead of reading a
> possibly highly volatile lockfile. You can also doublecheck the
> lockfile inode, see if it is equal to your own lockfiles' inode, to
> confirm that you have the lock.
... the downside being that if, e.g., anyone tries to read the info in
the lockfile in the middle of a refresh(), they'll get a ValueError --
which your module seems to take care of, but it all looks a bit
convoluted to my eyes. I especially dislike that lock() has to call
locked() _after_ the check on ST_NLINK, because further down in lock()
there's code that might break a lock it really shouldn't (if it
weren't for the nonatomic-write-of-lock-contents issue).
I also see some bugs that are fixed in my module but still present in
yours, but those are rather irrelevant to this discussion (as they can
easily be ported to your module if that's the one we'll go for).
Does anyone but Thomas and me have any opinions on which locking
scheme is preferrable? I guess I don't have any very strong feelings
about the issue ;), so any sagely advise on the matter would be
[ I've tried to model my version of the module as closely as possible
after the way Exim does its lockfile locking, the idea being that
Exim really should have been around long enough to have solved these
things correctly. Reading the longish comment from ca. line 1206
and onwards in src/transports/appendfile.c of the Exim distribution
proved very helpful to me.
Of course, when it comes to lock stealing and other ugly (but
needed, I guess) stuff, Exim couldn't provide much help. ]
> Ah, well, I couldn't find any parts of Mailman in the CVS tree that
> acually _called_ steal, so I couldn't figure out what it should do
> in borderline cases.
The only place I know of is cron/gate_news.
> However, reading your LockFile.py made me realise how to steal the
> lock. LockFile.steal() now does its utmost best to steal the lock,
> only failing if someone else is busy stealing the lock from us. It
> returns 1 for success, 0 for failure, mostly because I wanted it to
> signify that in some way.
... but the caller should note that steal() returning 1 does not
necessarily imply that it is the owner of the lock (anymore). The
steal() stuff is bound to be full of race conditions, AFAICT -- so
whether it returns anything doesn't really matter (IMHO), as the
return code truly can't be trusted anyway.
> A watermark value of 0 is used for two different purposes in cron/gate_news:
> 1. the group has never been gatewayed, so "catch up" to the highest article
> 2. the highest article number seen for the group is 0 (which can be a valid
> "high article number" before the first posting in a newly created
> The result of this is the first article posted to the newly created newsgroup
> fails to get gatewayed to the mailing list because the watermark for that
> group is (still) 0 which is used as a flag to "catch up" so all that happens
> is the watermark gets set to 1.
> I believe I have fixed that in my copy by using the value None for the first
> time a group is gatewayed, allowing proper processing when the watermark is
> truly article number 0.
Hmmmm... When checking the contents of my "data/gate_watermarks"
marshal, I find several old mailing lists with an entry stating that
their watermark is 0 -- even though these lists aren't actually gating
As all these lists are rather old, I guess this could be how things
were stored in some early Mailman version -- back when gate_news were
forking processes to gate _all_ lists, regardless of whether the list
in question had requested that any gating should be done, and the
watermark file was written to by each and every child...
On checking this with CVS, it appears that revision 1.7 of gate_news
indeed has the behaviour I suspected, while 1.8, which was checked in
1998/12/18 00:22:23, seems to do it "right".
The upshot of this is that any list touch by rev. 1.7 (and maybe
earlier) which do not do any gating would possibly get *all* the
messages present in the group it's gating from posted to it if it
started gating after your patch was applied.
A possible way to approach all this would be to have "make update"
replace those 0 values in gate_watermarks with None values iff there
are no other None values (i.e. only change things on the first time
"make update" is run).
Or, even better, move the per-list watermark info from gate_watermarks
into the list's config.db, replacing any 0 values in gate_watermarks
with None values in config.db. This has the advantage of reducing the
number of different files that would have to be changed when
cloning/renaming a list.