Mailman 3 May 2001 - Mailman-Developers

Re: [Mailman-Users] Big problems with stale lockfiles on large list...
by barry＠digicool.com May 3, 2001

May 3, 2001

I'm CC'ing Mailman developers because I want to get some feedback from the Unix hackers amonst us; apologies for duplicates... >> We also experienced similar problems in the past with large >> lists. We found that if the admin is accessing the long and >> slow-loading members admin pages, and does not wait for the >> page to complete (e.g. clicks one of the admin links or hits >> refresh or stop) the lock will remain. As a temporary solution >> we have instructed our admins to wait for all pages to >> completely load, no stale locks since then. This is on a Cobalt >> RaQ4i (Redhat) with Apache 1.3x. Hope this helps, >>>>> "GT" == Graham TerMarsch <mailman(a)howlingfrog.com> writes: GT> Is probably related to what we're experiencing, but I'm GT> finding that even just due to the volume of hits on the GT> "subscribe" page, that we're having this problem. While GT> testing, I made sure that we had _no_ accesses to admin pages, GT> and that all of the "subscribe" hits that were coming through GT> waited for the complete response and didn't time out or have GT> the "stop" button pressed. Even in this scenario, I still GT> ended up with stale locks lingering around. [...] GT> Would it be correct to say that if the CGI process dies for GT> some unforseen reason (e.g. Apache kills it off because the GT> user pressed the "stop" button or the HTTP connection timed GT> out), that the lock from that process gets left around as a GT> lingering lock? After an all night hacking jag, lots of google searches, a re-read of Stevens Ch.10, and a short conversation with Guido, I believe I know what's going on. That's the good news. The bad news is that I don't think there's a perfect solution, but there probably is a better solution. FTR, here's my development environment: Apache 1.3.19, Python 2.1, RH6.2-ish, kernel 2.2.18, NS 4.74, and the current Mailman 2.1 CVS snapshot. Here's what I did: I loaded up a list with 60000 subscribers, then went to the members page. It did indeed take a long time, and if I let it run to completion, I get the page as expected and no locks. However, if I hit the stop button before the page is finished loading, I can see that the CGI process continues to run for a while and then it may or may not clear the locks. The page is not complete. Since sometimes the locks are cleared and sometimes they're left, it's pretty clear there are race conditions involved. I did a bit of web searching and here's what I think a stock Apache mod_cgi is supposed to do in this situation: - The user hits the stop button on the browser. The client will close the socket, which is usually eventually recognized by the server. - The cgi script meanwhile is merrily doing some long calculations, and eventually it will try to write output. It's at this point that it may be possible to detect that the socket has gone away. What's interesting is that it appears that the cgi script actually writes output to the parent Apache process, which perhaps buffers the output before sending to the client. In any event, it's the Apache parent process that gets a SIGPIPE indicating the socket it's writing to is closed. - Apache catches the SIGPIPE, turns around and sends a SIGTERM to the cgi process. Apache waits three seconds and if the cgi hasn't exited yet, it sends a SIGKILL. The default behavior for SIGTERM is to terminate the process, so whether it's the SIGTERM or SIGKILL that does the dirty work, in the end, the naive cgi script can get summarily killed without a chance at clean up. Enter Python. Python installs a few of its own signal handlers, most notably for SIGINT which it maps to the KeyboardInterrupt exception. Normally though, SIGTERM isn't caught by Python, so again, the naive Python cgi script will summarily die when receiving a SIGTERM. Under mod_cgi, even catching SIGTERM may not help if more than 3 seconds of processing occurs (I don't know if this is clock time or cpu time). Again, that's because Apache turns around and SIGKILLs the cgi, and that signal can't be caught (remember, this is what bit us a while back w.r.t. Postfix filter programs). So maybe the answer is to install a Python-level SIGTERM handler that will release the lock. To understand this, you need to know how Python handles signals. When you set a signal handler in Python, at the C level, Python installs its own signal handler, which simply sets a flag and returns. That's because Python will only run Python-level signal handlers in between bytecode boundaries. So your Python signal handler can get run at some arbitrary point in the future, and likely /not/ when the signal actually occurred. All this is very dodgy in the face of try/finally clauses, which is how cgi's like admin.py ensure that the mailing list is saved and unlocked when an exception occurs. For example, my initial solution was to install a signal handler that raised an exception, thinking that this would trigger the finally clause and do the normal save & unlock. But because of race conditions and interactions with the Python interpreter loop, I often saw that while the exception was getting raised, the finally clause wasn't always getting executed. Guido confirmed that this isn't a safe approach. So my next approach is to write a very minimal signal handler that only unlocks the list, and install this on SIGTERM. The trick is to make the signal handler, e.g. a nested function of main() in admin.py, and pass the MailList object into it using the default argument trick. This seems to work, in that the locks appear to be cleared in the cases where they were left laying around before. But because of all the race conditions, I can't be 100% sure. If you've read this far, the implication is that if the user hits the stop button, Mailman will in essence abort any changes to list configuration that this invocation may have made. Alternatively, we could try to save & unlock in the signal handler, but that raises the possibility of race conditions again. Also, it makes sense to move the save of the list data into the try: part of the clause and only do the unlocking in the finally. That way, the finally clause and the SIGTERM handler have the same semantics, and the list will get unlocked in the face of either an exception or a signal. But the list database will only get saved on sucessful completion of the task. I can live with those semantics (I think ;). So the code in admin.py looks something like: def main(): ... mlist = MailList.MailList(listname, lock=0) ... def sigterm_handler(signum, frame, mlist=mlist): mlist.Unlock() mlist.Lock() try: signal.signal(signal.SIGTERM, sigterm_handler) ... mlist.Save() finally: mlist.Unlock() I think there are still opportunities to get nailed, but I think this works as well as is possible. In my testing I've been able to clear the locks when the stop button is pushed. In the rare event that the list's config.db file gets corrupted (i.e. the handler gets called in the middle of mlist.Save()), we've always got the config.db.last file to fall back on. The other good news is that I think only admindb.py, admin.py, and confirm.py, need to be modified, since those are the only scripts that write to list attributes. I've been pretty diligent in Mailman 2.1 to only lock the mailing lists when necessary, so if the scripts are read-only, they don't acquire the lock (and get the state of the database as of the time of the initial Load()). Phew! -Barry

3 10

MHonArc integration
by Anand Kumria May 3, 2001

May 3, 2001

Hi folks, I've just setup another mailman site and once again I wanted to integrate it with MHonArc. Previously I'd been using shell script that played games with symlinks and the the archive files behind Mailman's back. I wanted something simpler to use, so after much judicious cutting and pasting of other code I've put together MHonArc.py which you can use as an almost compatible replacement for HyperArch.py There are a number of things left to do: - generate HTML index pages properly - store text versions of the archive along with each dir. - cleanup the code and document it This is my first real experience with Python, so no laughing (too loud). I think I've also noticed a bug where Mailman uses the udnerlying Pythong rfc822 object when Mailman's is more useful -- see comments in add_article. I've also simplified the date handling (the current archiving stuff hyperarch/pipermail convert a to/from unix epoch lots of times) Doing this also made clear that a nicer interface in Archive.py could be done which should handle, imo: - message date => volume (and thus path) conversions - make the archive date format local to each list e.g. I'd like to have 2001/05/01 instead of 20010501 - generation of HTML index pages - make the HTML template editable via the admin interface - expose some way to allow list-admins to configure any options the archiver might allow (MHonArc allows a lot) Feedback on all of this welcome. Regards, Anand

1 0

Re: [Mailman-Developers] case-sensitive e-mail addresses
by J C Lawrence May 2, 2001

May 2, 2001

On Wed, 2 May 2001 11:50:09 -0400 Jay R Ashworth <jra(a)baylink.com> wrote: > On Wed, May 02, 2001 at 08:43:17AM -0700, Chuq Von Rospach wrote: > But is this actually true? I haven't tried it, but I thought > sendmail still treated LHS's as case-sensitive... Its a config option. Ditto for Exim. I haven't looked at Postfix in this regard (and am sitting on a slow 28.8K link right now, so I'll leave that to the reader). <<Fervently praying for the DSL install in a few days>> -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] case-sensitive e-mail addresses
by J C Lawrence May 2, 2001

May 2, 2001

On Wed, 02 May 2001 08:26:59 -0700 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > On 5/2/01 8:05 AM, "Jay R. Ashworth" <jra(a)baylink.com> wrote: >> I'm afraid I must disagree: that section of 2821 implies that > Sorry, I disagree. "may be" means the standard allows you to > consider this as optional. That's good, because 821 made case > sensistivity mandatory (and the switch shows the clear intent of > the standards folks, which is moving to full case insensitivity, > but they must have felt completely removing it was too big a step > -- but they're telling anyone who might still have a case > sensitive mailer it's time to fix it) IIRC some of the old BitNet systems mandated case sensitive LHS and were not configurable in that regard. As a result the gateways would upcase everything for portability. Its questionable if any of them are still on the 'net, but that's another question. > In practice, Barry's approach is the best one. The only thing that > happens if you practice case sensitivity on the user part is that > users end up with multiple subscriptions, multiple copies and no > clue how to fix it or what's wrong (and it's your fault, your > mailer is screwed up). The principle of least surprise. People have enough problem with case sensitive passwords. -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] Re: Dates
by Owen Taylor May 2, 2001

May 2, 2001

David Champion <dgc(a)uchicago.edu> writes: > On 2001.05.01, in <20010501183541.06693(a)scfn.thpl.lib.fl.us>, > "Jay R. Ashworth" <jra(a)baylink.com> wrote: > > > correct. How would it possibly know? You're discussing ways to make a > > > correct guess in most cases, but not a way to ensure correct > > > information in all cases. > > > > Stipulated. But we're not rewriting for retransmission, only to keep > > the archives clean; right, Barry? > > That's what I understand, too. My personal taste is that an archive > reflect with complete accuracy and fidelity what was received from the > transmitter -- not the recipient's interpretation thereof. This is > only relevant to particulars of implementation, though, not to whether > the general solution is applied at all. For example, I think it's more > appropriate to leave the default at "don't munge incoming mail at > all". At the risk of prolonging this discussion, it may be worth pointing out that there are two different "dates" here: 1) The date in the Date: field displayed in the archive 2) The date used for sorting and for splitting archives by month/year. I'm of the opinion that 1) should always be the same date as generated by the MUA, but that using that date for 2) is, from experience, a bad idea, because some user's machines will have badly screwed up clocks, and you get archive volumes for ridiculous periods, etc. Currently, Mailman, currently, always uses the same date for both 1) and 2) - either the MUA date, or the date when the mail passed through mailman. From my perspective, arguing about which one of these is superior is somewhat pointless, since as soon as you use the same date for both, you are going to have problems. Regards, Owen

5 17

case-sensitive e-mail addresses
by SillyGod May 2, 2001

May 2, 2001

Hello All, Please, could someone advise me if there is still an issue with using case-sensitive e-mail addresses with mailman? ie. john.doe(a)email.com vs. John.Doe(a)email.com would both get delivered? Or would the case-incorrect version bounce? Thanks! *Larry*

5 15

Re: [Mailman-Developers] case-sensitive e-mail addresses
by Nigel Metheringham May 2, 2001

May 2, 2001

jra(a)baylink.com said: > But is this actually true? I haven't tried it, but I thought sendmail > still treated LHS's as case-sensitive... Depending on your rewrite rules, sendmail can be either case sensitive or case insensitive. Exim also has a choice:- > locally_caseless > Type: boolean > Default: true > Domains in mail addresses are specified as being case-independent, but > this it not true of local parts. For most Unix systems, however, it is > desirable that local parts of local mail addresses be treated in a > case-independent manner, since most users expect that mail to OBailey > and obailey, for example, will end up in the same mailbox. By default, > when it is processing an address whose domain is local, Exim > lower-cases the local part at the start of processing, on the > assumption that account names in the password file are in lower-case. > For installations that want to draw case distinctions, this option is > provided. When turned off, local local parts are handled verbatim > during delivery. If there are names containing upper case letters in > the password file, the most convenient way to provide for caseless > mail delivery is to set up a smartuser director as the first director, > and to make it do a lowercased lookup of the local part, in order to > translate it to the correctly cased version, using the new_address > option. There is a special place in hell reserved for those that use mixed case usernames on Unix and also expect caseless operation. Nigel. -- [ Nigel Metheringham Nigel.Metheringham(a)InTechnology.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ] [ - Comments in this message are my own and not ITO opinion/policy - ]

1 0

Re: [Mailman-Developers] Re: Dates
by J C Lawrence May 2, 2001

May 2, 2001

On Wed, 2 May 2001 00:19:15 -0400 Jay R Ashworth <jra(a)baylink.com> wrote: > On Wed, May 02, 2001 at 12:10:30AM -0400, Barry A. Warsaw wrote: JRA> And therefore, not the best approach, by concensus, apparently. JRA> Barry? Have I misunderstood you? >> IMO, the archiver ought to be doing any date clobbering, but >> Pipermail is too painful to fix. Again, what do other external >> archivers do? I.e. if you passed an outrageously dated message >> to MHonArc, would it dutifully put the message in the April 2006 >> slot? > Ok; that's what I thought I'd heard. That is what it does (and what I want). -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Re: [Mailman-Developers] Re: Dates
by J C Lawrence May 2, 2001

May 2, 2001

On Tue, 1 May 2001 17:21:29 -0500 David Champion <dgc(a)uchicago.edu> wrote: > But it's not the MLM's business to judge whether a Date: header is > correct. How would it possibly know? You're discussing ways to > make a correct guess in most cases, but not a way to ensure > correct information in all cases. There are two interesting facts about a message: 1) The message as the author wrote it. 2) When the message arrived at the list in terms of time and in terms of list sequence. Past that, its all quibbles. Altering either one destroys historical accuracy. Bad idea. -- J C Lawrence claw(a)kanga.nu ---------(*) http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

1 0

Dates
by Dan Mick May 1, 2001

May 1, 2001

Personally, I want the date in the Date header to be the one that shows up. I don't care when it got to the list server, finally; I care when it was sent. If the sender is too stupid to set up his machine, I want to know that, too.

3 10