[Mailman-Developers] Faulty Member Subscribe/Unsubscribes

[Mark Sapiro]

On 9/29/2011 8:58 AM, Andrew Case wrote:
> 
> When I did this I saw that maybe 1-5% of the time a user was still omitted
> from the list (user was silently removed).  I think that because these are
> processed by the queue runner on a different host and because the
> timestamp check is being done on an NFS stored file, there is potential
> that the qrunner for this doesn't yet have an updated mtime for that file
> (or even a small ntp time drift could cause this).  When I commented out
> the caching part of the code in MailList.py this bug never seems to show
> up:
>             #if mtime < self.__timestamp:
>             #    # File is not newer
>             #    return None, None


Actually, that is not the cache. It is just the test for whether the
current list object, cached or whatever, needs to be reloaded from disk.

I think that your configuration with NFS and possible time jitter
between servers makes the bug more likely.


> So I think there may still be a race condition here, but the chances of it
> are unlikely that human interaction would trigger this.  If however, you
> have a script that is subscribing users (one after another), this could
> still come up.  I actually happen to have such a script, but I run it on
> the same host as the qrunners, so I haven't experienced this before.


It can happen even where everything is on a single host, but as I said,
I think your configuration makes it more likely.


> In my case I think it's probably not worth keeping the performance gain
> that the caching adds for sake of consistency.


Attached is a patch to remove list caching from the qrunners. This patch
has the additional advantage of limiting the growth of the qrunners over
time. Old entries were supposed to be freed from the cache, but a self
reference in the default MemberAdaptor prevented this from occurring.

For reasons of trying not to be disruptive this patch and the bug fix I
sent earlier were never applied to the 2.1 branch. I think this was a
mistake, and I will apply them for Mailman 2.1.15.


> Attached is the modified stress test I'm using.


Thanks.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: disable_cache.patch
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20110929/43bad8ad/attachment.ksh>