[Mailman-Developers] Memory pinned in ram, with huge lists

[Mark Sapiro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Sapiro wrote:
> Jesus Cea wrote:
> 
>> I changed the OldStyleMemberships constructor to:
> 
>> """
>> class OldStyleMemberships(MemberAdaptor.MemberAdaptor):
>>     def __init__(self, mlist):
>>         import weakref
>>         self.__mlist = weakref.proxy(mlist)
>> """
> 
>> to keep only a weak reference to the mailing list, breaking the cycle.
> 
> 
> Thanks very much for your efforts in debugging this.
> 
> 
>> 2. Now, since cache in evicted frequently, mailing list data must be
>> reloaded every time. This is a performance hit, but my mailing list are
>> huge but with little traffic (maybe a couple of mails per week), so this
>> is a non issue for me.
> 
> 
> The use of the cache has been changed for 2.2. See the full thread at
> <http://mail.python.org/pipermail/mailman-developers/2008-August/020329.html>
> for more information. In 2.2, the cache will be less effective anyway,
> and the impact doesn't seem too severe.
> 
> I am going to implement your change to OldStyleMemberships for 2.2. I'm
> almost inclined to drop the cache all together as I think with the 2.2
> logic, hits may be rare. In theory, the logic can avoid a second read of
> the pickle if the runner first instantiates the list unlocked and
> subsequently locks it, but I suspect this normally happens in the same
> clock second so the second read wouldn't be avoided anyway.


There is a problem with the suggested change. If we are running under
Python 2.6, the creation of the proxy

        self.__mlist = weakref.proxy(mlist)

Produces two of the following messages

Exception RuntimeError: 'maximum recursion depth exceeded while calling
a Python object' in <type 'exceptions.AttributeError'> ignored

These do not occur with Python 2.5.1. Presumably these are due to
something within the structure of the list object itself and possibly
render some parts of the list object unavailable to OldStyleMemberships
via the proxy object. So far, I haven't identified an operational
problem due to this, but because of this and the other considerations
mentioned above, I'm now inclined to just abandon the list cache in
Runner.py

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFJREeQVVuXXpU7hpMRAsXyAJ9mRNP2jArqQoHLzX4DUoDkBeNXzQCcDruz
FsRzPpaYptdRVcfZ5VXBy5Q=
=sDrD
-----END PGP SIGNATURE-----