[Mailman-Users] Python process size grows 30x in 8 hours (memory
barry at list.org
Wed Jul 2 18:03:42 CEST 2008
-----BEGIN PGP SIGNED MESSAGE-----
On Jul 2, 2008, at 11:15 AM, Mark Sapiro wrote:
> At some point in the next day or so, I'm going to make a modified
> scripts/post script which will queue incoming messages in qfiles/bad
> and then move them to qfiles/in only if they are under a certain size.
> I'm really curious to see if that will help.
This should be moved to mailman-developers, but in general it's an
interesting idea. In MM3 I've split the incoming queue into two
separate queues. The incoming queue now solely determines the
disposition of the message, i.e. held, rejected, discarded or
accepted. If accepted, the message is moved to a pipeline queue where
it's munged for delivery (i.e. headers and footers added, etc.).
MM3 also has an LMTP queue runner, which I'd like to make the default
delivery mechanism for 3.0 and possibly 2.2 (yes, I still have a todo
to back port MM3's new process architecture to 2.2). Although it's
not there right now, it would be trivial to add a check on the raw
size of the message before it's parsed. If it's too large then it can
be rejected before the email package attempts to parse it, and that
would give the upstream LTMP client (i.e. your MTA) a better diagnostic.
It still makes sense to put a size limit in your MTA so it never hits
the LMTP server because the string will still be in the Python
process's memory. But at least you won't pay the penalty for parsing
such a huge message just to reject it later.
>> Trying to find a way to look at the contents of the heap or at
>> least limit
>> its growth.
>> Or is there not a way expire & restart mailman processes analogous
>> to the
>> apache httpd process expiration (designed to mitigate this kind of
>> growth over time)?
> bin/mailmanctl could be modified to do this automatically, but
> currently only does it on command (restart) or signal (SIGINT), but I
> gather you're already running a cron that does a periodic restart.
This is a good idea. It might be better to do this in
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
-----END PGP SIGNATURE-----
More information about the Mailman-Users