[Mailman-Users] Python process size grows 30x in 8 hours (memory

Mark Sapiro mark at msapiro.net
Thu Jul 3 05:01:22 CEST 2008


Fletcher Cocquyt wrote:
>
>On 7/2/08 8:15 AM, "Mark Sapiro" <mark at msapiro.net> wrote:
>
>> At some point in the next day or so, I'm going to make a modified
>> scripts/post script which will queue incoming messages in qfiles/bad
>> and then move them to qfiles/in only if they are under a certain size.
>> I'm really curious to see if that will help.
>
>Yes, having a global incoming maxmessagesize limit and handler (what will
>the sender receive back?) for mailman would be useful.


The attached 'post' file is a modified version of scripts/post.

It does the following compared to the normal script.

The normal script reads the message from the pipe from the MTA and
queues it in the 'in' queue for processing by an IncomingRunner. This
script receives the message and instead queues it in the 'bad' queue.
It then looks at the size of the 'bad' queue entry (a Python pickle
that will be just slightly larger than the message text). If the size
is less than MAXSIZE bytes (a parameter near the beginning of the
script, currently set to 1000000, but which you can change as you
desire), it moves the queue entry from the 'bad' queue to the 'in'
queue for processing.

The end result is queue entries smaller than MAXSIZE will be processed
normally, and entries >= MAXSIZE will be left in the 'bad' queue for
manual examination (with bin/dumpdb or bin/show_qfiles) and either
manual deletion or manual moving to the 'in' queue for processing.

The delivery is accepted by the MTA in either case so the poster sees
nothing unusual.

This is not intended to be used in a normal production environment. It
is only intended as a debug aid to see if IncomingRunners will not
grow so large if incoming message size is limited.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan



More information about the Mailman-Users mailing list