[Mailman-Users] Mailman Borks on Large Attachments

[Lachlan Michael]

Mark Sapiro さんは書きました:
> Lachlan Michael wrote:
>   
>> I have a problem when using mailman. The problem manifests itself when a
>> user sends a largish attachment (say 300kB) such as a digital camera
>> picture to a list. The lists I run are small private groups and if
>> possible  I would like to allow users to send these kind of attachments.
>>
>> The server is question is vanilla FreeBSD 6.3 machine, using mailman
>> 2.1.9_5 from ports with sendmail. The machine has 1GB of physical memory.
>> Python 2.5 is used (default settings). The maximum message size for this
>> list was set at 150kB, in an effort to stop this problem (which it didn't
>> seem to do), but I would like to set this limit higher (maybe to 1MB).
>>     
>
>
> Setting max_message_size doesn't help because the error occurs before
> the message size is checked.
>
>
> <snip>
>   
>> Mailman fails with a "memory error" (see logs below).
>>
>> This error then caused ALL mail to ALL other lists on the same machine to
>> be stop being processed, although they were queued and could be sent
>> later. Eventually some days later I realized that mail was not getting
>> through and investigated.
>>
>> A simple restart of mailman did not help, and I found a suspicious file in
>> qfiles. The suspicious file had a .bak extension I moved it away. Then a
>> restart of mailman allowed all queued messages to be sent correctly (with
>> the exception of the suspicious mail file).
>>     
>
>
> This part of the problem is fixed in Mailman 2.1.10b3 which is now
> available (see <http://www.list.org/download.html>).
>
> Prior to Mailman 2.1.9, the error would have occurred once and the
> message would have been lost. Beginning in 2.1.9, a queue file
> backup/recovery mechanism was implemented which allowed messages that
> were in process when a crash occurred to be recovered, but the problem
> here is that if the message itself causes the crash, a loop occurs
> such as you experienced. This is addressed in 2.1.10b3 by counting the
> number of times a .bak file has been recovered and moving it aside
> after 3 tries.
>
> It would be wonderful if you could install the latest
> Mailman/Queue/Switchboard.py from
> <http://codebrowse.launchpad.net/~mailman-coders/mailman/2.1/files>
> and see if this mitigates the problem.
>
>
>   
Ok, I will definitely try this.

>> Using less I saw that the mail
>> included a large picture file.  There are 11 members in the list that this
>> mail was sent to. The offending mail file size is shown below (ls -lh).
>>
>> -rw-rw----   1 mailnull  mailman   599K Jan 30 23:36
>> 1201703766.518079+a5beef9fbd8e4eea60ab77049b3bed4e877a98fc.bak
>>
>> My questions is, is this a bug, a mis-configuration on my part, or
>> expected behaviour of mailman under these conditions for this machine
>> (i.e. am I hitting python or FreeBSD limits)?
>>     
>
>
> MemoryError is beyond the control of both Mailman and Python. Python is
> requesting additional memory form the OS for the process and is being
> denied. This is some kind of hardware or OS limitation. It could be
> that the OS itself is running out of real or virtual memory or the
> process is running into some process limit. This is a FreeBSD question.
>
>
>   
Ok, I will take this question over to the FreeBSD lists.
>> My expectation would be that
>> 1) the user gets an error that the file sent was too big
>>     
>
>
> We can't do that in this case, because we haven't successfully parsed
> the message so we don't even know who the sender is. If there were no
> MemoryError, and the message just exceeded max_message_size, this is
> exactly what would happen.
>
>
>   
>> 2) all other lists and other mail are not affected
>>     
>
>
> The 2.1.10b3 fix addresses this. However, the way things are currently
> set, Incoming runner will crash 3 times before moving this message
> aside. If this happens again with a second message, IncomingRunner
> will crash 3 more times. On the fourth such message, IncomingRunner
> will reach its limit of 10 restarts, so if these errors continue,
> Mailman's mail processing will eventually stop.
>
>
>   
Great!
>> I guess many people would be jumping up and down if this occurred
>> regularly, so I imagine that this a local configuration problem, but I
>> would appreciate if anybody had any advice for debugging or changing my
>> environment to remove this error.
>>     
>
>
> 500KB is not a big message. This should not be happening. I am only
> guessing, but I think there must be some per-process memory limitation
> in your OS that is set way too low.
>
>   
My suspicion is something like that too, although from all the tests I 
have done the memory itself seems good.

Thanks again for your kind explanation and help. Mailman is a great 
piece of software.

Lachlan