[Email-SIG] Design Thoughts Summary

[Barry Warsaw]

On Nov 14, 2009, at 5:12 PM, Matthew Dixon Cowles wrote:

> Thank you. I am virtually 100% in agreement that this document
> represents what people have agreed on and that it represents what is
> sensible to do.

As am I.  Fantastic work in pulling this all together David.

I'm a bit slammed right now, but a quick comment...

>> * The API needs to at a minimum have hooks available for an
>> application to store data on disk rather than holding everything in
>> memory.
>
> I remain unconvinced that this is worth the trouble. Yes, the Twisted
> folks say that they can't use the email module because they may be
> receiving hundreds of messages at once. But can anyone do anything
> with hundreds of messages at once other than write them to disk?
>
> And would anything actually be improved by reading hundreds of files
> at once, in small chunks, looking for MIME separators?

Mailman has a similar problem.  Even if we get just a few big  
messages, they can crush the system.  You could argue that the MTA  
should just block messages with 50MB bodies if the underlying Mailman  
code can't handle it, but I still think we can do better.

I think we're fine if all the headers and MIME structure were kept in  
memory it would be fine.  But I do think we just want to be able to  
never store the raw body content in memory (perhaps unless needed, on  
demand).  Mailman for example rarely cares about the bytes of say an  
image/jpeg body.

I'm very psyched to see this work progress!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/email-sig/attachments/20091115/18133bf3/attachment.pgp>