[Python-ideas] New imaplib implementation for Python 3.2+ standard library

[Terry Reedy]

On 7/26/2011 12:54 PM, Maxim Khitrov wrote:
> On Tue, Jul 26, 2011 at 10:42 AM, Michael Foord<fuzzyman at gmail.com>  wrote:

>> Well I'm sure Menno would be interested if you could actually demonstrate
>> those limitations rather than merely assert that. :-)
>>
>> (As he implemented to overcome problems with imaplib and that is why people
>> use it.)
>>
>> Not that you're necessarily wrong, but I'm skeptical.
>
> Most of the limitations are outlined in my README file. I'm not sure
> what kind of a demonstration would convince you. The problem is that
> if your applications do not require things like data compression,
> asynchronous/parallel command execution, or multiple literals in
> commands, you will not agree that there is anything wrong with the
> existing tools. It's very much a matter of personal experience.
>
> One of the things I'm trying to address with my library is strict
> adherence to the current version of the IMAP4 protocol. The other is
> performance; hence the implementation of extensions such as SASL-IR,
> IDLE, non-synchronizing literals, multiappend, and compression.
>
> On the performance side, if you have an application that's trying do
> some sort of processing of a 6 GB mailbox with 700,000 messages in it,
> executing a separate FETCH command for each message will take you a
> week to finish. If you try to be clever and FETCH 1000 messages at a
> time, for example, you'll quickly run into a few problems:
>
> 1. The data for all 1000 messages is buffered in memory before you
> ever to get to see anything.
>
> 2. If the server fails to retrieve one of those messages, which
> actually happens rather frequently, at least on Gmail servers, you
> lose the data for the other 999. You'll get a 'NO' response from
> imaplib or an exception from IMAPClient, and an explanation of what
> went wrong instead of any data.
>
> 3. If you requested data for messages 1:1000 and the server decides to
> send you a FLAGS update for message 9999, imaplib will return that
> response as part of the data for the original request.
>
> All are interface design problems, which are inherited by IMAPClient.
>
> - Max
>
> P.S. It is not my intention to discourage the use of IMAPClient in any
> way. Its existence is a good thing for 99% of the users, because it
> does address a number of key imaplib issues with just the response
> parser and a UTF-7 codec. My point is that there are real-world use
> cases out there that cannot be handled by imaplib or IMAPClient, and
> for those, I'm offering my library as a more general solution that
> should satisfy the remaining 1% :)

I think a proper iterator rather than batch mode fetch interface will 
benefit more than 1% of people. I also think RFC-based modules should be 
updated to complete-as-possible-and-sensible implementations of current 
RFCs that have replaced (in usage) previous RFCs.

-- 
Terry Jan Reedy