On 7/26/2011 12:54 PM, Maxim Khitrov wrote:
On Tue, Jul 26, 2011 at 10:42 AM, Michael Foord<fuzzyman@gmail.com> wrote:
Well I'm sure Menno would be interested if you could actually demonstrate those limitations rather than merely assert that. :-)
(As he implemented to overcome problems with imaplib and that is why people use it.)
Not that you're necessarily wrong, but I'm skeptical.
Most of the limitations are outlined in my README file. I'm not sure what kind of a demonstration would convince you. The problem is that if your applications do not require things like data compression, asynchronous/parallel command execution, or multiple literals in commands, you will not agree that there is anything wrong with the existing tools. It's very much a matter of personal experience.
One of the things I'm trying to address with my library is strict adherence to the current version of the IMAP4 protocol. The other is performance; hence the implementation of extensions such as SASL-IR, IDLE, non-synchronizing literals, multiappend, and compression.
On the performance side, if you have an application that's trying do some sort of processing of a 6 GB mailbox with 700,000 messages in it, executing a separate FETCH command for each message will take you a week to finish. If you try to be clever and FETCH 1000 messages at a time, for example, you'll quickly run into a few problems:
1. The data for all 1000 messages is buffered in memory before you ever to get to see anything.
2. If the server fails to retrieve one of those messages, which actually happens rather frequently, at least on Gmail servers, you lose the data for the other 999. You'll get a 'NO' response from imaplib or an exception from IMAPClient, and an explanation of what went wrong instead of any data.
3. If you requested data for messages 1:1000 and the server decides to send you a FLAGS update for message 9999, imaplib will return that response as part of the data for the original request.
All are interface design problems, which are inherited by IMAPClient.
- Max
P.S. It is not my intention to discourage the use of IMAPClient in any way. Its existence is a good thing for 99% of the users, because it does address a number of key imaplib issues with just the response parser and a UTF-7 codec. My point is that there are real-world use cases out there that cannot be handled by imaplib or IMAPClient, and for those, I'm offering my library as a more general solution that should satisfy the remaining 1% :)
I think a proper iterator rather than batch mode fetch interface will benefit more than 1% of people. I also think RFC-based modules should be updated to complete-as-possible-and-sensible implementations of current RFCs that have replaced (in usage) previous RFCs. -- Terry Jan Reedy