[Twisted-Python] Hacking imap
Hi, Just wanted to give a heads-up that I intend feed you guys a patch for the imap server to properly handle literals in all places they need to be handled towards the end of this week. Unfortunately, it's quite a large change since it requires a complete restructuring of how arguments are parsed. Basically, instead of having each command parse it's own arguments, the idea is to list what arguments each command expects, and have the command dispatcher parse the arguments and possibly request continuation for literals. I expect to have more patches coming, as I need this for the project we're working on. It'd be nice if we could coordinate the work, so we don't end up with incompatible fixes and improvements, and try to avoid duplicate work. /Anders -- -- Of course I'm crazy, but that doesn't mean I'm wrong. Anders Hammarquist | iko@cd.chalmers.se Physics student, Chalmers University of Technology, | Hem: +46 31 88 48 50 G|teborg, Sweden. RADIO: SM6XMM and N2JGL | Mob: +46 707 27 86 87
On Tue, Jun 24, 2003 at 01:38:32AM +0200, Anders Hammarquist wrote:
Hi,
Just wanted to give a heads-up that I intend feed you guys a patch for the imap server to properly handle literals in all places they need to be handled towards the end of this week.
Could you elaborate on "properly handle"? I am currently making changes in this area as well, mainly in the direction of avoiding all the in-memory buffering the code currently does. Jp -- "One World, one Web, one Program." - Microsoft(R) promotional ad "Ein Volk, ein Reich, ein Fuhrer." - Adolf Hitler
In a message of Tue, 24 Jun 2003 01:23:53 EDT, Jp Calderone writes:
On Tue, Jun 24, 2003 at 01:38:32AM +0200, Anders Hammarquist wrote: Could you elaborate on "properly handle"? I am currently making changes in this area as well, mainly in the direction of avoiding all the in-memory buffering the code currently does.
Certainly, There are lots of places where literals aren't handled (I think that the only place where they are handled is in APPEND, at least from what my testing show). Anyplace where the syntax allows a string, a literal is allowed. The idea is to have the command dispatcher parse the arguments for each command, and then pass the arguments to the command handling function. This way, if the argument is a literal or a string, it can be passed as a file object (but there is no need - for example, if a mailbox name is given as a literal, there is little reason to give it in a file object). So instead of naming the handler functions <state>_<COMMAND>, I'll have <state>_<COMMAND> be a tuple containing ( <handler_function>, <arg_parser>, <arg_parser>, ...) and then call the handler_function with the results of the arg_parser functions. /Anders -- -- Of course I'm crazy, but that doesn't mean I'm wrong. Anders Hammarquist | iko@cd.chalmers.se Physics student, Chalmers University of Technology, | Hem: +46 31 88 48 50 G|teborg, Sweden. RADIO: SM6XMM and N2JGL | Mob: +46 707 27 86 87
On Tue, Jun 24, 2003 at 01:28:41PM +0200, Anders Hammarquist wrote:
In a message of Tue, 24 Jun 2003 01:23:53 EDT, Jp Calderone writes:
On Tue, Jun 24, 2003 at 01:38:32AM +0200, Anders Hammarquist wrote: Could you elaborate on "properly handle"? I am currently making changes in this area as well, mainly in the direction of avoiding all the in-memory buffering the code currently does.
Certainly,
There are lots of places where literals aren't handled (I think that the only place where they are handled is in APPEND, at least from what my testing show). Anyplace where the syntax allows a string, a literal is allowed.
Ahh yes, I believe I blocked out that particular memory :( Thanks for tackling it :)
The idea is to have the command dispatcher parse the arguments for each command, and then pass the arguments to the command handling function. This way, if the argument is a literal or a string, it can be passed as a file object (but there is no need - for example, if a mailbox name is given as a literal, there is little reason to give it in a file object).
So instead of naming the handler functions <state>_<COMMAND>, I'll have <state>_<COMMAND> be a tuple containing ( <handler_function>, <arg_parser>, <arg_parser>, ...) and then call the handler_function with the results of the arg_parser functions.
This sounds like a great approach. If I could make a request (and maybe you were going to do this anway), it would be nice if the arg_parsers could work incrementally: this is mostly of use when receiving an actual message, to avoid buffering it all in memory (or even buffering it on disk and pushing off parsing it until it is completely received). Maybe this means <arg_parser> should be a factory that returns a parser, so that state can be more easily managed when parsing the full list-expression returned with from a fetch command. Sound reasonable? Jp -- http://catandgirl.com/view.cgi?44
In a message of Tue, 24 Jun 2003 16:30:38 EDT, Jp Calderone writes:
So instead of naming the handler functions <state>_<COMMAND>, I'll have <state>_<COMMAND> be a tuple containing ( <handler_function>, <arg_parser>, <arg_parser>, ...) and then call the handler_function with the results of the arg_parser functions.
This sounds like a great approach. If I could make a request (and maybe you were going to do this anway), it would be nice if the arg_parsers could work incrementally: this is mostly of use when receiving an actual message,
That was the idea. Each arg_parser gets (what's left of) the current line, and if they need more data, they'll have to arrange for reading it. (Otherwise the easy fix would have been to have lineReceived() read literals as well. I tried this approach first, but I didn't like it.)
Maybe this means <arg_parser> should be a factory that returns a parser, so that state can be more easily managed when parsing the full list-expression returned with from a fetch command.
That's a thought. I'll keep it in mind. /Anders -- -- Of course I'm crazy, but that doesn't mean I'm wrong. Anders Hammarquist | iko@cd.chalmers.se Physics student, Chalmers University of Technology, | Hem: +46 31 88 48 50 G|teborg, Sweden. RADIO: SM6XMM and N2JGL | Mob: +46 707 27 86 87
participants (2)
-
Anders Hammarquist -
Jp Calderone