I'm working on a project (http://www.cakem.net/) that involves networking. I have to do a lot of decoding of network data into an internal representation, and encoding internal representations into a network format.
The decoding part is going to present a problem, a problem that could easily be solved by continuations.
I want to right the code so that it will work when there is no guarantee that I'll have an entire message before I start decoding it. This means the parser may have have to stop at any random point in time, and then be restarted again when more data is available. I see no way to do this without continuations, or threads, and I refuse to use threads for something like this.
For now, I think I will do one of two things. Either build in the assumption that I will have the entire messages before I start parsing, or organize parsing as a series of steps that can be aborted in the middle and then restarted, and use exceptions to track whether the step was fully completed or not. Sort of a poor man's continuation.
Anyway, I'm not a Python developer, just a user. I just wanted to add this to the things you consider when you decide what you intend to do next with Python. Continuations are important, and AFAIK, the stackless mod is the only way to get them.
Eric, I've taken a look at your protocol, and it looks to be simple enough to just decode the header (maybe partially) and determine (based on the length of the data received) if the entire message has been received yet. Certainly it does end up reparsing the header a few times, but as long as you only check to see if the message has been received on data read (which is guaranteed to be at least 1 byte for select/poll read events, and is commonly much larger), there shouldn't be too much reparsing. I've actually done the same thing with some nontrivial headers, and have had no problems maxing out a 100mbit connection with a PII-400 and a properly subclassed asyncore.dispatcher. Even though it would be computationally more efficient to use continuations or generators, they are not necessary. You will likely find more overhead with the cryptographic portions of your protocol than you will reparsing the header multiple times. - Josiah