[Python-Dev] Is there another way to solve the continuation problem?

[rushing at nightmare.com]

Skip Montanaro writes:
 > Can exceptions be coerced into providing the necessary structure
 > without botching up the application too badly?  Seems that at some
 > point where you need to do some I/O, you could raise an exception
 > whose second expression contains the necessary state to get back to
 > where you need to be once the I/O is ready to go.  The controller
 > that catches the exceptions would use select or poll to prepare for
 > the I/O then dispatch back to the handlers using the information
 > from exceptions.

 > [... code ...]

Well, you just re-invented the 'Reactor' pattern! 8^)

http://www.cs.wustl.edu/~schmidt/patterns-ace.html

 > One thread, some craftiness needed to construct things.  Seems like
 > it might isolate some of the statefulness to smaller functional
 > units than a pure state machine.  Clearly not as clean as
 > continuations would be.  Totally bogus?  Totally inadequate?  Maybe
 > Sam already does things this way?

What you just described is what Medusa does (well, actually, 'Python'
does it now, because the two core libraries that implement this are
now in the library - asyncore.py and asynchat.py).  asyncore doesn't
really use exceptions exactly that way, and asynchat allows you to add 
another layer of processing (basically, dividing the input into
logical 'lines' or 'records' depending on a 'line terminator').

The same technique is at the heart of many well-known network servers,
including INND, BIND, X11, Squid, etc..  It's really just a state
machine underneath (with python functions or methods implementing the
'states').  As long as things don't get too complex.  Python
simplifies things enough to allow one to 'push the difficulty
envelope' a bit further than one could reasonably tolerate in C.  For
example, Squid implements async HTTP (server and client, because it's
a proxy) - but stops short of trying to implement async FTP.  Medusa
implements async FTP, but it's the largest file in the Medusa
distribution, weighing in at a hefty 32KB.

The hard part comes when you want to plug different pieces and
protocols together.  For example, building a simple HTTP or FTP server
is relatively easy, but building an HTTP server *that proxied to an
FTP server* is much more difficult.  I've done these kinds of things,
viewing each as a challenge; but past a certain point it boggles.

The paper I posted about earlier by Matthew Fuchs has a really good
explanation of this, but in the context of GUI event loops... I think
it ties in neatly with this discussion because at the heart of any X11
app is a little guy manipulating a file descriptor.

-Sam