[Python-Dev] 'stackless' python?

[rushing at nightmare.com]

Tim Peters writes:
 > I'm not a fan of continuations myself; coroutines can be
 > implemented faithfully via threads (I posted a rather complete set
 > of Python classes for that in the pre-DejaNews days, a bit more
 > flexible than Icon's coroutines); and:

Continuations are more powerful than coroutines, though I admit
they're a bit esoteric.  I programmed in Scheme for years without
seeing the need for them.  But when you need 'em, you *really* need
'em.  No way around it.

For my purposes (massively scalable single-process servers and
clients) threads don't cut it... for example I have a mailing-list
exploder that juggles up to 2048 simultaneous SMTP connections.  I
think it can go higher - I've tested select() on FreeBSD with 16,000
file descriptors.

[...]

BTW, I have actually made progress borrowing a bit of code from SCM.
It uses the stack-copying technique, along with setjmp/longjmp.  It's
too ugly and unportable to be a real candidate for inclusion in
Official Python.  [i.e., if it could be made to work it should be
considered a stopgap measure for the desperate].

I haven't tested it thoroughly, but I have successfully saved and
invoked (and reinvoked) a continuation.  Caveat: I have to turn off
Py_DECREF in order to keep it from crashing.

  | >>> import callcc
  | >>> saved = None
  | >>> def thing(n):
  | ...     if n == 2:
  | ...             global saved
  | ...             saved = callcc.new()
  | ...     print 'n==',n
  | ...     if n == 0:
  | ...             print 'Done!'
  | ...     else:
  | ...             thing (n-1)
  | ... 
  | >>> thing (5)
  | n== 5
  | n== 4
  | n== 3
  | n== 2
  | n== 1
  | n== 0
  | Done!
  | >>> saved
  | <Continuation object at 80d30d0>
  | >>> saved.throw (0)
  | n== 2
  | n== 1
  | n== 0
  | Done!
  | >>> saved.throw (0)
  | n== 2
  | n== 1
  | n== 0
  | Done!
  | >>> 

I will probably not be able to work on this for a while (baby due any
day now), so anyone is welcome to dive right in.  I don't have much
experience wading through gdb tracking down reference bugs, I'm hoping
a brave soul will pick up where I left off. 8^)

http://www.nightmare.com/stuff/python-callcc.tar.gz
ftp://www.nightmare.com/stuff/python-callcc.tar.gz

-Sam