[Python-Dev] ob_refcnt access

[Sam Rushing]

Mark Hammond writes:
 > I tried, and look what happened :-) Seriously, some if this stuff
 > would be way cool.
 > 
 > Bit I also understand completely the silence on this issue.  When
 > the thread started, there was much discussion about exactly what
 > the hell these continuation/coroutine thingies even were.  However,
 > there were precious few real-world examples where they could be
 > used.  A few acedemic, theoretical places, but the only real
 > contender I have seen brought up was Medusa.  There were certainly
 > no clear examples of "as soon as we have this, I could change abc
 > to take advantage, and this would give us the very cool xyz"

Part of the problem is that we didn't have the feature to play with.
Many of the possibilities are showing up now that it's here...

The basic advantage to coroutines is they allow you to turn any
event-driven/state-machine problem into one that is managed with
'normal' control state; i.e., for loops, while loops, nested procedure
calls, etc...

Here are a few possible real-world uses:

==================================================

Parsing. I remember a discussion from a few years back about the
distinction between 'push' and 'pull' model parsers.  Coroutines let
you have it both ways; you can write a parser in the most natural way
(pull), but use it as a 'push'; i.e. for a web browser.

==================================================

"http sessions".  A single 'thread' of control that is re-entered
whenever a hit from a particular user ('session') comes in to the web
server:

[Apologies to those that have already seen this cheezy example]

def ecommerce (session):
  session.login() # sends a login form, waits for it to return
  basket = []
  while 1:
    item = session.shop_for_item()
    if item:
      basket.append (item)
    else:
      break
  if basket:
    session.get_shipping_info()
    session.get_payment_info()
    session.transact()

'session.shop_for_item()' will resume the main coroutine, which will
resume this coroutine only when a new hit comes in from that
session/user, and 'return' this hit to the while loop.

I have a little web server that uses this idea to play blackjack:

  http://www.nightmare.com:7777/
  http://www.nightmare.com/stuff/blackjack_httpd.py

[though I'm a little fuzzy on the rules].

Rather than building a state machine that keeps track of where the
user has been, and what they're doing, you can keep all the state in
local variables (like 'basket' above) - in other words, it's a much
more natural style of programming.

==================================================

One of the areas I'm most excited about is GUI coding.  All GUI's are
event driven.  All GUI code is therefore written in a really twisted,
state-machine fashion; interactions are very complex.  OO helps a bit,
but doesn't change the basic difficulty - past a certain point
interesting things become too complex to try...

Mr. Fuchs' paper ("Escaping the event loop: an alternative control
structure for multi-threaded GUIs") does a much better job of
describing this than I can:

http://cs.nyu.edu/phd_students/fuchs/
http://cs.nyu.edu/phd_students/fuchs/gui.ps

==================================================

Tim's example of 'dumping' a computation in the middle and storing it
on disk (or sending it over a network), is not a fantasy... I have a
'stackless' Scheme system that does this right now.

==================================================

Ok, final example.  Isn't there an interface in Python to call a
certain function after every so many vm insns?  Using coroutines you
could hook into this and provide non-preemptive 'threads' for those
platforms that don't have them.  [And the whole thing would be written
in Python, not in C!]

==================================================

 > So, if anyone else if feeling at all like me about this issue, they
 > are feeling all warm and fuzzy knowing that a few smart people are
 > giving us the facility to do something we hope we never, ever have
 > to do. :-)

"When the only tool you have is a hammer, everything looks like a
nail".  I saw the guys over in the Scheme shop cutting wood with a
power saw; now I feel like a schmuck with my hand saw.

You are right to be frightened by the strangeness of the underlying
machinery; hopefully a simple and easy-to-understand interface can be
built for the C level as well as Python.  I think Christian's 'frame
dispatcher' is fairly clear, and not *that* much of a departure from
the current VM; it's amazing to me how little work really had to be
done!

-Sam