[Python-Dev] comments on PEP 219

[Jeremy Hylton]

Here are some comments on Gordon's new draft of PEP 219 and the
stackless dev day discussion at Spam 9.

I left the dev day discussion with the following takehome message:
There is a tension between Stackless Python on the one hand and making
Python easy to embed in and extend with C programs on the other hand.
The PEP describes this as the major difficulty with C Python.  I won't
repeat the discussion of the problem there.

I would like to seem a somewhat more detailed discussion of this in
the PEP.  I think it's an important issue to work out before making a
decision about a stack-light patch.

The problem of nested interpreters and the C API seems to come up in
several ways.  These are all touched on in the PEP, but not in much
detail.  This message is mostly a request for more detail :-).

  - Stackless disallows transfer out of a nested interpreter.  (It
    has, too; anything else would be insane.)  Therefore, the
    specification for microthreads &c. will be complicated by a
    listing of the places where control transfers are not possible.
    The PEP says this is not ideal, but not crippling.  I'd like to
    see an actual spec for where it's not allowed in pure Python.  It
    may not be crippling, but it may be a tremendous nuisance in
    practice; e.g. remember that __init__ calls create a critical
    section.

  - If an application makes use of C extensions that do create nested
    interpreters, they will make it even harder to figure out when
    Python code is executing in a nested interpreter.  For a large
    systems with several C extensions, this could be complicated.  I
    presume, therefore, that there will be a C API for playing nice
    with stackless.  I'd like to see a PEP that discusses what this C
    API would look like.

  - Would allow of the internal Python calls that create nested
    functions be replaced?  I'm thinking of things like
    PySequence_Fast() and the ternary_op() call in abstract.c.  How
    hard will it be to convert all these functions to be stackless?
    How many functions are affected?  And how many places are they
    called from?

  - What is the performance impact of adding the stackless patches?  I
    think Christian mentioned a 10% slowdown at dev day, which doesn't
    sound unreasonable.  Will reworking the entire interpreter to be
    stackless make that slowdown larger or smaller?

One other set of issues, that is sort-of out of bounds for this
particular PEP, is what control features do we want that can only be
implemented with stackless.  Can we implement generators or coroutines
efficiently without a stackless approach?

Jeremy