[Python-Dev] Simple generators, round 2

[Neil Schemenauer]

On Mon, Mar 19, 2001 at 04:49:37PM +0100, Christian Tismer wrote:
> A possible bug may be in frame_resume, where you are doing
> +       f->f_back = tstate->frame;
> without taking care of the prior value of f_back.

Good catch.  There is also a bug when f_suspendvalue is being set
(Py_XDECREF should be called first).

[Christian on disallowing resume on frame already running]
> 1) introduce a lock flag for frames which are currently
>    executed by some interpreter on the C stack. This is
>    what Stackless does currently.
>    Maybe you can just use your new f_suspendvalue field.
>    frame_resume must check that this value is not NULL
>    on entry, and set it zero before resuming.

Another good catch.  It would be easy to set f_stackbottom to
NULL at the top of PyEval_EvalFrame.  resume already checks this
to decide if the frame is resumable.

> 2) Do not expose the resume and suspend methods to the
>    Python user, and recode Generator.py as an extension
>    module in C. This should prevent abuse of frames.

I like the frame methods.  However, this may be a good idea since
Jython may implement things quite differently.

> Proposal for a different interface:
> I would change the interface of PyEval_EvalFrame
> to accept a return value passed in, like Stackless
> has its "passed_retval", and maybe another variable
> that explicitly tells the kind of the frame call,
> i.e. passing the desired why_code. This also would
> make it easier to cope with the other needs of Stackless
> later in a cleaner way.
> Well, I see you are clearing the f_suspendvalue later.
> Maybe just adding the why_code to the parameters
> would do. f_suspendvalue can be used for different
> things, it can also become the place to store a return
> value, or a coroutine transfer parameter.
> 
> In the future, there will not obly be the suspend/resume
> interface. Frames will be called for different reasons:
> suspend  with a value  (generators)
> return   with a value  (normal function calls)
> transfer with a value  (coroutines)
> transfer with no value (microthreads)

The interface needs some work and I'm happy to change it to
better accommodate stackless.  f_suspendvalue and f_stackbottom
are pretty ugly, IMO.  One unexpected benefit: with
PyEval_EvalFrame split out of eval_code2 the interpreter is 5%
faster on my machine.  I suspect the compiler has an easier time
optimizing the loop in the smaller function.

BTW, where is this stackless light patch I've been hearing about?
I would be interested to look at it.  Thanks for your comments.

  Neil