[pypy-dev] Re: [pypy-svn] r18743 - pypy/dist/pypy/module/stackless

[Christian Tismer]

Ok, let me give it a try, adding comments.

We introduce an RPython type ``continuation`` and a built-in function
``yield_continuation()`` that work as follows (see example below):

* The built-in function ``yield_continuation()`` causes the current
   function's state to be captured in a new ``continuation`` object that
   is returned to the parent.  Only one frame, the current one, is
   captured this way.  The current frame is suspended and the caller
   continues to run.

Thhis is a problem, since this extra state is potentially able to
re-run its caller twice.

* A ``continuation`` object can be jumped to by calling its ``switch()``
   method with no argument.

Ok.

* ``yield_continuation()`` and ``switch()`` themselves return a new
   ``continuation`` object: the freshly captured state of the caller of
   the source ``switch()`` that was just executed, or None in the case
   described below.

switch() is ok. yield_co() is a problem for me.

* ``yield_continuation()`` only works when called from a function whose
   return type is precisely ``continuation``.  In other words this
   function must also contain a regular return statement that returns a
   ``continuation``.  When the captured continuation is later resumed and
   the function eventually returns via this regular return statement,
   then the returned ``continuation`` object designates the place where
   execution should switch to (i.e. execution does not return to the
   original caller).  In such an implicit switch, a None is returned to
   the ``switch()`` or ``yield_continuation()`` at place that we switch
   to.

did not understand this at all, will try again, tomorrow.

* every continuation must be resumed once and only once.  Not resuming
   it at all causes a leak.  Resuming it several times causes a crash.

I understand the only once. But why the leak issue? We have its state,
why don't we allow to collect?

* a function that called ``yield_continuation()`` should not raise.  It
   would have no implicit continuation to propagate the exception to.
   That would be a crashingly bad idea.

The following example would print the numbers from 1 to 7 in order::

     def g():
         print 2
         cont_before_5 = yield_continuation()
         print 4
         cont_before_7 = cont_before_5.switch()
         print 6
         return cont_before_7

     def f():
         print 1
         cont_before_4 = g()
         print 3
         cont_before_6 = cont_before_4.switch()
         print 5
         cont_after_return = cont_before_6.switch()
         print 7
         assert cont_after_return is None

     f()

The example is correct and easier to understand than the description.
I'm anyway still slightly hesitant to expose that kind of interface
at all. We are raising expectations that we should better not fulfil.
These continuations are one-shots in the first-place, which isn't
obvious. This is why I went for tasklets, explicitly bound to a new
function. I agree that this fork-alike approach is much nicer, but
do we always need to sell our mother for something that looks nice?

If I would expose this interface, then I would make it
full-continuations, and have the full catastrophe. But exposing
castrated continuations, just for the sake of it? If you don't
want peopleto use something, be aware and don't expose an interface,
or they will demand the whole thing from you, the other day.

I don't really see the need to expose these at ll-level. What people
want is to be able to spawn a seperate Python no-thread. What you
propose is much more, and I have learnt that this is a bad thing to
do which can spoil all of your project, like it did to mine.

Well, I'm undecided. The interface makes some sense, that's without
doubt. But I think we need to be careful.

We absolutely should evict the name "continuation". The association
with this will always be Stackless Python, and it will strongly
influence any positive cooperation with Guido in the future. I don't
think that I want this for our project. Guido has not, does not,
never will and doesn't want to understand them, although I have to
admit that he really tried. He just came to the conclusion that
this is a bad thing.
And after all, I'm not that confident that I think they aren't.

After I did the continuations for Python, I was the proudest man
in the world, and I tried to get this into Python, in a very
pushy way. And I had the best support by people like Tim Peters.
But that as not enough, and I failed miserably. I hardly ever
really recovered from that, it was a major insult for my person.
Of course, I did Stackless 2.0 and 3.0 afterwards, as kind of
compromise to get the best out of what remained, but actually
the Stackless 1.0 with its continuations was the best thing I ever
did, and also the hardest one. Everything else is just makulature,
peanuts.

If we are defining something co-xxxx-alike, we should design and name
it in a way that makes sure that we are thinking of simple, concurrent
threadlets to be run in parallel, nothing fancy that is able to return
more than once to its caller. I know that you tried to model this, but
it still looks very much close to my full-continuation sample programs,
which probably very few people understand at all. It is not that I say
you are trying the same, it is just that you are playing with fire that
we don't want to see in PyPy, even if it just looks like that.

Please feel free to do a module with stuff like that, but please
*DO NOT* name it stackless, because this is exactly what Stackless
no longer wants to be and never will be, again!!

That's actually why I was upset and I asked for prior discussion.
I'm not at all against what you did, instead I appreciate your
good and understanding work. But please leave Stackless out of this,
because my project's current semantics are far apart from anything
looking like continuations. I have never been burnt that much in
any project so far, and I won't go this path again at any time, again.

Or you might tell me to stop selling peanuts and do the right thing,
again, ignoring Guido-nesses et al, although I doubt the world is ready
for real continuations.

sincerely and sort-of sadly -- chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/