Is Stackless Python DEAD?

[Michael Hudson]

I've refreshed my memory somewhat, but this certainly shouldn't be
taken as definitive.

Martin von Loewis <loewis at informatik.hu-berlin.de> writes:

> Michael Hudson <mwh at python.net> writes:
> 
> > > - it adds 17 new members to the frame object, doubling the number
> > >   of members. Usage of some of these members isn't obvious to me.
> > 
> > See below about map & friends.
> 
> If they are all just for the variables in map etc., a better solution
> must be found - don't think it is desirable to have them in every
> frame, if they are just used inside map.

I thought this at the time I looked at the code.

> However, from inspecting the patch, I doubt this is the case. Some of
> them are general-purpose, but I couldn't easily figure out what, for
> example, this first-instruction business is good for.

Sorry, my memory had failed me somewhat.  I've downloaded the code
now.

I think f_first_instr is there to transfer information between
eval_code2_setup and eval_code2_loop.  As such I don't think it's at
all necessary; I think eval_code2_loop could just go

	_PyCode_GETCODEPTR(f->f_code, &first_instr);

Another mystery, this time about core Python: why go to so much
trouble to ensure that code_object->co_code is only a readable buffer?

(a) I'm pretty sure it's always a string
(b) I don't see any error checking, so if it's something more
    complicated than a string and bf_getreadbuffer fails, the
    interpreter is going to go "boom".

Oh well, just another mystery about the buffer interface.

Not sure what f_next_instr is for either; it seems to always satisfy

unsigned char* first;
_PyCode_GETCODEPTR(f->f_code, &first);
f->f_next_instr == first + f->f_lasti

It may be another optimization...

I think f_stackpointer does what f_stacktop does in today's Python (it
was introduced by the generator patch).

f_statusflags records various hairy things about the frame.  I
certainly don't understand the details, but I don't think you can go
without this.

f_execute is usually eval_code2_loop, but can also be
builtin_map_loop.  This is fundamental.

f_dealloc is used by the continuation module, as are f_node, f_co,
f_coframes.  I /really/ don't understand that module.

f_depth seems to be used only by map().  Oh no, continuations too.

f_age really does seem to be only used by map().  To store the length
of the longest sequence being iterated over, it seems.  Nice name.

Same for f_reg3.

So there definitely seems to be some room for cleanup here.  I'd need
to understand continuationmodule.c to see exactly how much.

> > So when map wants to call the Python function, it needs to stuff all
> > the local data it cares about into a frame object (see above), push
> > this frame object onto the (Python) stack, *return* to the interpreter
> > in such a way that the mapping function is called next and then every
> > time(!) that returns have the interpreter call back into map again.
> 
> Why couldn't map behave as if it was a plain Python function,
> implemented as
> 
> def map(fun,args):
>   res = []
>   for a in args:
>     res.append(fun(a))
> 
> [I know that map is implemented in a more-involved way; it should
> still be possible to find its Python equivalent, then think how
> stackless would execute this equivalent]
> 
> For the additions to frame_state, that would mean that map should
> reserve a number of localsplus variables, instead of outright adding
> to frame_state on the C level.

I wasn't trying to justify the existing implementation, just explain it.

I agree with you here.

> > > - The code adds PREPARE macros into each branch of ceval. Why?
> > > - It adds a long list of explicitly not-supported opcodes into
> > >   the ceval switch, instead of using 'default:'. No explanation
> > >   for that change is given, other than 'unused opcodes go here'.
> > >   Is it necessary to separately maintain them? Why?
> > 
> > This was an optimization Chris used to try and get back some of the
> > performance lost during the stackless changes.  IIRC, he handles
> > exceptions and return values as "pseudo-opcodes" rather than using the
> > WHY_foo constants the current ceval.c uses.  I never really understood
> > this part.
> 
> If it is an optimization, I think we should take a step back and first
> try to understand what it tries to optimize, and how it does
> that. Perhaps we find that it isn't needed at all, and that the
> problem could be solved in a different way. If you don't understand
> it, it can't go into Python.

And here.

> > I think integrating stackless into the core is a fairly huge amount of
> > work.  I'd like to think I could do it, given several months of
> > full-time effort (which isn't going to happen).  About the only likely
> > way I see for it to get in is for it to become important to Zope
> > Corp. for some reason, and them paying Tim or Guido (or Chris) to do
> > it.
> 
> I don't think these are the options. I don't volunteer to support this
> code myself, either, but if somebody would step forward and claim that
> she understands it all, and is willing to present it to Guido in a way
> that he understands it also, and if all the hackish parts of it would
> be replaced by understandable code, I think it could go into Python.
> It just needs a determined volunteer to work on it.

Yeesss, but does anyone have the time for this?  I mean, I've just
spent an hour or so I probably shouldn't have writing this article,
and I've barely scratched the surface.

I don't want to discourage anyone, but I thinking liking a challenge
is a prerequisite.

I might devote an evening or two to it and see where I get.

Homological algebra beckons -- brain relief in this context!

Cheers,
M.

-- 
  LINTILLA:  You could take some evening classes.
    ARTHUR:  What, here?
  LINTILLA:  Yes, I've got a bottle of them.  Little pink ones.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 12