[Python-Dev] Removing the block stack (was Re: PEP 343 and __with__)

[Calvin Spealman]

On 10/6/05, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:09 PM 10/5/2005 -0700, Neal Norwitz wrote:
> >The general idea is to allocate the stack in one big hunk and just
> >walk up/down it as functions are called/returned.  This only means
> >incrementing or decrementing pointers.  This should allow us to avoid
> >a bunch of copying and tuple creation/destruction.  Frames would
> >hopefully be the same size which would help.  Note that even though
> >there is a free list for frames, there could still be
> >PyObject_GC_Resize()s often (or unused memory).  WIth my idea,
> >hopefully there would be better memory locality, which could speed
> >things up.
>
> Yeah, unfortunately for your idea, generators would have to copy off bits
> of the stack and then copy them back in, making generators slower.  If it
> weren't for that part, the idea would probably be a good one, as arguments,
> locals, cells, and the block and value stacks could all be handled that
> way, with the compiler treating all operations as base-pointer offsets,
> thereby eliminating lots of more-complex pointer management in ceval.c and
> frameobject.c.

If we had these seperate stacks for each thread, would it be possible
to also create a stack for generator calls? The current call
operations could possibly do a check to see if the function being
called is a generator (if they don't have a generator bit, could they,
to speed this up?). This generator-specific stack would be used for
the generator's frame and any calls it makes on each iteration. This
may pose threat of a bottleneck, allocating a new stack in the heap
for every generator call, but generators are generally iterated more
than created and the stacks could be pooled, of course.

I don't know as much as I'd like about the CPython internals, so I'm
just throwing this out there for commenting by those in the know.