Q: Python 2.0 preliminary features?

[Greg Ewing]

Guido van Rossum wrote:
> 
> The magic may easily be
> triggered at the wrong moment -- e.g. the debugger may be looking at
> your stack frame, and depending on whether it copies the object it
> pulls out of your stack frame into a local variable before looking at
> it or not, it will see either a function or the magic object.

You have a point there. This shows up a deficiency in my
implementation that I hadn't noticed before.

The *intention* was that the local_function -> closure
transformation should only occur in one context, that is,
when the local variable is referred to by name in the
function or a function defined within it.

To implement this properly, the compiler should keep track
in its symbol table of which local variables were created
by def statements, and emit a closure-creating opcode after 
each operation which loads one of them.

This would require some major reorganisations of the
compiler, since the symbol table would have to encompass
the lexically enclosing environments of the function being
compiled, not just its own locals.

If such a reorganisation were undertaken, it would have
other benefits as well. It would completely solve the
speed problem, since loading of non-functions would not
be affected at all. It would also enable access to
intermediate-level variables in constant time, since the
compiler would know at which lexical level each variable
resides.

Incidentally, there is an analogous problem with the
function->method transformation: it is impossible to have
a class variable which holds a function. If you assign
a function to a class attribute and read it back again,
you get either an unbound_method or a bound_method (depending
on whether you fetch it through the class or an instance).
This piece of "magic" frequently trips people up.

One final note: I never intended that the existence of
local_function objects should be hidden from users. If
implemented, the mechanism would have to be clearly
explained in the manual, just like tbe bound_method
mechanism is now.

One final final note: I agree that this is a hack which
should not be necessary. If you're ever trying to choose
between implementing garbage collection or lexical closures
first, please go for garbage collection!

> Note that this is also one of the reasons why I'm feeling uneasy with
> the stackless Python patch set ... it uses a magic object

I wouldn't reject the whole idea of a stackless interpreter
on the basis of that implementation, though. I suspect the
author used a special object because it got the job done with
minimal changes to the existing code. There are other ways to
signal to the interpreter what is to be done next.

> (another reason is that it can't be
> done in JPython so it can't be made a part of the language standard)

Has supporting JPython become an official guiding principle
behind mainstream Python development? That may turn out to be
somewhat restrictive in the long term.

Greg