[Python-Dev] Re: anonymous blocks vs scope-collapse

[Jim Jewett]

>> (2)  Add a way to say "Make this function I'm calling use *my* locals
>> and globals."  This seems to meet all the agreed-upon-as-good use
>> cases, but there is disagreement over how to sensibly write it.  The
>> calling function is the place that could get surprised, but people
>> who want thunks seem to want the specialness in the called function.

> I think there are several problems with this. First, it looks
> difficult to provide semantics that cover all the corners for the
> blending of two namespaces. What happens to names that have a
> different meaning in each scope? 

Programming error.  Same name ==> same object.  

If a function is using one of _your_ names for something incompatible,
then don't call that function with collapsed scope.  The same "problem"
happens with globals today.  Code in module X can break if module Y
replaces (not shadows, replaces) a builtin with an incompatible object.

Except ...
> (E.g. 'self' when calling a method of
> another object; or any other name clash.) 

The first argument of a method *might* be a special case.  It seems
wrong to unbind a bound method.  On the other hand, resource
managers may well want to use unbound methods for the called
code.

> Are the globals also blended?  How?

Yes.  The callee does not even get to see its normal namespace.
Therefore, the callee does not get to use its normal name resolution.

If the name normally resolves in locals (often inlined to a tuple, today), 
it looks in the shared scope, which is "owned" by the caller.  This is 
different from a free variable only because the callee can write to this 
dictionary.

If the name is free in that shared scope, (which implies that the 
callee does not bind it, else it would be added to the shared scope) 
then the callee looks up the caller's nested stack and then to the 
caller's globals, and then the caller's builtins.

> Second, this construct only makes sense for all callables; 

Agreed.  

But using it on a non-function may cause surprising results
especially if bound methods are not special-cased.

The same is true of decorators, which is why we have (at least 
initially) "function decorators" instead of "callable decorators".

> it makes no sense when the callable is implemented as
> a C function, 

Or rather, it can't be implemented, as the compiler may well
have optimized the variables names right out.  Stack frame
transitions between C and python are already special.

> or is a class, or an object with a __call__ method. 

These are just calls to __init__ (or __new__) or __call__.
These may be foolish things to call (particularly if the first
argument to a method isn't special-cased), but ... it isn't
a problem if the class is written appropriately.  If the class
is not written appropriately, then don't call it with collapsed 
scope.

> Third, I expect that if we solve the first two
> problems, we'll still find that for an efficient implementation we
> need to modify the bytecode of the called function.

Absolutely.  Even giving up the XXX_FAST optimizations would 
still require new bytecode to not assume them.  (Deoptimizing 
*all* functions, in *all* contexts, is not a sensible tradeoff.)

Eventually, an optimizing compiler could do the right thing, but ... 
that isn't the point.  

For a given simple algorithm, interpeted python is generally slower 
than compiled C, but we write in python anyhow -- it is fast enough, 
and has other advantages.  The same is true of anything that lets 
me not cut-and-paste.  

> Try to make sure that it can be used in a "statement context" 
> as well as in an "expression context". 

I'm not sure I understand this.  The preferred way would be
to just stick the keyword before the call.  Using 'collapse', it
would look like:

    def foo(b):
        c=a        
    def bar():
        a="a1"
        collapse foo("b1")
        print b, c        # prints "b1", "a1"
        a="a2"
        foo("b2")        # Not collapsed this time
        print b, c        # still prints "b1", "a1"

but I suppose you could treat it like the 'global' keyword

    def bar():
        a="a1"
        collapse foo   # forces foo to always collapse when called within bar
        foo("b1")
        print b, c        # prints "b1", "a1"
        a="a2"
        foo("b2")        # still collapsed
        print b, c        # now prints "b2", "a2"

>> [Alternative 3 ... bigger that merely collapsing scope]
>> (3)  Add macros.  We still have to figure out how to limit their obfuscation.
>> Attempts to detail that goal seem to get sidetracked.

> No, the problem is not how to limit the obfuscation. The problem is
> the same as for (2), only more so: nobody has given even a *remotely*
> plausible mechanism for how exactly you would get code executed at
> compile time.

macros can (and *possibly* should) be evaluated at run-time.  

Compile time should be possible (there is an interpreter running) and 
faster, but ... is certainly not required.

Even if the macros just rerun the same boilerplate code less efficiently,
it is still good to have that boilerplate defined once, instead of cutting 
and pasting.  Or, at least, it is better *if* that once doesn't become 
unreadable in the process.

-jJ