[Python-Dev] Re: scope-collapse (was: anonymous blocks)

[Robert Brewer]

[Jim Jewett]
> (2)  Add a way to say "Make this function I'm calling 
> use *my* locals and globals."  This seems to meet all
> the agreed-upon-as-good use cases, but there is disagreement
> over how to sensibly write it.  The calling function is
> the place that could get surprised, but people who want
> thunks seem to want the specialness in the 
> called function.

[Guido]
> I think there are several problems with this. First, it looks
> difficult to provide semantics that cover all the corners for the
> blending of two namespaces. What happens to names that have a
> different meaning in each scope?

[Jim]
> Programming error.  Same name ==> same object.

[Guido]
> Sounds like a recipe for bugs to me. At the very least it is a total
> breach of abstraction, which is the fundamental basis of the
> relationship between caller and callee in normal circumstances. The
> more I understand your proposal the less I like it.

[Jim]
> If a function is using one of _your_ names for something 
> incompatible, then don't call that function with collapsed
> scope.  The same "problem" happens with globals today.
> Code in module X can break if module Y replaces (not shadows,
> replaces) a builtin with an incompatible object.
> 
> Except ...
> (E.g. 'self' when calling a method of
> another object; or any other name clash.)
> 
> The first argument of a method *might* be a special case.  It seems
> wrong to unbind a bound method.  On the other hand, resource
> managers may well want to use unbound methods for the called
> code.

Urg. Please, no. If you're going to blend scopes, the callee should have
nothing passed to it. Why would you possibly want it when you already
have access to both scopes which are to be blended?


[Guido]
> Are the globals also blended?  How?

[Jim]
> Yes.  The callee does not even get to see its normal namespace.
> Therefore, the callee does not get to use its normal name 
> resolution.

[Guido]
> Another breach of abstraction: if a callee wants to use an imported
> module, the import should be present in the caller, not in the callee.

Yes, although if a callee wants to use a module that has not been
imported by the caller, it should be able to do so with a new import
statement (which then affects the namespace of the caller).

[Guido again]
> It really strikes me as an endless source of errors that these
> blended-scope callees (in your proposal) are ordinary
> functions/methods, which means that they can *also* be called without
> blending scopes. Having special syntax to define a callee intended for
> scope-blending seems much more appropriate (even if there's also
> special syntax at the call site).

Agreed. They shouldn't be ordinary functions at all, in my mind. That
means one can also mark the actual call on the callee side, instead of
the caller side; in other words, you wouldn't need a "collapse" keyword
at all if you formed the callee with a "defmacro" or other (better ;)
keyword. I guess if y'all find it surprising, you could keep "collapse".


[Jim]
> If the name normally resolves in locals (often inlined to a 
> tuple, today),
> it looks in the shared scope, which is "owned" by the 
> caller.  This is
> different from a free variable only because the callee can 
> write to this
> dictionary.

[Guido]
> Aha! This suggests that a blend-callee needs to use different bytecode
> to avoid doing lookups in the tuple of optimized locals, since the
> indices assigned to locals in the callee and the caller won't match up
> except by miracle.

[Guido]
> Third, I expect that if we solve the first two
> problems, we'll still find that for an efficient implementation we
> need to modify the bytecode of the called function.

[Jim]
> Absolutely.  Even giving up the XXX_FAST optimizations would
> still require new bytecode to not assume them.  (Deoptimizing
> *all* functions, in *all* contexts, is not a sensible tradeoff.)

I'm afraid I'm only familiar with CPython, but wouldn't callee locals
just map to XXX_FAST indices via the caller's co_names tuple?

Remapping jump targets, on the other hand, would be something to quickly
ban. You shouldn't be able to write trash like:

defmacro keepgoing:
    else:
        continue

[Guido]
> Try to make sure that it can be used in a "statement context"
> as well as in an "expression context".
...
> I'm trying to sensitize you to potential uses like this:
> 
> def foo(b):
>     c=a
> def bar():
>     a = "a1"
>     print collapse foo("b1")

If the callees aren't real functions and don't get passed anything, the
"sensible" approach would be to disallow expression-context use of them.
Rewrite the above to:

defcallee foo:
    c = a

def bar():
    a = "a1"
    collapse foo
    print c


Robert Brewer
MIS
Amor Ministries
fumanchu at amor.org