[Python-Dev] scope-collapse

[Jim Jewett]

[Jim Jewett]
>> >> (2)  Add a way to say "Make this function I'm calling use *my* locals
>> >> and globals."  This seems to meet all the agreed-upon-as-good use
>> >> cases, but there is disagreement over how to sensibly write it.

[Guido]
>> > What happens to names that have a
>> > different meaning in each scope?

[Jim]
>> Programming error.  Same name ==> same object.

> Sounds like a recipe for bugs to me. At the very least it is a total
> breach of abstraction, which is the fundamental basis of the
> relationship between caller and callee in normal circumstances.

Yes.  Collapsing scope is not a good idea in general.  But there
is no way to avoid (some version of) it under the thunk or even
the resource-manager suggestions.  I interpret that to mean that
either 
    The code *can* get ugly and you rely on conventions.
or 
    These constructs should not be added to the language.

The pretend-it-is-a-generator proposals try to specify that only 
certain names will be shared, in only certain ways.  That might
work (practicality beats purity) but I suspect it will evolve into a
wart.  It won't be quite strong enough to solve the problem 
completely (particularly with multiple blocks), but it will be 
strong enough to obfuscate when mishandled.

>> Yes.  The callee does not even get to see its normal namespace.
>> Therefore, the callee does not get to use its normal name resolution.

> This seems to me to repeat all the mistakes of the dynamic scoping 
> of early Lisps (including GNU Emacs Lisp I believe).

With one exception -- the caller must state explicitly that the collapse
it happening, and even then, it only goes down one level at a time.
Still an ugly tool, but at least not an ugly surprise.

> It really strikes me as an endless source of errors that these
> blended-scope callees (in your proposal) are ordinary
> functions/methods, which means that they can *also* be called without
> blending scopes. Having special syntax to define a callee intended for
> scope-blending seems much more appropriate (even if there's also
> special syntax at the call site).

This might well be a good restriction.  The number of times it causes
annoyance (why do I have to code this twice?) should be outweighed
by the number of times it saves a surprise (oops -- those functions 
both defined the same keyword argument).

>> If the name normally resolves in locals (often inlined to a tuple, today),
>> it looks in the shared scope, which is "owned" by the caller.  This is
>> different from a free variable only because the callee can write to this
>> dictionary.

> Aha! This suggests that a blend-callee needs to use different bytecode
> to avoid doing lookups in the tuple of optimized locals

Yes.  I believe the translation is mechanical, so that the compiler could
choose (or create) the right version based on the caller, but ... I agree
that making them a separate kind of callable would simplify things.

> (I meant this to read "does not make sense for all callables".)
> (And I presume you read it that way. :-)

nah... I think it takes special justification to do use anything but
duck typing in python.  If functions can intersperse with boilerplate,
than other callables (and even other suites, such as class definitions) 
should be able to do the same.  But I also agree that it makes sense 
to wait until it can be done sensibly.

Just as @decorator only applies to functions (even if the specific
decorator could accept something else), this interspersing should
probably not apply to non-functions until the "but I can't use a 
function" use cases are clear.

>> For a given simple algorithm, interpeted python is generally slower
>> than compiled C, but we write in python anyhow -- it is fast enough,
>> and has other advantages.  The same is true of anything that lets
>> me not cut-and-paste.

> Whatever. Any new feature that causes a measurable slowdown for code
> that does *not* need the feature has a REALLY hard time getting
> accepted,

Agreed.  But the scope-collapse penalty is restricted to the caller
(which ordered the collapse) and the immediate callees during
the collapsed call.  

>> I'm not sure I understand this.  The preferred way would be
>> to just stick the keyword before the call.  Using 'collapse', it
>> would look like:
                                  # (Added comment to make the ugliess
potential more explicit)
>>     def foo(b):         # Yes, parameters are in the namespace.          
>>         c=a
>>     def bar():
>>         a="a1"
>>         collapse foo("b1")
>>         print b, c        # prints "b1", "a1"
>>         a="a2"
>>         foo("b2")        # Not collapsed this time
>>         print b, c        # still prints "b1", "a1"

> I'm trying to sensitize you to potential uses like this:

> def bar():
>    a = "a1"
>    print collapse foo("b1")

In this case, it would print None, as foo didn't bother to return anything.

> but I suppose you could treat it like the 'global' keyword

>>     def bar():
>>         a="a1"
>>         collapse foo   # forces foo to always collapse when called within bar
>>         foo("b1")
>>         print b, c        # prints "b1", "a1"
>>         a="a2"
>>         foo("b2")        # still collapsed
>>         print b, c        # now prints "b2", "a2"

> Would make more sense if the collapse keyword was at the module level.

???  Are you suggesting that everything defined in the module must live
in a single namespace, just because the collapse was wanted in one place?

-jJ