[Python-Dev] variable name resolution in exec is incorrect

[Colin H]

By hardest to induce I mean the default compile exec(code_str, {}, {})
would still be class namespace, but it's pretty insignificant.

On Fri, May 28, 2010 at 12:32 AM, Colin H <hawkett at gmail.com> wrote:
> This option sounds very promising - seems right to do it at the
> compile stage - i.e. compile(code_str, name, "closure") as you have
> suggested.  If there were any argument against, it would be that the
> most obvious behaviour (function namespace) is the hardest to induce,
> but the value in knowing you're not breaking anything is pretty high.
>
> Cheers,
> Colin
>
> On Thu, May 27, 2010 at 4:42 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On 27/05/10 10:38, Guido van Rossum wrote:
>>>
>>> On Wed, May 26, 2010 at 5:12 PM, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>>>>
>>>> Lexical scoping only works for code that is compiled as part of a single
>>>> operation - the separation between the compilation of the individual
>>>> string
>>>> and the code defining that string means that the symbol table analysis
>>>> needed for lexical scoping can't cross the boundary.
>>>
>>> Hi Nick,
>>>
>>> I don't think Colin was asking for such things.
>>
>> Yes, I realised some time after sending that message that I'd gone off on a
>> tangent unrelated to the original question (as a result of earlier parts of
>> the discussion I'd been pondering the scoping differences between exec with
>> two namespaces and a class definition and ended up writing about that
>> instead of the topic Colin originally brought up).
>>
>> I suspect Thomas is right that the current two namespace exec behaviour is
>> mostly a legacy of the standard scoping before nested scopes were added.
>>
>> To state the problem as succinctly as I can, the basic issue is that a code
>> object which includes a function definition that refers to top level
>> variables will execute correctly when the same namespace is used for both
>> locals and globals (i.e. like module level code) but will fail when these
>> namespaces are different (i.e. like code in class definition).
>>
>> So long as the code being executed doesn't define any functions that refer
>> to top level variables in the executed code the two argument form is
>> currently perfectly usable, so deprecating it would be an overreaction.
>>
>> However, attaining the (sensible) behaviour Colin is requesting when such
>> top level variable references exist would actually be somewhat tricky.
>> Considering Guido's suggestion to treat two argument exec like a function
>> rather than a class and generate a closure with full lexical scoping a
>> little further, I don't believe this could be done in exec itself without
>> breaking code that expects the current behaviour. However, something along
>> these lines could probably be managed as a new compilation mode for
>> compile() (e.g. compile(code_str, name, "closure")), which would then allow
>> these code objects to be passed to exec to get the desired behaviour.
>>
>> Compare and contrast:
>>
>>>>> def f():
>> ...   x = 1
>> ...   def g():
>> ...     print x
>> ...   g()
>> ...
>>>>> exec f.func_code in globals(), {}
>> 1
>>
>>>>> source = """\
>> ... x = 1
>> ... def g():
>> ...   print x
>> ... g()
>> ... """
>>>>> exec source in globals(), {}
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "<string>", line 4, in <module>
>>  File "<string>", line 3, in g
>> NameError: global name 'x' is not defined
>>
>> Breaking out dis.dis on these examples is fairly enlightening, as they
>> generate *very* different bytecode for the definition of g().
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>> ---------------------------------------------------------------
>>
>