Repost: execfile() confusion

[Tim Peters]

[Nathaniel Gray, being unreasonable <wink>]
> ...
> Here's a slightly modified version of the files that
> will perhaps make my confusion clear:

Except there's no point to clearing it up:  you're chasing accidents, and
even if you figure out why they're happening, they may change in the next
release.  I added this text to the library ref man earlier today, under the
execfile() docs:

  \strong{Warning:} The default \var{locals} act as described for function
  \function{locals()} below:  modifications to the default \var{locals}
  dictionary should not be attempted.  Pass an explicit \var{locals}
  dictionary if you need to see effects of the code on \var{locals} after
  function \function{execfile()} returns.  \function{execfile()} cannot
  be used reliably to modify a function's locals.

> #### test2.py ####
> print 'test2 locals:', locals().keys()
> x = 'spam'
> print 'Post-assignment test2 locals:', locals().keys()
>
> #### test.py ####
> def doit():
>     joe = 99
>     #x = 'not spam'  # THE MAGIC LINE

When the line is commented out, locals() has only 'joe' as a key by the time
you call execfile(); when it's not commented out, locals has both 'joe' and
'x' as keys by then.  Both thanks to the PyFrame_FastToLocals() function in
the implementation.  However, execfile() is just a function, like len() or
math.sin(), and not at all like the exec statement in this respect:  the
corresponding PyFrame_LocalsToFast() is never called except after the "from
Module import *" and "exec" *statements*.  Indeed, exec used to be a builtin
function, and it was turned into a stmt primarily so that the
hyper-expensive tricks needed to materialize dicts back into function locals
would work reliably for exec.  This isn't done for execfile(), and cannot be
done unless execfile turns into a stmt too; but, as I explained earlier,
execfile already works fine for its intended purpose (rare use at module
scope, or when passing explicit namespace dicts).

The consequence is that execfile effectively gets a write-only snapshot of
the function locals at the time of call, and changes to that dict are never
reflected back into the function's true local variables (although they do
show up in the *dict* returned by locals() -- which is created at most once
per function invocation, and shared by all calls to locals() until the
function returns -- today.  Maybe not tomorrow.).

>     execfile( 'test2.py' ) # x = 'spam'
>     print 'x is now: %(x)s' % locals()   #No compiler tricks allowed!
> doit()
>
> Trial 1.  Here's the output with the "magic" line commented:
>         test2 locals: ['joe']
>         Post-assignment test2 locals: ['x', 'joe']
>         x is now: spam

Nothing surprising there, right?  The execfile'd code stuffs 'x'->'spam'
into the locals() dict.  But that dict is not written back into the
function's true locals, so "print x" would yield a NameError.

> Trial 2.  And when you uncomment it:
>         test2 locals: ['joe', 'x']
>         Post-assignment test2 locals: ['joe', 'x']
>         x is now: not spam

And nothing surprising there either <wink>.

> I just can't seem to find a consistent story here.

It's because you're confusing locals() with a function's local vrbls.
They're not the same thing:  locals() is just a one-way snapshot, most of
the time, and execfile flatly does not support the kind of tricks you're
trying to play.  Spend a tenth of this energy thinking of a clean way to do
whatever it is you're after, and you could enjoy life again.

> ...
> By the way, this behavior is the same in Python 1.4, so if it's a
> bug it's an old one.

Most parts of the local/global behavior are the same; other parts have
changed; what's changed the most is all the text added to the docs telling
people to stop doing things that Guido never believed anyone in their right
mind would even attempt <wink>.

execfile-is-to-exec-as-delicious-is-to-del-ly y'rs  - tim