execfile() and locals()

[fons at kokkinizita.net]

On Sun, Aug 15, 2010 at 11:24:25PM +0000, Steven D'Aprano wrote:

> On Sun, 15 Aug 2010 23:21:51 +0200, fons wrote:
> 
> > The documentation on execfile() and locals() makes it clear that code
> > executed from execfile() can not modify local variables in the function
> > from wich execfile() was called. Two questions about this:
> > 
> > 1. Is there some way to circumvent this limitation (apart from
> > explicitly copying variables to/from a dictionary passed as locals to
> > execfile()) ?
> 
> Chances are very good that if you're using exec or execfile inside a 
> function, you probably don't need to.

In this case that would probably be true (see below).

> As an optimization, local variables don't live inside a dict namespace. 
> The space for those locals is allocated at compile time, and locals() 
> returns a copy of the variables in a dict.
> 
> However, the action of exec is a little more subtle, and mysterious, as 
> seen by this example in Python 3.1:
> 
> >>> def f():
> ...     x = 1
> ...     print(locals())
> ...     exec("x=2; y=0")
> ...     print(locals())
> ...
> >>>
> >>> f()
> {'x': 1}
> {'y': 0, 'x': 1}
> 
> This shows that exec was able to create a new local variable, but not 
> modify an existing one -- this is completely unintuitive to me. I would 
> have guessed the opposite. And worse:
> 
> >>> f.__code__.co_varnames
> ('x',)
> >>> f.__code__.co_nlocals
> 1
> 
> So where does the newly-created local `y` live, if not in a dict 
> namespace and not in the usual variable slots?
> 
> Curiouser and curiouser... 

Indeed... and I tried your example also in 2.6, and there it 'just works'.

I've been reading a bit on execfile() and friends on various locations
on the web, and there seems to be some controversy about it. One poster
even wrote 'If you need execfile() you have a design problem'. I don't
agree - in this case I use an interpreted language exactly to be able
to do such things as made possible by execfile().

The application is a GUI one used to control electronic musical instruments
and audio processors in a 'live performance' situation. Its main window
shows a collection of buttons, rotary controls, sliders, etc., which trigger
things when used.

For example, clicking on a button could start a timed sequence of actions,
each action being e.g. a message sent to a synthesis program or an audio
processor, and/or some changes to the GUI itself.

Such a sequence could be quite complex, defining it may require loops
and conditions, or even interaction with other sequences, so the natural
way is to define it as code and not as data. Users of the system are
assumed to know basic Python, without being experts or expected to 
understand things such as the ones we are discussing.

Sequences will run for some time, many of them can run at the same time,
so each of them will be executed in a separate thread if required.

A basic requirement for the system is that all actions can be redefined
while the program is running, hence the need for execfile() or something
similar. Shift-click on a GUI item presents a menu with options to modify
its actions, one of them is to edit the file defining the code associated
with the item.

Now there are two ways to use execfile() or equivalent:

1. Actually execute the file when e.g. a button is clicked.
2. Execute it when the action is defined (i.e. when the user
has finished editing the file). In the case the file should
define a function which will be executed when triggered.

I'll probably use the second way - the function will be 
precompiled, and most errors will show up before it is
actually executed.

Ciao,

-- 
FA

There are three of them, and Alleline.