Using defaultdict as globals/locals for eval()

Dear Python developers, I have a question about "dict" subclasses and eval() that I would kindly like to ask. The problem I have is the following: For a web content system in Python, I am using Python's eval() function to run code embedded in templates. Currently, the code failes with a NameError if a variable is referenced inside the code that is not defined in the locals or globals dictionary. Unfortunately, this is a rather common situation: one often uses variables (like error codes) that aren't set all the time. Intializing these variables is a tedious job so a default value would suit me nicely. I created a derived class of the standard `dict' that fills in a default value when a key is not found. This is exactly the same as Guido describes in his "descintro" paper. I tried to use this dictionary as the "globals" parameter with eval(). As Guido already describes in his paper, this doesn't work. Quoting Guido: "The interpreter uses an internal function to access the dictionary, which bypasses our __getitem__() override. I admit that this can be a problem (although it is only a problem in this context, when a dict subclass is used as a locals/globals dictionary); it remains to be seen if I can fix this without compromising performance in the common case." Is there a solution to this problem in sight? Or altenately, is there a way I can find out which variables are used inside a compiled code block so I can initialize non-specified variables? I have a vague memory that the nested scopes feature has to determine at compile time which variables are being used in a code block. Thanks for your time! Greetings, Geert Jansen

Geert Jansen <geertj@boskant.nl> writes:
Is there a solution to this problem in sight?
No, although contributions are welcome.
Sure: you can parse the code, build an AST tuple (or list), and traverse that. This, of course, is off-topic for python-dev. Regards, Martin

The solution appears simple (use PyObject_GetItem etc. instead of PyDict_GetItem) but would cause serious performance hits: the dict API and its implementatio are highly optimized for this; PyObject_GetItem would add several levels of function calls, *plus* more reference count handling and exception handling. Oren Tirosh's global/builtin lookup speedup would be impossible. (Oren, how's that coming?) In addition, I'm concerned that this would become a popular tool to hack Python's semantics in all sorts of ways. While hacking the semantics is essential for some situations, in most cases this causes more problems than it solves because you can't use standard tools like PyChecker when you change something as fundamental as how variable references are resolved. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, Oct 28, 2002 at 09:14:53AM -0500, Guido van Rossum wrote:
Maybe eval() could check whether the "locals" and "globals" are plain dictionaries, or dictionary derived objects. This check would have to be done once, just after eval() is called. Depending on the result, PyDict_GetItem or PyObject_GetItem is used further on. Hmm, this sounds too trivial. I bet there are complications... Geert

Geert Jansen <geertj@boskant.nl> writes:
Is there a solution to this problem in sight?
No, although contributions are welcome.
Sure: you can parse the code, build an AST tuple (or list), and traverse that. This, of course, is off-topic for python-dev. Regards, Martin

The solution appears simple (use PyObject_GetItem etc. instead of PyDict_GetItem) but would cause serious performance hits: the dict API and its implementatio are highly optimized for this; PyObject_GetItem would add several levels of function calls, *plus* more reference count handling and exception handling. Oren Tirosh's global/builtin lookup speedup would be impossible. (Oren, how's that coming?) In addition, I'm concerned that this would become a popular tool to hack Python's semantics in all sorts of ways. While hacking the semantics is essential for some situations, in most cases this causes more problems than it solves because you can't use standard tools like PyChecker when you change something as fundamental as how variable references are resolved. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Mon, Oct 28, 2002 at 09:14:53AM -0500, Guido van Rossum wrote:
Maybe eval() could check whether the "locals" and "globals" are plain dictionaries, or dictionary derived objects. This check would have to be done once, just after eval() is called. Depending on the result, PyDict_GetItem or PyObject_GetItem is used further on. Hmm, this sounds too trivial. I bet there are complications... Geert
participants (5)
-
Geert Jansen
-
Guido van Rossum
-
martin@v.loewis.de
-
Neil Schemenauer
-
Raymond Hettinger