[issue26363] __builtins__ propagation is misleading described in exec and eval documentation
New submission from Xavier Combelle: According to my experiment in code, the current behavior of python3.5 is different that the document says. If I understand well the purpose of this behavior is to propagate the __builtins__ global constant if globals has not one. In https://docs.python.org/3.6/library/functions.html#eval it is written "If the globals dictionary is present and lacks ‘__builtins__’, the current globals are copied into globals before expression is parsed." only the __builtins__ looks copied not all the globals In https://docs.python.org/3.6/library/functions.html#exec It is written: "If the globals dictionary does not contain a value for the key __builtins__, a reference to the dictionary of the built-in module builtins is inserted under that key." it looks like it is not a reference to the built-in module builtin, but a reference to __builtin__ global ---------- title: builtins propagation is misleading described in exec and eval documentation -> __builtins__ propagation is misleading described in exec and eval documentation versions: +Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Changes by Julien <python@mandark.fr>: ---------- nosy: +sizeof _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Julien Palard added the comment: Hi Xavier, thanks for reporting, Your first point is right, the implementation being: if (PyDict_GetItemString(globals, "__builtins__") == NULL) { if (PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins()) != 0) return NULL; } See proposed diff. For the second point, it looks right to me in the documentation, literally: "A reference to the dictionary of the builtin module builtins is inserted", so: It's a dict:
exec('print(type(globals()["__builtins__"]))', {}) <class 'dict'>
It's the reference to the dict of the builtins module:
exec('globals()["__builtins__"]["len"] = "foo"', {}) len 'foo'
If you still think there's inconsistencies, please provide some code to reproduce it. ---------- keywords: +patch Added file: http://bugs.python.org/file45652/issue26363.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Xavier Combelle added the comment: It is not the dictionary of builtin module, which is inserted in , but the current __builtin__ global which happen to be normally the dictionnary of builtin. Hence in the following code, the builtins propagation works has expected.
eval("""eval('spam("hello world")',{})""",{"__builtins__":{"eval":eval,"spam":print}}) hello world
---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Julien Palard added the comment: Hi Xavier,
It is not the dictionary of builtin module, which is inserted in , but the current __builtin__ global
It looks wrong, I'll even say the exact contrary: It _is_ the dictionary of builtin module which is inserted in, not the current __builtin__ global, with this proof: $ ./python Python 3.7.0a0 (default, Nov 29 2016, 11:20:17) [GCC 5.4.1 20161019] on linux Type "help", "copyright", "credits" or "license" for more information.
print(id(__builtins__), id(__builtins__.__dict__)) 140325888797784 140325888840368 eval("""print(id(__builtins__))""", {}) 140325888840368
the current __builtin__ global which happen to be normally the dictionnary of builtin.
That's not necessarily true, according to [the builtins doc](https://docs.python.org/dev/library/builtins.html):
The value of __builtins__ is normally either this module or the value of this module’s __dict__ attribute.
Typically: $ ./python Python 3.7.0a0 (default, Nov 29 2016, 11:20:17) [GCC 5.4.1 20161019] on linux Type "help", "copyright", "credits" or "license" for more information.
import builtins id(builtins), id(builtins.__dict__), id(__builtins__) (139706743340120, 139706743382704, 139706743340120)
Here, __builtins__ is the module, not its dict. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Eryk Sun added the comment: As shown above, exec and eval default to calling PyEval_GetBuiltins when the globals dict doesn't define '__builtins__'. PyEval_GetBuiltins uses the current frame's f_builtins. If there isn't a current frame, it defaults to the interpreter's builtins, which should be the dict of the builtins module. If exec and eval didn't do this, the default behavior would be to create a minimal f_builtins dict for the new frame. This dict only contains a reference to None, and it doesn't get set as '__builtins__' in globals. For example: from inspect import currentframe from ctypes import pythonapi, py_object g = py_object({'currentframe': currentframe}) code = py_object(compile('currentframe()', '', 'eval')) frame = pythonapi.PyEval_EvalCode(code, g, g) >>> frame.f_builtins {'None': None} >>> frame.f_globals {'currentframe': <function currentframe at 0x7f2fa1d6c2f0>} This minimalist default isn't useful in general. exec and eval are saving people from the tedium of having to manually define a useful __builtins__ when passing a new globals. The frame object uses this __builtins__ to initialize its f_builtins. Also, it knows to look for __builtins__ as a module, as used by __main__: g = py_object({'currentframe': currentframe, '__builtins__': __builtins__}) frame = pythonapi.PyEval_EvalCode(code, g, g) >>> frame.f_builtins is vars(__builtins__) True ---------- nosy: +eryksun _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Xavier Combelle added the comment: Hi Julien, You are fully right that it is the builtin module dictionnary which is inserted in eval or exec context. However, if a "__builtins__" entry is passed to eval or exec, this builtin module dictionnary is modified hence the following work:
d={"tata":"tata"} print(eval("tata",{'__builtins__':d})) tata
---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Julien Palard added the comment: So, is there still an inconsistency in the documentation? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Xavier Combelle added the comment: not an inconsisties but in the eval documentaion nothing specify that the builtins propagate between levels updates ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Martin Panter added the comment: Xavier, you are welcome to propose your own version of the text, or build on Julien’s. See also Issue 22057, about copying all globals vs builtins. ---------- nosy: +martin.panter stage: -> patch review versions: +Python 2.7, Python 3.6, Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue26363> _______________________________________
Berker Peksag <berker.peksag@gmail.com> added the comment: Thank you for the report, Xavier. This is a duplicate of issue 22057. PR 8812 clarifies the behavior when a dictionary without a "__builtins__" key is passed as *globals* to eval(). I think that makes the opposite case much easier to understand.
eval("print(spam)", {'__builtins__': {'spam': 'eggs'}}) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'print' is not defined
eval("print(spam)", {'__builtins__': {'spam': 'eggs', 'print': print}}) eggs
Also, I never needed to pass a dictionary with a "__builtins__" key to eval() before, so I don't think it's an important detail to document. ---------- nosy: +berker.peksag resolution: -> duplicate stage: patch review -> resolved status: open -> closed superseder: -> The doc say all globals are copied on eval(), but only __builtins__ is copied type: -> behavior versions: +Python 3.8 -Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue26363> _______________________________________
participants (6)
-
Berker Peksag -
Eryk Sun
-
Julien
-
Julien Palard
-
Martin Panter
-
Xavier Combelle