Best practice for new namespace (from C/API)
Posted to the python developer list but suggested I mail this here instead. --- Hi, I'm writing because I'm working on a project where the user can run scripts that don't reference a file but use internal application text data. Otherwise we are not doing anything tricky, only that the scripts should each run independently (no cruft left from the previous scripts namespace, sharing sys.modules etc is fine). Something which is unclear to me even after looking over pythonrun.c is the correct way to setup a namespace. For years we have been doing this and it seemed to work fine... PyObject *d = PyDict_New(); // new namespace PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()); PyDict_SetItemString(d, "__file__", PyString_FromString(filename)); // fake, avoids sys.argv[0] being used for warnings. /* --- snip ---*/ PyEval_EvalCode(compiled_text, d, d); Recently a developer reported a bug where pickle wasn't working, it turns out that in a few places python expects the __main__ modules namespace to match that if the running script: _pickle.c's save_global() in this case eg: >>> spam = 10 >>> print(__import__("__main__").__dict__["spam"]) ... 10 Once I found this was the problem it was simple to use __main__'s namespace however there are still things that are not clear about exactly how this should be done. Simplified code... PyObject *item, *dict= PyModule_GetDict(PyImport_AddModule("__main__")); PyDict_Clear(dict); PyDict_SetItemString(dict, "__builtins__", PyImport_AddModule("builtins")); item = PyUnicode_FromString( "__main__" ); PyDict_SetItemString( dict, "__name__", item ); Py_DECREF(item); PyDict_SetItemString(d, "__file__", PyString_FromString(filename)); // fake, avoids sys.argv[0] being used for warnings. /* --- snip ---*/ PyEval_EvalCode(compiled_text, dict, dict); Still this leaves me with the following questions... - Whats the best way to manage the namespace for running multiple scripts one after another? clear and initialize __main__'s dict each time?, keep a copy and restore it after each run? - should the original __main__ namespace be restored after running a script? - pickle expects: __import__("__main__").__dict__ == ***the current namespace***, is this apart of the python spec or just something specific to pickle? - PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) acts differently to PyDict_SetItemString(dict, "__builtins__", PyImport_AddModule("builtins")), using the PyEval_GetBuiltins() dict adds every member of __builtins__ when running: print(__import__("__main__").__dict__.keys()), rather then just showing __builtins__. -- - Campbell
participants (1)
-
Campbell Barton