Modules as global namespaces rather than dicts
This is an idea I have been playing with and seems to hold some promise. I think we should use a module instance as the standard global namespace rather than directly using its dict. I have a prototype version of CPython that does this, not working 100% yet though. Major changes: - In the frameobject, remove f_builtins and f_globals, add f_namespace that refers to the module. Make f_globals and f_builtins into properties. - Change ceval to use f_namespace, rather than carrying around globals and builtins. Change functions that take globals as a dict so they also accept a module object. - Change the module object to keep track of the builtins for it. - Change funcobject (e.g. PyFunction_NewWithQualName) to accept 'globals' as a module object - When given a dict and we now expect a module object, create an anonymous module to wrap it. This part is a bit tricky due to reference cycles and speed requirements. However, my hope is that if the internals of CPython can be made to pass modules instead of dicts around, these anonymous modules will become a rare case and so won't matter if they are a little slower. So, what is the purpose of all this trouble? - I believe quite a lot of Python internals can be simpler. For example, importlib is complicated by the fact that a dict is passed around when most of the logic would prefer to have the module. Grubbing in sys.modules to lookup the module object is ugly. The expression "exec(code, module)" is elegant to me. - I believe it will be possible to make global variable access work similar to fast locals. I.e. have each module contain a indexed list of global names, and use an array lookup rather than a dict lookup in the normal case. For backwards compatibility, my idea is to keep a flag on the module object noting if fast global behavior is okay. If someone grabs the module dict, e.g. using module.__dict__ , vars() or frame.f_globals then we clear the flag and revert to the slow dict behavior. Also, if a piece of code is executed in a different namespace, we have to revert to the slow case. That does not seem so hard to do. - I want to have properties for module globals and have them work from within the module. I.e. changing something from a global variable to a property should not require changes to other module code. I.e. LOAD_GLOBAL should trigger a property get. Regards, Neil
On 15 November 2017 at 06:34, Neil Schemenauer <nas-python-ideas@arctrix.com
wrote:
So, what is the purpose of all this trouble?
- I believe quite a lot of Python internals can be simpler. For example, importlib is complicated by the fact that a dict is passed around when most of the logic would prefer to have the module. Grubbing in sys.modules to lookup the module object is ugly. The expression "exec(code, module)" is elegant to me.
I like the idea in principle, but highlighting a particular backwards compatibility pain point: one of the complications in importlib is that we promise to keep the "sys.modules[__name__] = some_other_object" idiom working. That means the need to do that check exists regardless of whether importlib is passing the module itself around, or the module's dict. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Nov 15, 2017 at 8:44 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I like the idea in principle, but highlighting a particular backwards compatibility pain point: one of the complications in importlib is that we promise to keep the "sys.modules[__name__] = some_other_object" idiom working. That means the need to do that check exists regardless of whether importlib is passing the module itself around, or the module's dict.
Another point, perhaps more difficult to address: Would for instance globals() then return a module instead of a dict/mapping? ––Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +
On 2017-11-15, Koos Zevenhoven wrote:
Another point, perhaps more difficult to address: Would for instance globals() then return a module instead of a dict/mapping?
For compatibility, it would definitely have to return a dict. As a result, calling globals() would cause the "fast globals" flag to be cleared. My hope is that getting a reference to the module dict from user code is the exception rather than the norm. I.e. most modules would never have globals() called within their code. If it is too common, the flag will be cleared on most modules and we won't gain much speed.
14.11.17 22:34, Neil Schemenauer пише:
This is an idea I have been playing with and seems to hold some promise. I think we should use a module instance as the standard global namespace rather than directly using its dict. I have a prototype version of CPython that does this, not working 100% yet though.
Wouldn't this harm a performance? Especially after implementing PEP 562.
participants (4)
-
Koos Zevenhoven -
Neil Schemenauer -
Nick Coghlan -
Serhiy Storchaka