On 1 September 2013 23:03, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 1 Sep 2013 11:28:36 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
* PEP 3121 with a size of "0". As above, but avoids the module state APIs in order to support reloading. All module state (including type cross-references) is stored in hidden state (e.g. an instance of a custom type not exposed to Python, with a reference stored on each custom type object defined in the module, and any module level "functions" actually being methods of a hidden object). Still doesn't support loading a *fresh* copy due to the hidden PEP 3121 module cache.
Not sure what you mean by that:
import atexit id(atexit) 140031896222680 import sys del sys.modules['atexit'] import atexit id(atexit) 140031896221400
Ah, you're right - I misremembered the exact problem that broke xml.etree.ElementTree testing. PyModule_GetState is actually fine (since that pointer is hidden state on the module object), it's only PyState_GetModule that is broken when you import a second copy. So, here, when the second import happens, it breaks the original atexit module's callbacks, even though the two callback registries are properly isolated: $ ./python Python 3.4.0a1+ (default:575071257c92+, Aug 25 2013, 00:42:17) [GCC 4.7.2 20121109 (Red Hat 4.7.2-8)] on linux Type "help", "copyright", "credits" or "license" for more information.
import atexit atexit.register(print, "Hello World!") <built-in function print> import sys del sys.modules["atexit"] import atexit as atexit2 atexit2.register(print, "Goodbye World!") <built-in function print>
Goodbye World!
So I think PEP 3121 is actually as good as we can get on the hidden state front, but the important point is that it is the *PyState_GetModule* API that can't handle fresh imports - the second import will always replace the first one. So anyone affected needs to find some other way of passing the state, like using bound methods of a hidden type rather than ordinary callables. If you have to interoperate with a C API that only accepts a C callback without allowing additional state arguments, you're going to have trouble. I think atexit serves as a good example, though - that _Py_PyAtExit call will *always* be destructive (even if you still have a reference to the original module), so there should be a way for the module to explicitly indicate to the import system "you can only create this module once, and then you're committed - unloading it and importing it again won't work properly due to side effects on the process state". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia