[issue8098] PyImport_ImportModuleNoBlock() may solve problems but causes others.
report at bugs.python.org
Tue Jul 13 04:26:26 CEST 2010
Graham Dumpleton <Graham.Dumpleton at gmail.com> added the comment:
Nick, there is no main module in the same way there is when using the Python command line. Ie., there is no module that has __name__ being __main__. The closest thing is the WSGI script file which holds the application object for the WSGI application. There could technically be many of these for the one (sub)interpreter. These WSGI script files are already imported as a module with magic name based on file system path to the WSGI script file and so the module import lock would already be held while doing so.
The specific situation which is coming up is not related to the import of those script files but after the WSGI script file has already been loaded. That is, when threads from an external thread pool are later calling into the WSGI application to handle individual HTTP requests to the WSGI application. Being multithreaded, two such HTTP requests could be handled at the same time. If one of those threads lazily imports a Python module and thus acquires the module import lock while doing so, and at the same time another thread calls into any of the various Python library functions which internally use PyImport_ImportModuleNoBlock() then you will get the exception described.
Because it relates to what multiple threads are doing at the same, then it is totally unpredictable as whether one can get hit by this problem. This is why the problem is so frustrating. Your code could work fine most of the time then all of a sudden you could have one request which hits this problem if it is the first time to call these functions and another thread just happens to be importing a totally unrelate module at the same time.
So, that is the more in depth problem that is occurring in practice. I boiled it down to the simple example so people could see it and reproduce it independent of mod_wsgi.
Python tracker <report at bugs.python.org>
More information about the Python-bugs-list