module extension search order - can it be changed?
While trying to figure out why so many stat() calls fail before locating a module, it occurred to me that since most modules are written in Python it makes sense to consider ".py" files before ".so" and "module.so" files. Would changing the search order be possible or would it break something? Skip
While trying to figure out why so many stat() calls fail before locating a module, it occurred to me that since most modules are written in Python it makes sense to consider ".py" files before ".so" and "module.so" files. Would changing the search order be possible or would it break something?
I don't know if it would break something, but it was certainly intended as a feature: you could have an extension and a .py module in the same directory, the .py module would be used as a fallback if the .so cannot be built. --Guido van Rossum (home page: http://www.python.org/~guido/)
Skip> [check .py before .so] Guido> I don't know if it would break something, but it was certainly Guido> intended as a feature: you could have an extension and a .py Guido> module in the same directory, the .py module would be used as a Guido> fallback if the .so cannot be built. This doesn't quite make sense to me. Instead of ".so cannot be built" did you mean ".so cannot be imported"? In any case, the current solution to my underlying problem (so many failing stat() calls during module import) appears to be to create $prefix/python23.zip and populate it with the contents of the standard library using zipfile.PyZipFile. Skip
Skip> [check .py before .so]
Guido> I don't know if it would break something, but it was certainly Guido> intended as a feature: you could have an extension and a .py Guido> module in the same directory, the .py module would be used as a Guido> fallback if the .so cannot be built.
This doesn't quite make sense to me. Instead of ".so cannot be built" did you mean ".so cannot be imported"?
No, I meant what I said. Once the file is found, it will be tried and if that fails, the other files won't be tried. The idea was that if the .so cannot be built, it won't be installed -- but the .py will be there at all times.
In any case, the current solution to my underlying problem (so many failing stat() calls during module import) appears to be to create $prefix/python23.zip and populate it with the contents of the standard library using zipfile.PyZipFile.
Right, that's what zipimport is for. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> This doesn't quite make sense to me. Instead of ".so cannot be >> built" did you mean ".so cannot be imported"? Guido> No, I meant what I said. Once the file is found, it will be Guido> tried and if that fails, the other files won't be tried. The Guido> idea was that if the .so cannot be built, it won't be installed Guido> -- but the .py will be there at all times. In which case there is no .so file to try to find, thus no "feature". >> In any case, the current solution to my underlying problem (so many >> failing stat() calls during module import) appears to be to create >> $prefix/python23.zip and populate it with the contents of the >> standard library using zipfile.PyZipFile. Guido> Right, that's what zipimport is for. Any thought about having the installation process create and populate python23.zip? It's not particularly hard. I'm willing to submit a patch for the build/install process. Skip
>> This doesn't quite make sense to me. Instead of ".so cannot be >> built" did you mean ".so cannot be imported"?
Guido> No, I meant what I said. Once the file is found, it will be Guido> tried and if that fails, the other files won't be tried. The Guido> idea was that if the .so cannot be built, it won't be installed Guido> -- but the .py will be there at all times.
In which case there is no .so file to try to find, thus no "feature".
I'm not sure if you're stil objecting. The feature is that .so wins over .py. I though I've explained clearly why this is a feature. If not, please ask a more detailed question.
>> In any case, the current solution to my underlying problem (so many >> failing stat() calls during module import) appears to be to create >> $prefix/python23.zip and populate it with the contents of the >> standard library using zipfile.PyZipFile.
Guido> Right, that's what zipimport is for.
Any thought about having the installation process create and populate python23.zip? It's not particularly hard. I'm willing to submit a patch for the build/install process.
python23.zip is good for end users of programs written in Python, but not so good for Python programmers: AFAIK it won't show source lines in tracebacks for modules loaded from the zip file. So it should be an option. That said, I think it is a desirable option. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (2)
-
Guido van Rossum -
Skip Montanaro