Am 20.08.2013 19:39, schrieb PJ Eby:
On Tue, Aug 20, 2013 at 12:39 PM, Thomas Heller <theller@ctypes.org> wrote:
Ok, now I understand. But the zipfile could contain a loader-module for each extension which does something like this (this example extracts and loads 'bz2.pyd'): ...
(py2exe for Python 3, which is work in progress, uses this approach)
Setuptools has also done this since the egg format was developed, but it has some well-known problems, which unfortunately your example has worse versions of. ;-)
The code I posted was some 'pseudocode' how to extract and import an extension from a zip-file without coding it in C ;-). For example, TEMPDIR is not the usual TEMP directory, instead py2exe will use a per-process temp directory and cleanup after process exit. So, at least some of the problems you list below should be solved or solvable.
Setuptools takes the approach of keeping a per-user cache directory (so that cleanup isn't required, and so there are no security issues where somebody can replace a tempfile between you writing it and importing it), and it uses a unique subdirectory per egg so that different (say) bz2.pyd files can't conflict with each other. Even with these adjustments, Unix users frequently run into issues where the user a process is running as doesn't have access to a suitable cache directory, and so it's a common complaint about the use of zipped eggs.
I thought that at one point you (Thomas) had come up with a way to load modules into memory from a zipfile without needing to extract them. Was that you? If so, how did that work out? (ISTR that there was some sort of licensing issue, too.)
Yes, that was me. It worked out so-so, fine for extensions from wx-python and numpy, for example, not so good for others. Until recently it did only work for win32, not win64, but this will be fixed. And, of course, it is windows only! The code it is based on is MPL2.0 licensed: https://github.com/fancycode/MemoryModule Thomas