why is mmap a builtin module on windows?
Hi all, I want to use the mmap module from python trunk with python 2.5. On Linux I can easily replace it, as it is a dynamically loaded module. On windows it is a builtin module and I fear that I must compile python on windows (or resort to some other ugly hack) What is the reason for mmap being a builtin module? Regards, - Ralf
Ralf Schmitt wrote:
Hi all,
I want to use the mmap module from python trunk with python 2.5. On Linux I can easily replace it, as it is a dynamically loaded module. On windows it is a builtin module and I fear that I must compile python on windows (or resort to some other ugly hack)
What is the reason for mmap being a builtin module?
On Windows lots of modules are linked into the python main dll. The file PC/config.c contains a list of all modules. From the point of the maintainer it's much easier to link the modules into the main dll instead of creating standalone dlls. I also suspect that it's much faster because relocation is slow (see PC/dllbase_nt.txt). Martin or Mark can give you a better answer. Why do you want to overwrite the existing module instead of using a different name like ralfmmap? import ralfmmap as mmap sys.modules]'mmap'] = mmap Christian
On Jan 23, 2008 9:01 AM, Christian Heimes <lists@cheimes.de> wrote:
What is the reason for mmap being a builtin module?
On Windows lots of modules are linked into the python main dll. The file PC/config.c contains a list of all modules. From the point of the maintainer it's much easier to link the modules into the main dll instead of creating standalone dlls. I also suspect that it's much
faster because relocation is slow (see PC/dllbase_nt.txt). Martin or
Mark can give you a better answer.
ok
Why do you want to overwrite the existing module instead of using a different name like ralfmmap?
import ralfmmap as mmap sys.modules]'mmap'] = mmap
I thought about that (ugly hack) too. I would have to change the imports at a lot of places (and revert the changes when we switch to python 2.6). (whereas on Unix I would only have to do install the new mmap module). - Ralf
Ralf Schmitt wrote:
Why do you want to overwrite the existing module instead of using a different name like ralfmmap?
import ralfmmap as mmap sys.modules]'mmap'] = mmap
I thought about that (ugly hack) too. I would have to change the imports at a lot of places (and revert the changes when we switch to python 2.6). (whereas on Unix I would only have to do install the new mmap module).
It's not an ugly hack, it's a well known feature. Add you don't have to change a lot of places, too. It's sufficient to add the alias at the entry point of your application (the script that starts your app). Once the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets your ralfmmap. Christian
On Jan 23, 2008 9:35 AM, Christian Heimes <lists@cheimes.de> wrote:
It's not an ugly hack, it's a well known feature. Add you don't have to change a lot of places, too. It's sufficient to add the alias at the entry point of your application (the script that starts your app). Once the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets your ralfmmap.
Well, I have multiple scripts using multiple libraries using the new mmap objects. So, I would have to change those scripts (if I don't change the libraries), or change the libraries. And when I want to distribute my apps with bbfreeze (similar to py2exe), I also need to handle that import hackery. This is what I call ugly :) (at least in comparison to linux). Regards, - Ralf
Hello, Ralf Schmitt:
It's not an ugly hack, it's a well known feature. Add you don't have to change a lot of places, too. It's sufficient to add the alias at the entry point of your application (the script that starts your app). Once the alias sys.modules]'mmap'] = ralfmmap is set, every import mmap gets your ralfmmap.
Well, I have multiple scripts using multiple libraries using the new mmap objects. So, I would have to change those scripts (if I don't change the libraries), or change the libraries. And when I want to distribute my apps with bbfreeze (similar to py2exe), I also need to handle that import hackery. This is what I call ugly :) (at least in comparison to linux).
You can also modify site.py and put the hack there. -- Amaury Forgeot d'Arc
On Windows lots of modules are linked into the python main dll. The file PC/config.c contains a list of all modules. From the point of the maintainer it's much easier to link the modules into the main dll instead of creating standalone dlls. I also suspect that it's much faster because relocation is slow (see PC/dllbase_nt.txt). Martin or Mark can give you a better answer.
Actually, that *is* the current answer. That plus a remark "Contributions are welcome, as long as they a) come with a clear, objective policy on what should go into pythonxy.dll and what not, and b) automate all aspects of adding modules that should not go into pythonxy.dll according to the policy." Regards, Martin
On Jan 23, 2008 10:12 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
On Windows lots of modules are linked into the python main dll. The file PC/config.c contains a list of all modules. From the point of the maintainer it's much easier to link the modules into the main dll instead of creating standalone dlls. I also suspect that it's much faster because relocation is slow (see PC/dllbase_nt.txt). Martin or Mark can give you a better answer.
Actually, that *is* the current answer. That plus a remark "Contributions are welcome, as long as they a) come with a clear, objective policy on what should go into pythonxy.dll and what not, and
I'd say anything that is needed by "import sys, os" is a candidate for being included. Currently even the _csv module is a builtin module on windows. But then I don't know how much slower importing from a .pyd file is..
b) automate all aspects of adding modules that should not go into pythonxy.dll according to the policy."
i.e. create visual studio project files for those modules? and make them built automatically? Regards, - Ralf
Ralf Schmitt wrote:
i.e. create visual studio project files for those modules? and make them built automatically?
It's not that simple. You also have to integrate it into the build and carefully arrange the dependencies. You also have to find a free and sensible base address for the dll. At last the new modules must be integrated into the MSI installer script. Christian
On Jan 26, 2008 12:06 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
b) automate all aspects of adding modules that should not go into pythonxy.dll according to the policy."
i.e. create visual studio project files for those modules? and make them built automatically?
And adjust msi.py to package them automatically.
I have a working built environment now on windows (turns out it was rather easy). If time permits, I might have a look at it. - Ralf
participants (4)
-
"Martin v. Löwis" -
Amaury Forgeot d'Arc -
Christian Heimes -
Ralf Schmitt