Re: [Python-checkins] python/dist/src/Python import.c,2.210,2.211
On vrijdag, nov 29, 2002, at 21:47 Europe/Amsterdam, jvr@users.sourceforge.net wrote:
Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1:/tmp/cvs-serv20813/Python
Modified Files: import.c Log Message: Slightly improved version of patch #642578: "Expose PyImport_FrozenModules in imp". This adds two functions to the imp module: get_frozenmodules() and set_frozenmodules().
Something that's been bothering me about frozen modules in the
classical sense (i.e. those that are stored in C static data
structures) is that the memory used by them is gone without any chance
at recovery. For big frozen Python programs that are to be run on small
machines this is a waste of precious memory.
With modules "frozen" with set_frozenmodules you could conceivably free
the data again after it has been imported (similar to what
MacPython-OS9 does with modules "frozen" in "PYC " resources).
Would that be worth the added complexity?
--
- Jack Jansen
Jack Jansen wrote:
On vrijdag, nov 29, 2002, at 21:47 Europe/Amsterdam, jvr@users.sourceforge.net wrote:
Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1:/tmp/cvs-serv20813/Python
Modified Files: import.c Log Message: Slightly improved version of patch #642578: "Expose PyImport_FrozenModules in imp". This adds two functions to the imp module: get_frozenmodules() and set_frozenmodules().
Something that's been bothering me about frozen modules in the classical sense (i.e. those that are stored in C static data structures) is that the memory used by them is gone without any chance at recovery. For big frozen Python programs that are to be run on small machines this is a waste of precious memory.
With modules "frozen" with set_frozenmodules you could conceivably free the data again after it has been imported (similar to what MacPython-OS9 does with modules "frozen" in "PYC " resources).
Would that be worth the added complexity?
Allocating the data on the heap would prevent sharing the static data between processes (provided you have multiple of the running), so if you want to add that feature, please make it an option and not the default. mxCGIPython heavily relies on the fact that frozen modules are shared between processes. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
M.-A. Lemburg wrote:
Allocating the data on the heap would prevent sharing the static data between processes (provided you have multiple of the running), so if you want to add that feature, please make it an option and not the default.
Don't worry: I think extending the freeze mechanism even further is a waste of time; I think Jack's concerns could also be addressed with an import hook. Also, about the checkin that that triggered Jack's remark: I'm very likely to back out the imp.[sg]et_frozenmodules patch as what I wanted to achieve would be a much better candidate for the zip import mechanism, which is sadly still pending. I'll have a closer look at that and see whether I can help getting this in before 2.3.a1. Just
Jack Jansen
Something that's been bothering me about frozen modules in the classical sense (i.e. those that are stored in C static data structures) is that the memory used by them is gone without any chance at recovery.
With virtual memory, that's probably not a problem, since it'll get paged out after it's been used. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
participants (4)
-
Greg Ewing -
Jack Jansen -
Just van Rossum -
M.-A. Lemburg