Multiple interpreters retaining huge amounts of memory

[Bronner, Gregory]

On the off chance that anyone is still following this:
I've got a relatively simple example of a program that loads 100
interpreters (sequentially) which all load the same swig module, do
something trival, and exit.

Each cycle leaks (or loses) 132k, which is a significant hit -- in my
real program the hit is around 800k/interpreter.

I ran it through purify (after rebuilding python with the puremodule, no
pymalloc, no optimization, no threads, and debugging), and while the
results are somewhat ambiguous, it appears that Py_EndInterpreter isn't
cleaning up:


A) The site module
B) The builtins module 

Is there some way to properly clean these up prior to the end of
Py_EndInterpreter? It seem to zap everything wether or not it can
correctly clean up the interpreter/modules, and after it runs, all
pointers have been destroyed.


Thanks



 

-----Original Message-----
From: Rhamphoryncus [mailto:rhamph at gmail.com] 
Sent: Thursday, February 07, 2008 12:38 PM
To: python-list at python.org
Subject: Re: Multiple interpreters retaining huge amounts of memory

On Feb 2, 10:32 pm, Graham Dumpleton <Graham.Dumple... at gmail.com>
wrote:
> The multi interpreter feature has some limitations, but if you know 
> what you are doing and your application can be run within those 
> limitations then it works fine.

I've been wondering about this for a while.  Given the severe
limitations of it, what are the use cases where multiple interpreters do
work?  All I can think of is that it keeps separate copies of loaded
python modules, but since you shouldn't be monkey-patching them anyway,
why should you care?


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

This message is intended only for the personal and confidential use of the designated recipient(s) named above.  If you are not the intended recipient of this message you are hereby notified that any review, dissemination, distribution or copying of this message is strictly prohibited.  This communication is for information purposes only and should not be regarded as an offer to sell or as a solicitation of an offer to buy any financial product, an official confirmation of any transaction, or as an official statement of Lehman Brothers.  Email transmission cannot be guaranteed to be secure or error-free.  Therefore, we do not represent that this information is complete or accurate and it should not be relied upon as such.  All information is subject to change without notice.

--------
IRS Circular 230 Disclosure:
Please be advised that any discussion of U.S. tax matters contained within this communication (including any attachments) is not intended or written to be used and cannot be used for the purpose of (i) avoiding U.S. tax related penalties or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein.