[IronPython] Garbage collection of imported modules

[Dino Viehland]

It's not just tied to debug because there's also performance reasons to generate it as an uncollectible type.  In 1.x there is a -X:GenerateAsSnippets command line option which forces the modules to be collectible (you could also programmatically set Options.GenerateModulesAsSnippets to true).

If you want to do this for someone off code I'd suggest using the built-in compile or execfile functions as they'll ensure the code is collectible.

In 2.0 the story is the same bug the command line option is -X:LightweightScopes.

We could consider adding another way to reload modules which would do so in a collectible manner (e.g. clr.ReloadModule) if necessary.


-----Original Message-----
From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Christian Muirhead
Sent: Friday, July 25, 2008 10:29 AM
To: Discussion of IronPython
Subject: [IronPython] Garbage collection of imported modules

Hi guys -

If I execute the following:

###
from System.Diagnostics import Process

M = 1024 * 1024

import bigmodule

for i in range(1000):
     m = reload(bigmodule)
     print i, Process.GetCurrentProcess().PrivateMemorySize64 / M
###

(in IP 1.1.1), I see steadily increasing memory usage - the assemblies
created by reloading each module are not being garbage collected.
Talking to Giles (the boss here), he remembered a conversation with
someone at the Languages.NET conference about how some assemblies aren't
collectable, while others that are generated without debugging
information can be collected.

Looking at the source for PythonEngine I can see the
EngineOptions.ClrDebuggingEnabled property, with a comment saying
essentially the same thing.

However, when I try to use this with the following code, I still get
increasing memory usage.

###
from textwrap import dedent
from IronPython.Hosting import PythonEngine, EngineOptions
from System.Diagnostics import Process

M = 1024 * 1024

o = EngineOptions()
o.ClrDebuggingEnabled = False
p = PythonEngine(o)
p.Sys.path.append('.')

for i in range(1000):
     p.Execute(dedent('''\
     import bigmodule
     reload(bigmodule)
     '''))
     print i, Process.GetCurrentProcess().PrivateMemorySize64 / M
###

So the assemblies generated from compiling the imported module are still
not collectable. (I've also tried running the first script in an
interpreter (with ClrDebuggingEnabled off) embedded in a C# application,
to make sure there's nothing untoward happening because of nested
PythonEngines, but I see the same behaviour).

Is there any way to have these modules collected? Is the situation the
same in IP 2.*?

Thanks,
Christian

--
Christian Muirhead
Resolver Systems
christian.muirhead at resolversystems.com
We're hiring! http://www.resolversystems.com/jobs/


Resolver Systems Ltd
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79 Registered in England and Wales as company
number 5467329.
Registered address: 843 Finchley Road, London NW11 8NA, UK
_______________________________________________
Users mailing list
Users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com