How to load new class definitions at runtime?

[Dave Brueck]

Carlos Ribeiro wrote:
> I'm looking for ways to load new class definitions at runtime (I'm not
[snip]
> Now, talking about executing arbitrary code retrieved from a database
> isn't exactly a safe proposition. But I can't see any other way to
> make it work. I'm now looking for more ideas on how do other people
> solved the same problem in other projects. Pointers are welcome.

We didn't use a database, but just stored them on disk - we considered the app's 
directory to be within our security wall, and made sure it was pretty well 
locked down. Basically, if it is compromised, you can already get at all the 
secret data, so sneaking in evil Python code doesn't really give you any 
advantage. YMMV of course.

As for class definitions, we used the 'new' module to create updated modules 
(and also replaced the previous entry in sys.modules - not sure if any other 
bases need covering, but it hasn't broken yet ;-) ) and then established two 
conventions for "hot swappable" modules:

1) No module level code. Modules can have optional Setup() and Teardown() 
functions that get called if present (the most typical use case is to pass state 
from the old version of the module to the new version - the return value from 
Teardown gets passed to the new module's Setup function).

2) Don't store references to hot swap modules for long periods of time. When you 
need to use one of thoes modules, you get a reference to a module by calling an 
API (e.g. GetModule() ) and it also takes care of updating the module if it has 
changed on disk. So we had lots of code like:

def foo(arg):
   someLib = GetModule('someLib')
   someLib.biff()
   ...
   someLib.bar()

etc. - i.e. the reference is kept for relatively short amounts of time so that 
if the module gets reloaded, no code will be using the old version for very long.

Overall it has worked really well; we try to avoid as much as possible modules 
that maintain their own state because reloading can be tricky (in our simplistic 
implementation you can lose state changes if one occurs between Teardown and 
Setup - fortunately for us we haven't had any cases where that matters).

-Dave