[Python-ideas] Enhance reload

Joseph Maurer clarkksv at yahoo.com
Sun Sep 30 16:57:09 CEST 2007

Ok, my idea of a temporary "replace" attribute wrapped around a reload-like function is not a good idea for Python given that things can be dynamically added to modules at any time.

Here is my original high level design:

The way I implemented this feature in Slick-C is with indirection. In Python terms, this means that a separate data structure that isn't reference counted holds the method/function object data.  The method/function object is changed to just contain a pointer to it. The data structure which holds all method/function data should probably be a non-reference counted dictionary.  When a function is deleted, it's name remains in the dictionary but the entry needs to be changed to indicate that it is "null/invalid".  When a deleted function is called, an exception should be raised. Adding a function/method means replacing the data in the dictionary. This type of implementation is simple. There's an insignificant amount of overhead on a function/method call (i.e. instead of "func->data" you have  "func=*pfunc;if ( func->isInvalid() ) throw exception; else func->data" ).

Technically this algorithm leaks memory since deleted functions/methods are never removed.  My response is who cares. When the interpreter cleanup everything function is called, you simple deallocate everything in the dictionary.

Instead of a temporary "replace" attribute wrapped into a reload-like call, how about giving modules a user settable "replace" attribute.  At any time, the user can set/reset this attribute.  This would specify how the user wanted functions/methods processed.  Always added or always replaced. The "replace" attribute would likely need to be pass through to function objects, class objects, and method objects. For the macro language scenarios, I would just mark every module that got loaded with this attribute.

The proposed implementation I have given is intended to by very "file" oriented (which maps to a Python module).  

Would this work in the current code base? 

I'm assuming the following:

When a function is added/executed, the module structure is accessible.
When a class is added (i.e. class myclass), the module structure is accessible.
When a method is added/executed, at least the class structure is accessible?

I hope you see where I'm going here.  The executed "class myclass" code which defines a new class can copy the module "replace" attribute.  the executed "def myfunction" code which defines a new method can copy the class "replace" attribute.

The function/method object structure could remain the same except for the addition of a new function/method pointer member.  

The additional code for a function call would look like this:

   // Did this function get defined in "replace" mode?
   if ( func->doReplace() ) { 
       // For this one, use the indirect pointer and not the other member data.
       func= func->pfunc;
       if ( !func->isValid() ) {
          throw exception.  // Python exeception, not C++ exception
          return here...
    //  Now do what we used to do

Given the OO nature of Python, a separate function/method type for a replacable function/method could be defined but I suspect it isn't worth the effort.  The above psuedo code is very efficient "doReplace" would probably be just a boolean/int member.  The "isValid" called would be efficient as well.

One thing my proposed implementation does not cover is adding new data members to a class.  I think it is acceptable for this not to be handled.

Please shot this down this high level implementation if it won't work in the current code base.

Also, what does everyone think about the idea of some sort of "replace" attribute for the module?  How should it get set?  "import module; module.replace=1". I'm probably showing a little lack of knowledge here. Teach me and I'll get it.

Pinpoint customers who are looking for what you sell. 

More information about the Python-ideas mailing list