a standard approach to module-replace-itself-in-sys.modules
Not that long ago we had a discussion about the esoteric, yet totally supported and not going anywhere, trick of having a module replace itself in sys.modules. Presently there isn't any convention surrounding the technique. It's simply a matter of sticking something else into sys.modules[__name__]. The problem is that there are some subtle things that can go wrong in the import system if you aren't careful and people generally won't know that. (Then again the whole technique isn't exactly in the mainstream!) To help reduce the risk here, I propose that we provide a helper in importlib.util and an associated convention. The helper will be a class decorator and the convention will be to decorate 1 class at the bottom of the module, where the class will contain any relevant customizations. The decorator will do the following: 1. create a new module subclass; 2. merge the decorated class namespace into the subclass; 3. create an instance of the subclass; 4. merge the original module namespace into the instance; 5. set sys.modules[__name__] to the new instance. For example: @importlib.util.replace_module class CustomModule: CHEDDAR = None @property def limburger (self): return None There's more to it but that captures the gist. I've been exploring API ideas, but ultimately we don't need much to get the ball rolling. In the meantime, I've published an unrefined but mostly working version of replace_module() online: https://bitbucket.org/ericsnowcurrently/odds_and_ends/src/default/replace_mo... -eric
On Thu, Mar 13, 2014 at 4:01 AM, Eric Snow
Not that long ago we had a discussion about the esoteric, yet totally supported and not going anywhere, trick of having a module replace itself in sys.modules. Presently there isn't any convention surrounding the technique. It's simply a matter of sticking something else into sys.modules[__name__].
The problem is that there are some subtle things that can go wrong in the import system if you aren't careful and people generally won't know that. (Then again the whole technique isn't exactly in the mainstream!) To help reduce the risk here, I propose that we provide a helper in importlib.util and an associated convention.
Do we actually want to promote this practice by codifying support for it in the stdlib? If you're doing this to get decorators for module attributes then you should say this is for that goal and the solution is getting the module swapping hack to be easier to use (and might call for a better solution). Otherwise this proposal feels like it's trying to codify behaviour that is rarely necessary.
On Mar 13, 2014, at 08:54 AM, Brett Cannon wrote:
Do we actually want to promote this practice by codifying support for it in the stdlib? If you're doing this to get decorators for module attributes then you should say this is for that goal and the solution is getting the module swapping hack to be easier to use (and might call for a better solution). Otherwise this proposal feels like it's trying to codify behaviour that is rarely necessary.
I don't think it necessarily has to be promoted as something we want people to generally do, but if they're going to do it I do think it would be nice to have a bless (i.e. stdlib) way. -Barry
On Mar 13, 2014 6:54 AM, "Brett Cannon"
Do we actually want to promote this practice by codifying support for it in the stdlib?
I guess I was expecting that question from you. :) I figure if the practice is officially supported then we should help folks doing it to not shoot themselves in the foot. As to how much such a tool will promote the practice, I'm not sure that it will, particularly if the helper is located out of the way. However I'm sure you've watched similar additions unfold and have a better sense of that than I do.
If you're doing this to get decorators for module attributes then you should say this is for that goal and the solution is getting the module swapping hack to be easier to use (and might call for a better solution). Otherwise this proposal feels like it's trying to codify behaviour that is rarely necessary.
Agreed. Furthermore I doubt I'll push on this idea without a strong need first. More than anything I wanted to share what seemed like an elegant solution to the "problem", even if we don't need to solve it right away. If I get more serious about it I'll hit up folks that are using the current hack (Django?) and see if it improves anything for them. -eric
participants (3)
-
Barry Warsaw
-
Brett Cannon
-
Eric Snow