[Python-ideas] Module __getattr__ [Was: breaking out of module execution]

[Gregory P. Smith]

On Mon, Apr 30, 2012 at 7:33 PM, Guido van Rossum <guido at python.org> wrote:

> On Mon, Apr 30, 2012 at 5:47 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> > Gregory P. Smith wrote:
> >
> >> Making modules "simply" be a class that could be subclasses rather than
> >> their own thing _would_ be nice for one particular project I've worked
> on
> >> where the project including APIs and basic implementations were open
> >> source
> >> but which allowed for site specific code to override many/most of those
> >> base implementations as a way of customizing it for your own specific
> (non
> >> open source) environment.
>
> > This makes no sense to me. What does the *licence* of a project have to
> do
> > with the library API? I mean, yes, you could do such a thing, but surely
> you
> > shouldn't. That would be like saying that the accelerator pedal should
> be on
> > the right in cars you buy outright, but on the left for cars you get on
> > hire-purchase.
>
> That's an irrelevant, surprising and unfair criticism of Greg's
> message. He just tried to give a specific example without being too
> specific.
>

heh. right. I didn't want to drag people into a project that shall not be
named because I dislike its design to begin with... I was just suggesting a
case where we _would_ have found it useful to treat a module as a class
that could be subclassed.  A better overall design pattern using classes in
the code's public APIs to start with would have prevented all such need.

I am not strongly in favor of modules being classes but it is an
interesting thing to ponder from time to time.


> > Nevertheless, I think your focus here is on the wrong thing. It seems to
> me
> > that you are jumping to an implementation, namely that modules should
> stop
> > being instances of a type and become classes, without having a clear
> idea of
> > your functional requirements.
> >
> > The functional requirements *might* be:
> >
> > "There ought to be an easy way to customize the behaviour of attribute
> > access in modules."
> >
> > Or perhaps:
> >
> > "There ought to be an easy way for one module to shadow another module
> with
> > the same name, but still inherit behaviour from the shadowed module."
> >
> > neither of which *require* modules to become classes.
> >
> > Or perhaps it is something else... it is unclear to me exactly what
> problems
> > you and Jim wish to solve, or whether they're the same kind of problem,
> > which is why I say the functional requirements are unclear.
> >
> > Changing modules from an instance of ModuleType to "a class that could
> be a
> > subclass" is surely going to break code. Somewhere, someone is relying on
> > the fact that modules are not types and you're going to break their
> > application.
> >
> >
> >
> >> Any APIs that were unfortunately defined as a
> >> module with a bunch of functions in it was a real pain to make site
> >> specific overrides for.
> >
> >
> > It shouldn't be. Just ensure the site-specific override module comes
> first
> > in the path, and "import module" will pick up the override module
> instead of
> > the standard one. This is a simple exercise in shadowing modules.
> >
> > Of course, this implies that the override module has to override
> > *everything*. There's currently no simple way for the shadowing module to
> > inherit functionality from the shadowed module. You can probably hack
> > something together, but it would be a PITA.
>
> If there is a bunch of functions and you want to replace a few of
> those, you can probably get the desired effect quite easily:
>
>  from base_module import *  # Or the specific set of functions that
> comprise the API.
>
>  def funct1(<args>): <my version>
>  def funct2(<args>): <my version>
>
> Not that I would recommend this -- it's easy to get confused if there
> are more than a very small number of functions. Also if
> base_module.funct3 were to call func2, it wouldn't call the overridden
> version.
>
> But all attempts to view modules as classes or instances have lead to
> negative results. (I'm sure I've thought about it at various times in
> the past.)
>
> I think the reason is that a module at best acts as a class where
> every method is a *static* method, but implicitly so. Ad we all know
> how limited static methods are. (They're basically an accident -- back
> in the Python 2.2 days when I was inventing new-style classes and
> descriptors, I meant to implement class methods but at first I didn't
> understand them and accidentally implemented static methods first.
> Then it was too late to remove them and only provide class methods.)
>

This "oops, I implemented static methods" is a wonderful bit of history! :)


>
> There is actually a hack that is occasionally used and recommended: a
> module can define a class with the desired functionality, and then at
> the end, replace itself in sys.modules with an instance of that class
> (or with the class, if you insist, but that's generally less useful).
> E.g.:
>
>  # module foo.py
>
>  import sys
>
>  class Foo:
>    def funct1(self, <args>): <code>
>    def funct2(self, <args>): <code>
>
>  sys.modules[__name__] = Foo()
>
> This works because the import machinery is actively enabling this
> hack, and as its final step pulls the actual module out of
> sys.modules, after loading it. (This is no accident. The hack was
> proposed long ago and we decided we liked enough to support it in the
> import machinery.)
>
> You can easily override __getattr__ / __getattribute__ / __setattr__
> this way. It also makes "subclassing" the module a little easier
> (although accessing the class to be used as a base class is a little
> tricky: you'd have to use foo.__class__). But of course the kind of
> API that Greg was griping about would never be implemented this way,
> so that's fairly useless. And if you were designing a module as an
> inheritable class right from the start you're much better off just
> using a class instead of the above hack.
>
> But all in all I don't think there's a great future in stock for the
> idea of allowing modules to be "subclassed". In the vast, vast
> majority of cases it's better to clearly have a separation between
> modules, which provide no inheritance and no instantiation, and
> classes, which provide both. I think Python is better off this way
> than Java, where all you have is classes (its packages cannot contain
> anything except class definitions).
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120508/e47283b5/attachment.html>