On Mon, Apr 30, 2012 at 5:47 PM, Steven D'Aprano wrote:

Gregory P. Smith wrote:

...

Making modules "simply" be a class that could be subclasses rather than their own thing _would_ be nice for one particular project I've worked on where the project including APIs and basic implementations were open source but which allowed for site specific code to override many/most of those base implementations as a way of customizing it for your own specific (non open source) environment.

This makes no sense to me. What does the *licence* of a project have to do with the library API? I mean, yes, you could do such a thing, but surely you shouldn't. That would be like saying that the accelerator pedal should be on the right in cars you buy outright, but on the left for cars you get on hire-purchase.

That's an irrelevant, surprising and unfair criticism of Greg's message. He just tried to give a specific example without being too specific.

Nevertheless, I think your focus here is on the wrong thing. It seems to me that you are jumping to an implementation, namely that modules should stop being instances of a type and become classes, without having a clear idea of your functional requirements.

The functional requirements *might* be:

"There ought to be an easy way to customize the behaviour of attribute access in modules."

Or perhaps:

"There ought to be an easy way for one module to shadow another module with the same name, but still inherit behaviour from the shadowed module."

neither of which *require* modules to become classes.

Or perhaps it is something else... it is unclear to me exactly what problems you and Jim wish to solve, or whether they're the same kind of problem, which is why I say the functional requirements are unclear.

Changing modules from an instance of ModuleType to "a class that could be a subclass" is surely going to break code. Somewhere, someone is relying on the fact that modules are not types and you're going to break their application.

...

Any APIs that were unfortunately defined as a module with a bunch of functions in it was a real pain to make site specific overrides for.

It shouldn't be. Just ensure the site-specific override module comes first in the path, and "import module" will pick up the override module instead of the standard one. This is a simple exercise in shadowing modules.

Of course, this implies that the override module has to override *everything*. There's currently no simple way for the shadowing module to inherit functionality from the shadowed module. You can probably hack something together, but it would be a PITA.

If there is a bunch of functions and you want to replace a few of those, you can probably get the desired effect quite easily: from base_module import * # Or the specific set of functions that comprise the API. def funct1(<args>): <my version> def funct2(<args>): <my version> Not that I would recommend this -- it's easy to get confused if there are more than a very small number of functions. Also if base_module.funct3 were to call func2, it wouldn't call the overridden version. But all attempts to view modules as classes or instances have lead to negative results. (I'm sure I've thought about it at various times in the past.) I think the reason is that a module at best acts as a class where every method is a *static* method, but implicitly so. Ad we all know how limited static methods are. (They're basically an accident -- back in the Python 2.2 days when I was inventing new-style classes and descriptors, I meant to implement class methods but at first I didn't understand them and accidentally implemented static methods first. Then it was too late to remove them and only provide class methods.) There is actually a hack that is occasionally used and recommended: a module can define a class with the desired functionality, and then at the end, replace itself in sys.modules with an instance of that class (or with the class, if you insist, but that's generally less useful). E.g.: # module foo.py import sys class Foo: def funct1(self, <args>): <code> def funct2(self, <args>): <code> sys.modules[__name__] = Foo() This works because the import machinery is actively enabling this hack, and as its final step pulls the actual module out of sys.modules, after loading it. (This is no accident. The hack was proposed long ago and we decided we liked enough to support it in the import machinery.) You can easily override __getattr__ / __getattribute__ / __setattr__ this way. It also makes "subclassing" the module a little easier (although accessing the class to be used as a base class is a little tricky: you'd have to use foo.__class__). But of course the kind of API that Greg was griping about would never be implemented this way, so that's fairly useless. And if you were designing a module as an inheritable class right from the start you're much better off just using a class instead of the above hack. But all in all I don't think there's a great future in stock for the idea of allowing modules to be "subclassed". In the vast, vast majority of cases it's better to clearly have a separation between modules, which provide no inheritance and no instantiation, and classes, which provide both. I think Python is better off this way than Java, where all you have is classes (its packages cannot contain anything except class definitions). -- --Guido van Rossum (python.org/~guido)

On Mon, Apr 30, 2012 at 7:33 PM, Guido van Rossum wrote:

On Mon, Apr 30, 2012 at 5:47 PM, Steven D'Aprano wrote:

...

Gregory P. Smith wrote:

...

Making modules "simply" be a class that could be subclasses rather than their own thing _would_ be nice for one particular project I've worked on where the project including APIs and basic implementations were open source but which allowed for site specific code to override many/most of those base implementations as a way of customizing it for your own specific (non open source) environment.

...

This makes no sense to me. What does the *licence* of a project have to do with the library API? I mean, yes, you could do such a thing, but surely you shouldn't. That would be like saying that the accelerator pedal should be on the right in cars you buy outright, but on the left for cars you get on hire-purchase.

That's an irrelevant, surprising and unfair criticism of Greg's message. He just tried to give a specific example without being too specific.

heh. right. I didn't want to drag people into a project that shall not be named because I dislike its design to begin with... I was just suggesting a case where we _would_ have found it useful to treat a module as a class that could be subclassed. A better overall design pattern using classes in the code's public APIs to start with would have prevented all such need. I am not strongly in favor of modules being classes but it is an interesting thing to ponder from time to time.

...

Nevertheless, I think your focus here is on the wrong thing. It seems to me that you are jumping to an implementation, namely that modules should stop being instances of a type and become classes, without having a clear idea of your functional requirements.

The functional requirements *might* be:

"There ought to be an easy way to customize the behaviour of attribute access in modules."

Or perhaps:

"There ought to be an easy way for one module to shadow another module with the same name, but still inherit behaviour from the shadowed module."

neither of which *require* modules to become classes.

Or perhaps it is something else... it is unclear to me exactly what problems you and Jim wish to solve, or whether they're the same kind of problem, which is why I say the functional requirements are unclear.

Changing modules from an instance of ModuleType to "a class that could be a subclass" is surely going to break code. Somewhere, someone is relying on the fact that modules are not types and you're going to break their application.

...

Any APIs that were unfortunately defined as a module with a bunch of functions in it was a real pain to make site specific overrides for.

It shouldn't be. Just ensure the site-specific override module comes first in the path, and "import module" will pick up the override module instead of the standard one. This is a simple exercise in shadowing modules.

Of course, this implies that the override module has to override *everything*. There's currently no simple way for the shadowing module to inherit functionality from the shadowed module. You can probably hack something together, but it would be a PITA.

If there is a bunch of functions and you want to replace a few of those, you can probably get the desired effect quite easily:

from base_module import * # Or the specific set of functions that comprise the API.

def funct1(<args>): <my version> def funct2(<args>): <my version>

Not that I would recommend this -- it's easy to get confused if there are more than a very small number of functions. Also if base_module.funct3 were to call func2, it wouldn't call the overridden version.

But all attempts to view modules as classes or instances have lead to negative results. (I'm sure I've thought about it at various times in the past.)

I think the reason is that a module at best acts as a class where every method is a *static* method, but implicitly so. Ad we all know how limited static methods are. (They're basically an accident -- back in the Python 2.2 days when I was inventing new-style classes and descriptors, I meant to implement class methods but at first I didn't understand them and accidentally implemented static methods first. Then it was too late to remove them and only provide class methods.)

This "oops, I implemented static methods" is a wonderful bit of history! :)

There is actually a hack that is occasionally used and recommended: a module can define a class with the desired functionality, and then at the end, replace itself in sys.modules with an instance of that class (or with the class, if you insist, but that's generally less useful). E.g.:

# module foo.py

import sys

class Foo: def funct1(self, <args>): <code> def funct2(self, <args>): <code>

sys.modules[__name__] = Foo()

This works because the import machinery is actively enabling this hack, and as its final step pulls the actual module out of sys.modules, after loading it. (This is no accident. The hack was proposed long ago and we decided we liked enough to support it in the import machinery.)

You can easily override __getattr__ / __getattribute__ / __setattr__ this way. It also makes "subclassing" the module a little easier (although accessing the class to be used as a base class is a little tricky: you'd have to use foo.__class__). But of course the kind of API that Greg was griping about would never be implemented this way, so that's fairly useless. And if you were designing a module as an inheritable class right from the start you're much better off just using a class instead of the above hack.

But all in all I don't think there's a great future in stock for the idea of allowing modules to be "subclassed". In the vast, vast majority of cases it's better to clearly have a separation between modules, which provide no inheritance and no instantiation, and classes, which provide both. I think Python is better off this way than Java, where all you have is classes (its packages cannot contain anything except class definitions).

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas