Needed: Real-world examples for Python's Cooperative Multiple Inheritance

[Kirill Simonov]

Hi Raymond,

We've been using cooperative inheritance to implement stacked utilities 
such as WSGI middleware or connecting to a database.

An example of a WSGI middleware stack:

# Declare the interface and provide the default implementation.
class WSGI(Utility):
     def __call__(self, environ, start_response):
         # The main WSGI application is implemented here.
         start_response("200 OK", [('Content-Type', 'text/plain')])
         return ["Hello World!"]

# GZip middleware (may be defined in a different module or a plugin)
class GZIP(WSGI):
     # To indicate relative position in the middleware stack
     weights(100)
     def __call__(self, environ, start_response):
         # Call the next middleware in the stack to generate data.
         # Also, need to wrap start_response here...
         generator = super(GZIP, self).__call__(environ, start_response)
         # Pack the output...

# Error handling middleware (defined in a different module or a plugin)
class LogErrors(WSGI):
     weights(1000)
     def __call__(self, environ, start_response):
         # Call the next middleware in the stack, catch any errors.
         try:
             generator = super(LogErrors, self).__call__(environ,
                                                  start_response)
         except:
             # Log errors...

# Now glue them all together
def wsgi(environ, start_response):
     wsgi = WSGI() # !!!
     return wsgi(environ, start_response)

The trick here is that the constructor of `WSGI` (actually, 
`Utility.__new__()`) is overridden.  Instead of producing a new instance 
of `WSGI` , it does the following:
- use `__subclasses__()` to find all components of the utility;
- order the components by their weights;
- create a new class: `type(name, list_of_components, {})`;
- return an instance of the class.

Here is another example, database connection.

# The interface, no default implementation.
class Connect(Utility):
     def __init__(self, host, port, user, password, database):
         self.host = host
         self.port = port
         self.user = user
         self.password = password
         self.database = database
     def __call__(self):
         raise NotImplementedError()

# Public API
def connect(host, port, user, password, database):
     # Same trick here.
     connect = Connect(host, port, user, password, database)
     return connect()

# PostgreSQL implementation (defined in a plugin)
import psycopg2
class PGSQLConnect(Connect):
     def __call__(self):
         return psycopg2.connect(...)

# Connection pooling (defined in a plugin)
class Pooling(Connect):
     weights(100)
     def __call__(self):
         # Check if we could reuse an existing connection
         # ...
         # If no free connections available
         connection = super(Pooling, self).__call__()
         # Save it somewhere and return it...

Note that utility instances are short-lived so any persistent state must 
be kept elsewhere.


We also use the same pattern to implement Cecil/Diesel-style 
multimethods and general predicate dispatch, but that's probably outside 
the scope of your question.

A public version of the code lives here:
https://bitbucket.org/prometheus/htsql
Unfortunately it doesn't exactly match my examples above: connection 
pooling and most of the wsgi middleware are still to be ported, 
`weights()` is missing, etc.


Hope it helps.

Thanks,
Kirill

On 11/24/2010 03:08 PM, Raymond Hettinger wrote:
> I'm writing-up more guidance on how to use super() and would like to
> point at some real-world Python examples of cooperative multiple
> inheritance.
>
> Google searches take me to old papers for C++ and Eiffel, but that
> don't seem to be relevant to most Python programmers (i.e. a
> WalkingMenu example where a submenu is both a Entry in a Menu and a
> Menu itself).  Another published example is in a graphic library where
> some widgets inherit GraphicalFeature methods such as location, size
> and NestingGroupingFeatures such as finding parents, siblings, and
> children.  I don't find either of those examples compelling because
> there is no particular reason that they would have to have overlapping
> method names.
>
> So far, the only situation I can find where method names necessarily
> overlap is for the basics like __init__(), close(), flush(), and
> save() where multiple parents need to have their own initialization
> and finalization.
>
> If you guys know of good examples, I would appreciate a link or a
> recap.
>
> Thanks,
>
>
> Raymond
>