[Python-Dev] Return type of alternative constructors

[Nick Coghlan]

On 9 May 2016 at 08:50, Guido van Rossum <guido at python.org> wrote:
> On Sun, May 8, 2016 at 4:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> P.S. The potential complexity of that is one of the reasons the design
>> philosophy of "prefer composition to inheritance" has emerged -
>> subclassing is a powerful tool, but it does mean you often end up
>> needing to care about more interactions between the subclass and the
>> base class than you really wanted to.
>
> Indeed!
>
> We could also consider this a general weakness of the "alternative
> constructors are class methods" pattern. If instead these alternative
> constructors were folded into the main constructor (e.g. via special keyword
> args) it would be altogether clearer what a subclass should do.

Unfortunately, even that approach gets tricky when the inheritance
relationship crosses the boundary between components with independent
release cycles.

In my experience, this timeline is the main one that causes the pain:

* Base class is released in Component A (e.g. CPython)
* Subclass is released in Component B (e.g. PyPI module)
* Component A releases a new base class construction feature

Question: does the new construction feature work with the existing
subclass in Component B if you combine it with the new version of
Component A?

When alternate constructors can be implemented as class methods that
work by creating a default instance and using existing public API
methods to mutate it, then the answer to that question is "yes", since
the default constructor hasn't changed, and the new convenience
constructor isn't relying on any other new features.

The answer is also "yes" for existing subclasses that only add new
behaviour without adding any new state, and hence just use the base
class __new__ and __init__ without overriding either of them.

It's when the existing subclasses overrides __new__ or __init__ and
one or both of the following is true that things can get tricky:

- you're working with an immutable type
- the API implementing the post-creation mutation is a new one

In both of those cases, the new construction feature of the base class
probably won't work right without updates to the affected subclass to
support the new capability (whether that's supporting a new parameter
in __new__ and __init__, or adding their own implementation of the new
alternate constructor).

I'm genuinely unsure that's a solvable problem in the general case -
it seems to be an inherent consequence of the coupling between
subclasses and base classes during instance construction, akin to the
challenges with subclass compatibility of the unpickling APIs when a
base class adds new state.

However, from a pragmatic perspective, the following approach seems to
work reasonably well:

* assume subclasses don't change the signature of __new__ or __init__
* note the assumptions about the default constructor signature in the
alternate constructor docs to let implementors of subclasses that
change the signature know they'll need to explicitly test
compatibility and perhaps provide their own implementation of the
alternate constructor

You *do* still end up with some cases where a subclass needs to be
upgraded before a new base class feature works properly for that
particular subclass, but subclasses that *don't* change the
constructor signature "just work".

Cheers,
Nick.

P.S. It occurs to me that a sufficiently sophisticated typechecker
might be able to look at all of the calls to "cls(*args, **kwds)" in
class methods and "type(self)(*args, **kwds)" in instance methods, and
use those to define a set of type constraints for the expected
constructor signatures in subclassses, even if the current code base
never actually invokes those code paths.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia