[Python-Dev] Proposal: explicitly disallow function/class mismatches in accelerator modules

[Nick Coghlan]

On 12 July 2016 at 02:05, Paul Moore <p.f.moore at gmail.com> wrote:
> However, I don't think we can assume it'll be possible to make a
> high-level ruling on *every* possible situation. I don't think that's
> reasonable - there needs to be some level of judgement applied. The
> *documented* API is definitive, both implementations must follow the
> documented API - but if the documentation is silent or ambiguous, I
> don't think we should feel under undue pressure to replicate every
> behaviour of the CPython implementation - if we take that view, then
> we're back to implementation-defined semantics. Outside of documented
> behaviour, I don't think mandating compatibility without qualification
> is helpful.
>
> It's worth having guidelines on what implementers should be ensuring
> match between C and pure-Python implementation - and even more so
> having reminders about common pitfalls and places where extra work is
> needed to get the same behaviour - but I'd be concerned if the rules
> became dogma rather than guidance. We need to be prepared to trust the
> judgement of the core devs in this matter.

Based on this discussion, I've come to the conclusion that there are
only two cases where I'd like PEP 399 to document pre-emptive answers
to "What counts as sufficiently compatible?" question:

1. If an existing C API implementation is a public custom type, a
subsequently added Python fallback should also be a custom type, not a
closure or other non-type object that differs from the C
implementation in regards to subclassing and pickling support.
2. If an existing Python API implementation returns a closure, a
subsequently added accelerated version should either also return a
closure (e.g. if the accelerator is implemented in a language with
closure support, like Cython) or else be implemented as a factory
function returning an instance of a non-public type.

The first case is the one that applies to functools.partial:
preserving consistency in subclassing and pickling behaviour was
considered optional, even though it required disabling some of the
existing tests for the C version when testing the pure Python version.

The second case is, as far as I know, currently hypothetical, but
should make it clearer that it's important to watch out for
inadvertently *expanding* the inferred public API when implementing a
C accelerator - regardless of whether the C version or the Python
version is written first, we want to avoid the situation where the C
version implements a superset of the Python one, and folks mistakenly
consider that to be the supported cross-implementation definition of
the API.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia