[Python-Dev] PEP 575, 576, 579 and 580

[Nick Coghlan]

On 7 July 2018 at 23:38, Mark Shannon <mark at hotpy.org> wrote:
> Hi,
>
> We seem to have a plethora of PEPs where we really ought to have one (or
> none?).
>
> Traditionally when writing a new piece of software, one gathered
> requirements before implementing the code. Let us return to that venerable
> tradition.
>
> IMO, mailing lists are a terrible way to do software design, but a good way
> to gather requirements as it makes less likely that someone will be
> forgotten.

That's the purpose of PEP 579: gather the background information on
the problems that folks want to solve such that the competing proposed
solutions aren't defining the problem that needs to be solved in
different ways.

If PEP 579 isn't working as a problem specification from your
perspective, then I'd suggest posting a PR that Jeroen could review
(although I think this thread is a good idea as well).

> So, let us gather the requirements for a new calling API.

> Here are my starting suggestions:
>
> 1. The new API should be fully backwards compatible and shouldn't break the
> ABI
> 2. The new API should be used internally so that 3rd party extensions are
> not second class citizens in term of call performance.
> 3. The new API should not prevent 3rd party extensions having full
> introspection capabilities, supporting keyword arguments or another feature
> supported by Python functions.
> 4. The implementation should not exceed D lines of code delta and T lines of
> code in total size. I would suggest +200 and 1000 for D and T respectively
> (or is that too restrictive?).
> 5. It should speed up CPython for the standard benchmark suite.
> 6. It should be understandable.

I like points 1, 2, 3, and 6, but I think point 4 should be a design
trade-off rather than a requirement, since minimising the delta in
CPython becomes an anti-goal if the outcome of doing so is to make the
change harder to adopt for third party projects (at the same time, a
delta that's too large is unlikely to be accepted, reviewed and
merged, which is what makes it a trade-off).

I don't think point 5 is a goal here either, as the problem isn't that
these calling optimisations don't exist, it's that they don't
currently have a public API that third party projects can access (the
most recent METH_FASTCALL thread covers that pretty well).

My own additional concern that I think is also on the debatable border
between "design requirement" and "design trade-off" is whether or not
it's acceptable for us to require that existing third party projects
change their parent CPython type in order to access the optimised
calling conventions. Changing Python base types in an extension module
can end up being an annoyingly intrusive change, since it changes the
memory layout in your instances. Whether or not that's a problem
depends on exactly what you're doing, but when the new calling
convention is tied to a protocol that any type can implement (as PEP
580 proposes), the concern doesn't even arise.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia