[capi-sig]Re: CPython C API Design Guidelines

Hi,

As the migration to Python 3 showed us, any migration is painful and takes a very long time. Your project would only be successful if the critical mass of C extension move it. But it's really unclear to me why a maintainer would like to invest time in migrating to these new rings.

tooling and documentation should be provided to help to migrate to such new model.

More generally, I don't see the direct benefit for PyPy or other Python implementation. The proposal doesn't say anything about reference counting and GC which are the blocker issues of the current C API.

I expect that such API would be easier to be used for embedded Python, but it's non obvious what are the benefit for CPython in general. I expect a lot of refactoring and additional code to handle these new layers.

Le jeu. 21 févr. 2019 à 19:26, Steve Dower <steve.dower@python.org> a écrit :

For context as you continue reading, these are the API **rings** provided by CPython:

• Python ring (equivalent of the Python language)
• CPython ring (CPython-specific APIs)
• Internal ring (intended for internal use only)

These are the API **layers** provided by CPython:

• Optional stdlib layer (dependencies that must be explicitly required)
• Required stdlib layer (dependencies that can be assumed)
• Platform adaption layer (ability to interact with the platform)
• Core layer ("pure" mode with no platform interactivity)

I'm not sure of the kind of problems that you are trying to solve with these new separations with "rings".

For a concrete example, PyObject_GetItem is part of the Python ring while PyDict_GetItem is in the CPython ring.

I like that :-)

For CPython, including Python.h should only provide access to the Python ring. Accessing any other rings should produce a compile error.

That's a major backward incompatible change, right?

Right now, Python.h gives also what you call the "CPython ring".

Compatibility requirements for the CPython API match the CPython major.minor version. Specifically, code relying on the CPython API should only break or change behaviour if the major.minor version changes.

Currently, there is a "stable API" which is supposed to be compatible on multiple Python versions, not only a specific X.Y. Projects like PyQt uses this "stable ABI" (sorry, I wrote API and then ABI, the difference is subtle and confuses me). You don't say anything about it here.

Lower layers are required to maintain backwards compatibility more strictly than the layers above them.

Hum, I don't understand well the separation between the CPython implementation and the "Python ring" (API). CPython core evolves way more faster than its API. Here you only talk about the API, right?

Components within a layer that depend on other components within that layer must be treated as a single component for determining whether it may be included or omitted.

Currently, the whole API are based on some key features:

• PyObject object model and C structures
• CPython GC implementation
• CPython memory allocators

I'm not sure how adding more layers will help to move away from these "legacy contraints".

• Dynamic memory allocation/deallocation is part of the Platform adaptation layer, but there is no way to avoid it here. So any user of the core API will need to provide allocators and deallocators. The CPython Platform adaptation layer provides the "default" implementations, but if an embedder does not want to use these then targeting the Core layer will omit them.
• File system and standard streams are part of the Platform adaptation layer, which leaves open and sys.stdout (among others) without a default implementation. An application that wants to support these without adding more layers needs to provide its own implementations
• The core layer only exposes UTF-8 APIs. Encoding and decoding for the current platform requires the Platform adaptation layer, while arbitrary encoding and decoding requires the Optional stdlib layer.
• Imports in the core layer are satisfied by a "blind" callback. The Platform adaptation layer provides the support for frozen, bytecode and natively-encoded source imports, while the Optional stdlib layer is required for arbitrary encodings in source files

Oh wow, that's going a little bit too far into the complex "Python initialization API" problem. I would prefer to discuss it in a separated PEP / thread.

Victor

Night gathers, and now my watch begins. It shall not end until my death.