On Fri., 14 Jun. 2019, 2:05 am Steve Dower, <steve.dower@python.org> wrote:
On 13Jun2019 0816, Jeroen Demeyer wrote:
> On 2019-06-13 17:11, Steve Dower wrote:
>> The cost of that convenience is that
>> we can never optimise internals because they are now public API.
>
> I think that the opposite is true actually: the reason that people
> access internals is because there is no public API doing what they want.
> Having more public API should *reduce* the need for accessing internals.

Right, but we need to know what API that is. We can't just make
everything public by default.

> For example, _PyObject_GetMethod is not public API but it's useful
> functionality. So Cython is forced to reinvent _PyObject_GetMethod (i.e.
> copy verbatim that function from the CPython sources), which requires
> accessing internals.

What's wrong with using PyObject_GetAttr() and then doing
PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_METHOD_DESCRIPTOR) on the
result?

More importantly, why do you need to know that it's a method descriptor
and not just a callable object that can be accessed via an attribute?
And is this something that's generally needed, or is Cython just special
(I expect Cython to be special in many situations, which is why we have
a tiered API and expect Cython-generated code to be regenerated for
different CPython versions).

We don't expect most Cython code to be regenerated for different versions - we only expect it to be recompiled, as with any other extension.

Hence Jeroen's point: if something is useful enough for Cython to want to use it, it makes to provide a public API for it that hides any internal implementation details that may not remain stable across releases.

Cheers,
Nick.