[Python-Dev] Making PyInterpreterState an opaque type
steve.dower at python.org
Tue Feb 19 11:43:54 EST 2019
On 19Feb2019 0229, Jeroen Demeyer wrote:
> On 2019-02-19 04:04, Steve Dower wrote:
>> On 18Feb.2019 1324, Jeroen Demeyer wrote:
>>> For a very concrete example, was it really necessary to put
>>> _PyTuple_ITEMS in (4)? That's used in _functoolsmodule.c. Especially
>>> given that the very similar PySequence_Fast_ITEMS is in (2), that seems
>>> like a strange and arbitrary limiting choice.
>> The reason to do this is that we can "guarantee" that we've fixed all
>> users when we change the internal representation.
> I think that CPython should then at least "eat its own dog food" and
> don't use any of the internal functions/macros when implementing the
> stdlib. As I said before: if a function/macro is useful for implementing
> stdlib functionality like "functools" or "json", it's probably useful
> for external modules too.
I'm inclined to agree, but then I'm also one of the advocates for
breaking out as much as possible of the stdlib into pip-installable
modules, which would necessitate this :)
There are certainly parts of the stdlib that are there to _enable_ the
use of these features without exposing them - asyncio has some good
examples of this. But the rest probably don't. If they really do, then
we would have to define stable ways to get the same functionality (one
example - numpy currently relies on being able to check the refcount to
see if it equals 1, but we could easily provide a
"Py_HasNoOtherReferences" function that would do the same thing and also
allow us to one day move or remove reference counts without breaking numpy).
That said, one of the criteria for "are you eligible to use the internal
API" is "will users always have matched builds of this module" - for the
standard library, the answer is yes, and so they can use that API.
For third-party extension modules, the answer _might_ be yes, which
means they _might_ be able to use the API. But both of those "mights"
are outside of the control of the core development team, so we can't
take that responsibility for you.
The best we can do is make it easy to use the stable APIs, and make
using the unstable APIs a deliberate choice so that those who do so are
aware that they are now more responsible for their user's success than
they thought they were.
More information about the Python-Dev