On 23. 01. 22 12:44, Aviram Hassan wrote:
Hi all,
Tl;dr - As mentioned in issue https://bugs.python.org/issue5945,
PySequence_CheckandPyMapping_Checkshould be deprecated and discontinued. The reason for this is that they do not provide accurate results and can lead to unexpected behavior. (PyMapping_Check for a list will return True, and PySequence_Check for a mapping that doesn't subclass dict will also return true)
They should *not* be removed, or changed. The surprising behavior and naming is not reason enough to break existing, working code.
It's OK to add new ways to do things, and to discourage the old ones if they're obsolete. But there are existing extensions that use what was available, and those should continue to work.
This proposal is to have a process to deprecate those and introduce new functions instead.
Thanks to the latest changes in the C-API, we now have flags set for classes that are sequences or mapping - those were created for the pattern matching functionality.
This lets us have a path for clear and accurate determination of sequence or mapping.
The use cases for the change are as follows:
- C-API code that wishes to use the PySequence/PyMapping API and to guarantee correct behavior needs to check it is indeed a valid sequence/mapping - Static types casting from ambiguous “PyObject” to “PySequence” & “PyMapping” types as provided by PyO3 ( https://pyo3.rs/v0.15.1/conversions/tables.html#argument-types)The correct way to determine right now if an object is a sequence, is to check its
Py_TPFLAGS_SEQUENCEflag and check if it’s a string, bytes or bytearray as they are the only sequences not havingTPFLAGS_SEQUENCEper specification. This is cumbersome for most non-expert developers not aware of the nuances in the C-API. Having a C-API that provides this check out of the box can ease it for most.
PyMapping_CheckandPySequence_Checkare still widely used and are **part of the stable ABI** even though core developers recommend avoiding those. This is very odd to stabilize and is an API that should be deprecated.
It is stabilized because existing code depends on it.
Our suggested change is to add two new functions:
PyMapping_CheckNew&PySequence_CheckNew. They will be implemented using the flags check and the str/bytes/bytearray check. Those functions will be 100% accurate and follow the existing documentations.
"100% accurate" is a bold claim. Do you mean "accurate by definition", i.e. we should define "sequence" in terms of what PySequence_CheckNew returns?
The existing documentation for PyMapping_Check says:
Note that it returns 1 for Python classes with a __getitem__() method since in general case it is impossible to determine what type of keys it supports.
This doesn't sound like what the behavior you want. What other documentation are you referring to?
We will do the following process to improve C-API:
1. Add these functions. 2. Add a deprecation notice on `PyMapping_Check` and `PySequence_Check` and remove all built-in usage of it. We will recommend the use of the new functions.
All reasonable up to here (except there might be better names than *New).
3. Drop the old functions. 4. Alias the new functions to `PyMapping_Check` and `PySequence_Check` and add a deprecation notice of `PyMapping_CheckNew` and `PySequence_CheckNew`. 5. Rename `PyMapping_CheckNew` and `PySequence_CheckNew` to `Mapping_Check` and `PySequence_Check`.The timeline will probably be long as it’s part of stable ABI, but in our opinion, the change is for the better.
But for steps 4 and 5, I think the best timeline is "never".
Caveats:
- Sequences and Mappings will match the typing.Sequence and typing.Mapping abcs. Duck typing is further “discouraged”. We do not think of it as an issue since Python already has idioms and static type checkers that discourage this sort of duck typing.
Static type checkers are still an optional tool for people who want that sort of thing. Quite a popular tool, sure. But discouraging duck typing would be a very big change to the status quo. Perhaps the proposal should start with that, rather than put it in a caveats section?
- C extensions skipping multiple minor Python versions will use the new code without their knowledge, which is not backwards compatible. This is an issue with any removal of a stable ABI or language function, and we consider it acceptable.Thanks to Bar Harel for helping me with the proposal.
This is was initially discussed in this bpo: https://bugs.python.org/issue46376
Best regards,
Aviram
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org https://mail.python.org/mailman3/lists/capi-sig.python.org/ Member address: encukou@gmail.com