Originally C API didn't use the const qualifier. Over few last years the const qualifier was added to C API if that preserved backward compatibility. For example input "char *" parameters were changed to "const char *". This makes C API compatible with C++, eliminates C compiler warnings, and helps to found possible errors.
Now I have started to make changes that are not absolute compatible, and can need modifying third-party code (but unlikely).
* The const qualifier was added to "char *" fields name and doc of some structures. They always point to C string literals. https://bugs.python.org/issue28761
* The const qualifier was added to private global variable _Py_PackageContext. https://bugs.python.org/issue28748
Now I'm going to add the const qualifier to the result of PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8(). These functions return a reference to internal cached UTF8 representations of a string. It should never be modified. https://bugs.python.org/issue28769
Later I'm planning following changes:
* Add the const qualifier to the result of functions that return references to internal representation of immutable objects, like PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally can modify the content of immutable objets, this is very dangerous, because this can invalidates invariants and cached values. Third-party code shouldn't do this.
* Add the const qualifier to the format field of Py_buffer. It is a reference to C string literal or to the content of bytes object. Mutating its content is an error. Only _testbuffer overuses the format field of internal Py_buffer object for owning a reference to allocated memory. But this is not leaked outside.
What are you think about this?
On 18 December 2016 at 18:31, Serhiy Storchaka storchaka@gmail.com wrote:
Later I'm planning following changes:
- Add the const qualifier to the result of functions that return
references to internal representation of immutable objects, like PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally can modify the content of immutable objets, this is very dangerous, because this can invalidates invariants and cached values. Third-party code shouldn't do this.
- Add the const qualifier to the format field of Py_buffer. It is a
reference to C string literal or to the content of bytes object. Mutating its content is an error. Only _testbuffer overuses the format field of internal Py_buffer object for owning a reference to allocated memory. But this is not leaked outside.
What are you think about this?
As long as it's on the default branch with appropriate notes in the C porting section of the 3.7 What's New, turning these kinds of runtime errors into compilation errors sounds like the right thing to do to me.
One key aspect from my perspective is that code that is updated to correctly declare the destination storage as a const pointer will still compile against the old API variants that return a mutable pointer, so any problems this finds in third party code are likely to be resolved for older 3.x releases as well.
Cheers, Nick.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 12/18/2016 07:54 PM, Nick Coghlan wrote:
On 18 December 2016 at 18:31, Serhiy Storchaka storchaka@gmail.com wrote:
Later I'm planning following changes:
- Add the const qualifier to the result of functions that return
references to internal representation of immutable objects, like PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally can modify the content of immutable objets, this is very dangerous, because this can invalidates invariants and cached values. Third-party code shouldn't do this.
- Add the const qualifier to the format field of Py_buffer. It is a
reference to C string literal or to the content of bytes object. Mutating its content is an error. Only _testbuffer overuses the format field of internal Py_buffer object for owning a reference to allocated memory. But this is not leaked outside.
What are you think about this?
As long as it's on the default branch with appropriate notes in the C porting section of the 3.7 What's New, turning these kinds of runtime errors into compilation errors sounds like the right thing to do to me.
One key aspect from my perspective is that code that is updated to correctly declare the destination storage as a const pointer will still compile against the old API variants that return a mutable pointer, so any problems this finds in third party code are likely to be resolved for older 3.x releases as well.
Agreed. Anything the compiler ralfs on after adding 'const' (where the actual target must be immutable) already had the fuse smoldering. FWIW I help maintain some *old* C extensions (fifteen+ years and counting), and am as likely to be affected as anyone.
Tres - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com
2016-12-18 9:31 GMT+01:00 Serhiy Storchaka storchaka@gmail.com:
Originally C API didn't use the const qualifier. Over few last years the const qualifier was added to C API if that preserved backward compatibility. For example input "char *" parameters were changed to "const char *". This makes C API compatible with C++, eliminates C compiler warnings, and helps to found possible errors.
Since the "const" keyword does not impact the stable *ABI*, I think that it's fine to add use it in more places.
In the worst case, if an extension chose to be compiled with -Werror (convert warnings into errors), the maintainer will have to fix conversion warnings in the code. But it's easy to write C code which works with and without const (old and new Python *API*), using explicit cast (to const char* for example).
In the common case, it will just be a warning and nobody will notice it since more and more people use pip which compiles C extensions in the background and doesn't show GCC output anymore.
I agree that it can help to find real bugs.
Victor
-
Nick Coghlan -
Serhiy Storchaka -
Tres Seaver -
Victor Stinner