[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Stefan Krah]

On Mon, Nov 19, 2018 at 04:08:07PM +0100, Victor Stinner wrote:
> Le lun. 19 nov. 2018 à 13:18, Stefan Krah <stefan at bytereef.org> a écrit :
> > In practice people desperately *have* to use whatever is there, including
> > functions with underscores that are not even officially in the C-API.
> >
> > I have to use _PyFloat_Pack* in order to be compatible with CPython,
> 
> Oh, I never used this function. These functions are private (name
> prefixed by "_") and excluded from the limited API.
> 
> For me, the limited API should be functions available on all Python
> implementations. Does it make sense to provide PyFloat_Pack4() in
> MicroPython, Jython, IronPython and PyPy? Or is it something more
> specific to CPython? I don't know the answer. If yes, open an issue to
> propose to make this function public?

It depends on what the goal is: If PyPy wants to be able to use as many
C extensions as possible, then yes.

The function is just one example of what people have to use to be 100%
compatible with CPython (or copy these functions and maintain them ...).


Intuitively, it should probably not be part of a limited API, but I never
quite understood the purpose of this API, because I regularly need any
function that I can get my hands on.


> > I need PyUnicode_KIND()
> 
> IMHO this one should not be part of the public API. The only usage
> would be to micro-optimize, but such API is very specific to one
> Python implementation. For example, PyPy doesn't use "compact string"
> but UTF-8 internally. If you use PyUnicode_KIND(), your code becomes
> incompatible with PyPy.
> 
> What is your use case?

Reading typed strings directly into an array with minimal overhead.


> I would prefer to expose the "_PyUnicodeWriter" API than PyUnicode_KIND().
> 
> > need PyUnicode_AsUTF8AndSize(),
> 
> Again, that's a micro-optimization and it's very specific to CPython:
> result cached in the "immutable" str object. I don't want to put it in
> a public API. PyUnicode_AsUTF8String() is better since it doesn't
> require an internal cache.
> 
> > I *wish* there were PyUnicode_AsAsciiAndSize().
> 
> PyUnicode_AsASCIIString() looks good to me. Sadly, it doesn't return
> the length, but usually the length is not needed.

Yes, these are all just examples.  It's also very useful to be able to
do PyLong_Type.tp_as_number->nb_multiply or grab as_integer_ratio from
the float PyMethodDef.

The latter two cases are for speed reasons but also because sometimes
you *don't* want a method from a subclass (Serhiy was very good in
finding corner cases :-).


Most C modules that I've seen have some internals. Psycopg2:

PyDateTime_DELTA_GET_MICROSECONDS
PyDateTime_DELTA_GET_DAYS
PyDateTime_DELTA_GET_SECONDS
PyList_GET_ITEM
Bytes_GET_SIZE
Py_BEGIN_ALLOW_THREADS
Py_END_ALLOW_THREADS

floatobject.h and longintrepr.h are also popular.


Stefan Krah