On Mon., Nov. 19, 2018, 14:04 Neil Schemenauer On 2018-11-19, Antoine Pitrou wrote: There are important use cases for the C API where it is desired to have
fast type-specific access to Python objects such as tuples, ints,
strings, etc. This is relied upon by modules such as _json and _pickle,
and third-party extensions as well. Thank you for pointing this out. The feedback from Stefan on what
Cython would like (e.g. more access to functions that are currently
"internal") is useful too. Keeping our dreams tied to reality
is important. ;-P It seems to me that we can't "have our cake and eat it too". I.e. on
the one hand hide CPython implementation internals but on the other
hand allow extensions that want to take advantage of those internals
to provide the best performance. No, but those are different APIs as well. E.g. no one is saying CPython has
to do away with any of its API. What I and some others have said is the
CPython API is too broad to be called "universal". Maybe we could have a multiple levels of API: A) maximum portability (Py_LIMITED_API) B) source portability (non-stable ABI, inlined functions) C) portability but poor performance on non-CPython VMs
(PySequence_Fast_ITEMS, borrowed refs, etc) I don't know own how doable that is as e.g. borrowed refs are not pleasant
to simulate. D) non-portability, CPython specific (access to more internals like
Stefan was asking for). The extension would have to be
re-implemented on each VM or provide a pure Python
alternative. I think it would be nice if the extension module could explicitly
choose which level of API it wants to use. Yes, and I thought we were working towards nesting our header files so you
very clearly opted into your level of compatibility.
In my head there's:
- bare minimum, cross-VM, gets you FFI
- CPython API for more performance that we're willing to maintain
- Everything open for e.g. CPython with no compatibility guarantees
Due note my first point isn't necessarily worrying about crazy performance
to start. I would assume an alternative VM would help make up for this with
a faster runtime where dropping into C is more about FFI than performance
(we all know PyPy, for instance, wished people just wrote more Python code).
Otherwise we're back to the idea of standardizing on some Cython solution
to help make perfect easier without tying oneself to the C API (like
Julia's FFI solution). It would be interesting to do a census on what extensions are out
there. If they mostly fall into wanting level "C" then I think this
API overhaul is not going to work out too well. Level C is mostly
what we have now. No point in putting the effort into A and B if no
one will use them. It won't until someone can show benefits for switching. This is very much a
chicken-and-egg problem.
_______________________________________________ Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/brett%40python.org