On Thu, Feb 27, 2020 at 11:59 AM Steve Dower steve.dower@python.org wrote:
On Feb 27, 2020, at 10:35, Brett Cannon brett@python.org wrote:
I agree that it's probably time to start looking at stable ABI 4. Maybe that's HPy? Either way we seem to consistently have momentum on working on improving the stable ABI so now might be the time to figure out what ABI4 should be and get people to start moving/prepping for it by e.g. avoiding poking into structs in ABI3.
Doubling down, maybe ABI 4 should be focused on reducing the need for extension authors to even write C code. IOW, a new API that would provide
On 27Feb2020 1940, Barry Warsaw wrote: the services for Cython-like tool would consume. The vision would be that most extension authors could just write some type-hinted flavor of standard Python and this tool would generate the C for the extension module.
Most extension authors also need to write C code to interact with C modules - speedups are the minority and native lib wrappers are the majority, last time I counted. So that "standard" Python would also have to support native calls, and ideally in a less complicated way than via ctypes :)
I agree. There's a layering here of providing a C FFI that Cython and any such tool would compile down to. Plus embedding scenarios I think mean we can't get away from providing some C FFI.
I would vote for being very anti-optimisation in any new stable ABI, though.
I do think that we have all learned that simpler is better when it comes to a stable ABI and we would naturally trend towards that anyway. :)
But I think we all need to agree we want to work on a new stable ABI before we start trying to design it. ;)
-Brett
Import, __getattr__ and __call__ are the only Python operations you need to use any Python object, or to provide to be usable by any Python object. If we focus on those as the actual "channel" between runtime and extension, we can move a lot of boilerplate statically into the extension, where it is then safe from future changes in the runtime.
I also think there's a possibility to reintroduce optimisations through optional native interfaces, similar to how QueryInterface works in COM (e.g. you could ask an object for its Mapping interface, and if it gives you one, use it directly - otherwise you __getattr__ for "__getitem__" and go from there -- and all of this can bundle up inside the static boilerplate that gets compiled into the extension).
So you'd periodically want to recompile extensions to take advantage of any updates to the helper library, but fundamentally we've only got primitive-value-to-Python conversions, import, getattr and calls to worry about (not) breaking.
And you wouldn't get an automatic speed improvement in interop code just from compiling it, but I'm okay with that. If you can't get enough of a speedup in your actual native code to overcome the cost of interop, then you should stick with your already-working and far more portable Python code :)
Cheers, Steve