capi-sig August 2018

capi-sig@python.org

10 participants
10 discussions

[capi-sig]Adding an official (minimal) Cython-like tool.

by Eric Snow

Not that long ago Brett, Barry, and I were talking about how to get extension authors to move away from the C-API. Cython is the obvious choice, but it isn't an official tool nor does it necessarily make sense to make it one. Regardless, it would help all parties involved if there *were* an official tool that was part of CPython (i.e. in the repo). Cython could build on top of it and extension authors would be encouraged to use it or Cython. If such a thing makes sense, I figure we would follow the pattern set by asyncio (relative to twisted). -eric

3 years, 1 month

[capi-sig]Re: New pythoncapi GitHub organization and creation of a test suite for the Python C API

by Jeroen Demeyer

On 2018-08-31 10:31, Victor Stinner wrote: > I don't wante to write unit tests with Cython, cffi or anything else, > because I really need very thin control on everything. You don't need "very thin control on *everything*". You need very thin control on specific parts of the testsuite. And that's the nice thing about Cython: you can easily shift the amount of control that you take. You can write the usual unittest classes and test_FOO methods where you don't care much about control but write fine-tuned C code (either using Cython syntax or C syntax) where you do want control. Of course you may have other reasons to not want Cython...

3 years, 1 month

[capi-sig]New pythoncapi GitHub organization and creation of a test suite for the Python C API

by Victor Stinner

Hi, To be able to work on a fork of CPython, I decided to create a new GitHub organization: https://github.com/pythoncapi/ I didn't want to work on my personal fork of CPython (vstinner/cpython). The organization has a single member: me, but I invited Eric Snow, Barry Warsaw and Pablo Galindo. Tell me if you want to join! I moved my documentation there, the source of https://pythoncapi.readthedocs.io/ website: https://github.com/pythoncapi/pythoncapi/ And I just forked CPython: https://github.com/pythoncapi/cpython I will experiment to implement the new C API in this fork, to not bother the upstream CPython. I started with a very simple change, replace PyTuple_GET_ITEM() macro with a function call. I also started to write unit tests: https://github.com/pythoncapi/cpython/blob/pythoncapi/capi/tests/test_tuple… I quickly experimented to write my own C unit test framework, but I'm not really satisfied of the "verbose" API. I'm looking at googletest and Check, does someone have an experience with a C unit test framework? I don't wante to write unit tests with Cython, cffi or anything else, because I really need very thin control on everything. Victor

3 years, 1 month

[capi-sig]Re: Adding an official (minimal) Cython-like tool.

by Jeroen Demeyer

On 2018-07-31 18:42, Brett Cannon wrote: > Yes, what Eric is suggesting is a baseline tool like Cython be added to > Python itself so it becomes the minimum, common tool that we point all > extension authors to. You didn't answer Victor's "stupid question": we can already point extension authors to Cython. Why do we need a new tool which will very likely have less features than the already-existing Cython? > then we will have to provide an FFI compiler for people to use in at least > the simple cases. Cython is *not* an FFI tool. It can be used for FFI but that's just one of its many use cases.

3 years, 2 months

[capi-sig]Re: Adding an official (minimal) Cython-like tool.

by Jeroen Demeyer

On 2018-08-04 00:23, Barry Warsaw wrote: > I think if we were to go down this path, we’d need something that comes with CPython by default. Why?

3 years, 2 months

[capi-sig]Re: Let's change to C API!

by Jeroen Demeyer

On 2018-07-31 14:52, Victor Stinner wrote: > For C extensions which use (1), what's the benefit of moving to (2)? Source compatibility between Python releases: the source code compiles and works the same way on all Python versions (above a certain minimum version).

3 years, 2 months

[capi-sig]Let's change to C API!

by Victor Stinner

Hi, Last year, I gave a talk at the Language Summit (during Pycon) to explain that CPython should become 2x faster to remain competitive. IMHO all attempts to optimize Python (CPython forks) have failed because they have been blocked by the C API which implies strict constraints. I started to write a proposal to change the C API to hide implementation details, to prepare CPython for future changes. It allows to experimental optimization ideas without loosing support for C extensions. C extensions are a large part of the Python success. They are also the reason why PyPy didn't replace CPython yet. PyPy cpyext remains slower than CPython because PyPy has to mimick CPython which adds a significant overhead (even if PyPy developers are working *hard* to optimize it). I created a new to discuss how to introduce backward incompatible changes in the C API without breaking all C extensions: http://pythoncapi.readthedocs.io/ The source can be found at: https://github.com/vstinner/pythoncapi/ I would like to create a team of people who want to work on this project: CPython, PyPy, Cython and anyone who depends on the C API. Contact me in private if you want to be added to the GitHub project. I propose to discuss on the capi-sig mailing list since I would like to involve people from various projects, and I don't want to bother you with the high traffic of python-dev. Victor PS: I added some people as BCC ;-)

3 years, 2 months

[capi-sig]Re: Adding an official (minimal) Cython-like tool.

by Jeroen Demeyer

On 2018-08-01 21:07, Robert Bradshaw wrote: > +1. There has been some of this, e.g. PEP 509 and and the ongoing > discussions with PEP 580. There could be more. There are also some unresolved issues in the import system: see https://bugs.python.org/issue32797 and https://github.com/python/cpython/pull/6653 It would be good to resolve that, as it seriously hampers development of Cython code on Python 3.

3 years, 2 months

[capi-sig]Replacing Python C API with CFFI

by Simon Cross

Hello! Summary: It looks like fixing the Python C API is a lot of work and might fail, so I was wondering if it would be better to put effort into reducing the C API by making more use of CFFI within the Python standard library instead? Motivation: * It feels like the C API is currently very broad because it provides a second interface layer to a large chunk of the Python object space (in addition to the one visible to Python code). * I think it was Armin Rigo's observation when writing CFFI that there is a better API that already exists -- i.e. C types and function signatures. * If we're going to do a lot of work, would it not be better to push instead for moving towards CFFI as the interface between C and Python? I'm not sure that this is orthogonal to Victor's current proposal or not -- it might just be one route to achieving it. Advantages over the current route: * There's a clearer direction for people to head it, so it's easier to contribute. * A lot of work has already been done (e.g. it works for PyPy, a lot of libraries already exist that use it). Maybe Cython is an equally good option to CFFI in this scenario -- I don't know Cython though, so I have no deep opinion on that. Schiavo Simon

3 years, 2 months

[capi-sig]Re: Adding an official (minimal) Cython-like tool.

by Jeroen Demeyer

On 2018-08-01 21:07, Robert Bradshaw wrote: > Herein lies the primary difficulty with option #3. I have a hard time > imagining what a low-level, minimal-ish "core" of Cython would be. > What features would be omitted or simplifications would be made? > > In addition, I'm concerned that putting a barrier somewhere in the > middle of the existing Cython project, with one half in third-party > and the other part of CPython, would significantly harm development > velocity and make adoption and distribution more difficult. +1 I don't think that distutils + setuptools is a good example to follow.

3 years, 2 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

capi-sig August 2018