[Python-Dev] Using Cython for the stdlib (was: Let's change to CAPI!)

[Steve Dower]

I don’t think there would be any trouble linking in the generated C files. The built in modules like this just have their *_init() functions called at startup, so possibly nothing would even change.

Top-posted from my Windows 10 phone

From: Brett Cannon
Sent: Wednesday, 1 August 2018 17:20
To: Stefan Behnel
Cc: python-dev at python.org
Subject: [Python-Dev] Using Cython for the stdlib (was: Let's change to CAPI!)

[new thread as this no longer has anything to do with Victor's proposal]

On Tue, 31 Jul 2018 at 13:42 Stefan Behnel <stefan_ml at behnel.de> wrote:
Antoine Pitrou schrieb am 31.07.2018 um 09:45:
> On Tue, 31 Jul 2018 09:27:03 +0200
> Jeroen Demeyer <J.Demeyer at UGent.be> wrote:
>> On 2018-07-31 08:58, Antoine Pitrou wrote:
>>> I think Stefan is right that we
>>> should push people towards Cython and alternatives, rather than direct
>>> use of the C API (which people often fail to use correctly, in my
>>> experience).  
>>
>> I know this probably isn't the correct place to bring it up, but I'm 
>> sure that CPython itself could benefit from using Cython. For example, 
>> most of the C extensions in Modules/ could be written in Cython.
> 
> We don't depend on any third-party Python modules.  Adding a Cython
> dependency for CPython development would be a tough sell.

I don't really want to get into that discussion (it's more about processes
than arguments), but let me note that the CPython development already has a
couple of dependencies, such as github and its bots, or tools like argument
clinic (admittedly included), make and a C compiler (not included), and a
text editor. It's not like it's free of tools that help in writing and
maintaining the code. That's pretty much the level at which I also see
Cython. It's more complex than argument clinic, but it otherwise serves a
similar need.

You're right that since Cython generates C code that we could check in like we do all of our other generated code we could view Cython as another external tool we rely on for maintaining the code base.
 


> Also, a C extension can be built-in (linked statically into the
> interpreter), which I think would be hard to do with Cython.

Someone recently contributed a feature of hiding the pyinit function for
the embedding case, so people do these things already. This could use the
normal inittab mechanism, for example. What I think you might be referring
to is that Cython modules require the CPython runtime to be initialised to
a certain extent, so you couldn't implement "sys" in Cython, for example.

I think the key thing is that on Windows all extension modules are built-in modules, so that use-case would need to be supported (I don't know Cython well enough to know whether this would be doable if we converted as much as possible to Cython itself).
 
But Jeroen is right, Cython should be a viable option for (most of?) the
extension modules in the stdlib. Whether the CPython core devs would accept
it in their workflow or not is a totally different question.

I think it's definitely something to consider, although I think Victor's proposal to clean up the C API so we have a clearly delineated private API, a version-specific API, and a stable ABI is a separate idea and so this should be its own thread (plus Victor's is easier to accomplish in the short-term ;) . 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180801/95bfd05f/attachment.html>