On Wed, 1 Aug 2018 at 12:08 Robert Bradshaw <robertwb@math.washington.edu> wrote:
On Wed, Aug 1, 2018 at 11:01 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Tue, Jul 31, 2018 at 4:33 PM Jeroen Demeyer <J.Demeyer@ugent.be> wrote:
On 2018-07-31 18:42, Brett Cannon wrote:
Yes, what Eric is suggesting is a baseline tool like Cython be added to Python itself so it becomes the minimum, common tool that we point all extension authors to.
You didn't answer Victor's "stupid question": we can already point extension authors to Cython. Why do we need a new tool which will very likely have less features than the already-existing Cython?
You and Victor are right to ask why we need to change anything here. In my initial post I wanted to present a possibility for discussion while the context was fresh from the other threads. I agree that the justification is relatively weak. :) However, there are some points to consider.
Consider our three options here:
- do nothing (status quo); Cython remains a popular and and effective tool
- promote Cython as *the* tool (and maybe use Cython for stdlib extension modules)
- promote Cython as *the* high-level tool, add an official low-level tool to CPython (and maybe use the new tool for stdlib)
Arguably, in all three cases we should improve collaboration with the Cython project. Option #2 (and #3) would definitely require that. How can we do better in that regard?
In the interest of encouraging less use of the C-API, at least #2 would make a lot of sense. I expect it would require more/better collaboration between the Python core team and the Cython project.
+1. There has been some of this, e.g. PEP 509 and and the ongoing discussions with PEP 580. There could be more.
Also, we should help Cython with technical aspects of the problem space, including both changes in CPython that will aid Cython and changes in Cython that will help users.
Finally, I suppose there's another option: factor out useful parts of Cython for inclusion in the stdlib.
=========================
FWIW, here's some thoughts on option #3 (basically my original post). Again, my goal was to introduce a possibility for discussion. At the least consider that there may some useful points here even if the overall idea isn't viable.
Cython is a great tool for the Python ecosystem. It is a large project developed over time, meaning that on the one hand it holds a lot of valuable knowledge and lessons learned. On the other hand, it probably doesn't look like it would under a green-field approach. :) Combine all that together and you might see the inspiration for an official tool based on Cython.
Effectively, the goal would be to factor out the low-level, minimal-ish core of Cython, if such a "core" is meaningful.
Herein lies the primary difficulty with option #3. I have a hard time imagining what a low-level, minimal-ish "core" of Cython would be. What features would be omitted or simplifications would be made? The crux of Cython is compiling from a python(-like) syntax to C, end-to-end, which is the bulk of the project, and it's hard to "subset" this. In particular, we have prioritized adding features that require support in the language end-to-end (e.g. C++ support, from syntax to type checking to code generation) that is not easily built as a layer on top. (I suppose it's conceivable that one could make the syntax, type system, and code generation portions more modular and pluggable, but that's probably be a signifiant undertaking.)
In addition, I'm concerned that putting a barrier somewhere in the middle of the existing Cython project, with one half in third-party and the other part of CPython, would significantly harm development velocity and make adoption and distribution more difficult. (E.g. right now one gets new Cython features on already released versions of CPython; how would that story look if these features depended on upgrading the CPython portion of the library.) I also don't think users would be served by two entry points, each presenting it's own (Python-like, I'm assuming, unless the proposal is to develop a completely independent IR for human and machine consumption that spans Python and C) syntax, but with different extensions and restrictions.
There's also the flip side that Cython has to chase after the C API rather than being considered a part of it. Stefan and the Cython team do an admirable job of keeping up (and we all try to keep them in the loop when we think something might affect them), but if the C API was directly used by something like Cython that was in the stdlib then it would potentially help keep everything in sync with less effort.
But that's obviously just speculation on my part and who knows if anyone would have the time to even attempt this idea. :)
-Brett
It could be an interesting and illuminating exercise to try to define such a core, however. (And there certainly improvement/modernization we could bring to Cython itself.)
The result would be a new stdlib library, as well as a simple tool in the CPython repo. The development of asyncio, relative to twisted, demonstrates the sort of effort I had in mind, though it's not a perfect parallel. :)
There are several advantages to the low-level tool/lib:
- we could use the tool for stdlib extension modules (if we want to), whereas we're reluctant to use a third-party tool like Cython currently
- the CPython test suite would include tests for the tool/lib, meaning changes that break the them would be blocked from merging, whereas Cython has to deal with such breaking changes after the fact
- if built on top of the low-level tool, Cython would be easier to maintain (presumably) since the project would have to do less and wouldn't have to adapt nearly as much to changes in CPython
- an opportunity to distill the concepts and functionality of Cython (like happened with asyncio/twisted)
I concede that there are a lot of assumptions there. :) Furthermore, there are serious downsides/obstacles:
- someone would have to do the work (always the biggest obstacle!)
- tools like Cython would have to make significant changes to take advantage of the new tool
- there's a high risk of regressions and other bugs in both the new tool and in Cython
Regardless, this would not make any sense unless Stefan (and the Cython project) were in favor of the new tool and heavily involved in its design (and implementation). Given the reaction thus far I don't see that happening. :)
then we will have to provide an FFI compiler for people to use in at least the simple cases.
Cython is *not* an FFI tool. It can be used for FFI but that's just one of its many use cases.
FWIW, there has been talk in the past of bringing CFFI (or something based on it) into the stdlib (in part as a replacement for ctypes). IIRC, it hasn't happened because no one was interested enough to do the work. If we ended up doing it, I think it would coexist with Cython (or something similar) just fine, as Stefan implied when he explained about different tools for different needs.
-eric
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org
capi-sig mailing list -- capi-sig@python.org To unsubscribe send an email to capi-sig-leave@python.org