A proposal for a new tool to be implemented - It is often the case that developer write Code in Python and then convert to a C extension module for performance regions. A C extension module has a lot of boiler plate code - for instance the Structures required for each class, the functions for Module initialization etc. My Idea is a simple tool that uses introspection tools to take a Python module and to generate the relevant boiler plate for the module - including blank functions for the module classes and for methods. This tool would use type annotations (if given) to make sensible choices for parameter and attribute types, including using int and float directly rather than Internal objects (depending on tool options). The idea is that the C extension module would present the same 'API' as the original Python module. The tool would not attempt to write the C code within the functions/methods. I am aware of the challenges writing this tool, but I think a tool could make a good fist of the conversion. --------------------- What do you think ? Would you use the tool if provided ? If there is interested I probably could develop a version 0.1, but it would be useful to know if there is interest and if there is anyone else interested in joining me in this journey. -- Anthony Flury *Moble*: +44 07743 282707 *Home*: +44 (0)1206 391294 *email*: anthony.flury@btinternet.com
On Tue, May 10, 2022 at 1:20 AM Chris Angelico <rosuav@gmail.com> wrote:
Indeed -- Cython pretty much does all of this, while also letting you write the C part (or calling C code) easily as well :-) If you have a module written in Python, all you need to do to make it a C extension is run it through Cython :-) Cython uses its own annotation for typing -- but it is being extended to use the new pyton type annotations. It's a challenge, because Python types really are different than C types -- it's not always obvious what you want. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 10/05/22 8:05 pm, anthony.flury via Python-ideas wrote:
There's a much better tool in this space already: https://cython.org/ -- Greg
Cython seems to me rather different from what the original post was talking about. If I put class Foo: def bar(self): pass in a .pyx file and run it through Cython, the result is a 5600-line, 200-kilobyte lump of C that clearly isn't meant to be understood or modified by human beings. It is mostly boilerplate, but not the sort of boilerplate that could serve as the starting point for an extension module written (entirely) in C.
On Tue, May 10, 2022 at 8:07 PM Ben Rudiak-Gould <benrudiak@gmail.com> wrote:
Absolutely true. However, why would anyone want to hand-write an extension module in C? Hand writing a C extension is a serious pain. With Cython, you can either: - Write the extension module in Cython, and with enough care, get essentially the same thing as you would get with hand-written C or - Write the core logic in pure C, and then call that from Cython, then you are hand writing the core logic (which there is a reason to do by hand in C) and letting Cython handle all the boilerplate required to call it. The fact that Cython generates a lot of boilerplate for a do-almost-nothing module is an indication that there is actually a lot to be done, if you want the general case. What do you expect you'd get with the OP's suggestion? You might get the basic set up code, but that's actually not the hard part of a C extension anyway. I'm not sure it would buy you much. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Tue, May 10, 2022 at 1:20 AM Chris Angelico <rosuav@gmail.com> wrote:
Indeed -- Cython pretty much does all of this, while also letting you write the C part (or calling C code) easily as well :-) If you have a module written in Python, all you need to do to make it a C extension is run it through Cython :-) Cython uses its own annotation for typing -- but it is being extended to use the new pyton type annotations. It's a challenge, because Python types really are different than C types -- it's not always obvious what you want. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 10/05/22 8:05 pm, anthony.flury via Python-ideas wrote:
There's a much better tool in this space already: https://cython.org/ -- Greg
Cython seems to me rather different from what the original post was talking about. If I put class Foo: def bar(self): pass in a .pyx file and run it through Cython, the result is a 5600-line, 200-kilobyte lump of C that clearly isn't meant to be understood or modified by human beings. It is mostly boilerplate, but not the sort of boilerplate that could serve as the starting point for an extension module written (entirely) in C.
On Tue, May 10, 2022 at 8:07 PM Ben Rudiak-Gould <benrudiak@gmail.com> wrote:
Absolutely true. However, why would anyone want to hand-write an extension module in C? Hand writing a C extension is a serious pain. With Cython, you can either: - Write the extension module in Cython, and with enough care, get essentially the same thing as you would get with hand-written C or - Write the core logic in pure C, and then call that from Cython, then you are hand writing the core logic (which there is a reason to do by hand in C) and letting Cython handle all the boilerplate required to call it. The fact that Cython generates a lot of boilerplate for a do-almost-nothing module is an indication that there is actually a lot to be done, if you want the general case. What do you expect you'd get with the OP's suggestion? You might get the basic set up code, but that's actually not the hard part of a C extension anyway. I'm not sure it would buy you much. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
participants (5)
-
anthony.flury -
Ben Rudiak-Gould -
Chris Angelico -
Christopher Barker -
Greg Ewing