Python JIT Compilation Thoughts

Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!

On Mon, May 25, 2020 at 3:05 AM <redradist@gmail.com> wrote:
see numba: http://numba.pydata.org/ Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe? -CHB
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

There was also Pyston. https://blog.pyston.org/ On Tue, May 26, 2020 at 11:42 Brett Cannon <brett@python.org> wrote:
-- --Guido (mobile)

Thanks Guido !! I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project

Hello, On Wed, 27 May 2020 08:18:11 -0000 redradist@gmail.com wrote:
A recent attempt to collect known Python JITs/compilers: https://github.com/pfalcon/awesome-python-compilers (well, definitely more recent than https://wiki.python.org/moin/PythonImplementations#Compilers). And even that is not complete, I'm still too lazy to add mypyc there, as hopefully everyone knows about it anyway. -- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 27 May 2020 20:45:15 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote: []
So, it's added to that humble list.
But mypyc doesn't seem to be a JIT compiler anyway.
I don't think there's a big difference between a JIT and AOT compiler, even from end-users' point of view. Today they want their programs to run faster, tomorrow they want to send a binary to their grandma...
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 5/25/20 6:02 AM, redradist@gmail.com wrote:
Hi all,
I do not know maybe it was already discussed ...
It's been extensively discussed and attempted.
but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
Be careful with "will": nothing is simple in this arena. --Ned.

I have found very nice attempt of JIT for CPython https://github.com/tonybaloney/Pyjion It is working under .NET Runtime, but if it is possible in this project I believe it is possible in CPython as well Also to integrate JIT compilcation now it not so hard because we can use llvm ability for JIT Also I will be less hard if CPython started switching it code-base to modern C++ at least C++11 that will make more maintainable code than raw C

January 4, 2021 12:55 PM, redradist@gmail.com wrote:
I suggest that you fork cpython on github and provide a proof of concept. Doesn't need to be the final deal, but at least enough to show that it actually works with cpython. If you can provide a poc, your case will be much more convincing.

For me it easier to do with C++, I know C, but with C++ it is more maintainable and easier to add new feature and also easier to refactor ... I am not sure if CPython maintainers team will appropriate that I add C++ in code base ... But I can try, I will try to find time to create small proof of concept in C++

you really want to look at other Python JIT methods, including ones based on LLVM: numba: https://numba.pydata.org/ and the old unladen swallow project: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html and of course, not LLVM based, but PyPy is worth a look. In short: a lot of very smart people have put a lot of work into these efforts -- "m not saying you don't have good ideas, but it's clearly a very hard problem. -CHB On Mon, Jan 4, 2021 at 10:24 AM <redradist@gmail.com> wrote:
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Mon, May 25, 2020 at 3:05 AM <redradist@gmail.com> wrote:
see numba: http://numba.pydata.org/ Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe? -CHB
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

There was also Pyston. https://blog.pyston.org/ On Tue, May 26, 2020 at 11:42 Brett Cannon <brett@python.org> wrote:
-- --Guido (mobile)

Thanks Guido !! I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project

Hello, On Wed, 27 May 2020 08:18:11 -0000 redradist@gmail.com wrote:
A recent attempt to collect known Python JITs/compilers: https://github.com/pfalcon/awesome-python-compilers (well, definitely more recent than https://wiki.python.org/moin/PythonImplementations#Compilers). And even that is not complete, I'm still too lazy to add mypyc there, as hopefully everyone knows about it anyway. -- Best regards, Paul mailto:pmiscml@gmail.com

Hello, On Wed, 27 May 2020 20:45:15 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote: []
So, it's added to that humble list.
But mypyc doesn't seem to be a JIT compiler anyway.
I don't think there's a big difference between a JIT and AOT compiler, even from end-users' point of view. Today they want their programs to run faster, tomorrow they want to send a binary to their grandma...
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com

On 5/25/20 6:02 AM, redradist@gmail.com wrote:
Hi all,
I do not know maybe it was already discussed ...
It's been extensively discussed and attempted.
but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
Be careful with "will": nothing is simple in this arena. --Ned.

I have found very nice attempt of JIT for CPython https://github.com/tonybaloney/Pyjion It is working under .NET Runtime, but if it is possible in this project I believe it is possible in CPython as well Also to integrate JIT compilcation now it not so hard because we can use llvm ability for JIT Also I will be less hard if CPython started switching it code-base to modern C++ at least C++11 that will make more maintainable code than raw C

January 4, 2021 12:55 PM, redradist@gmail.com wrote:
I suggest that you fork cpython on github and provide a proof of concept. Doesn't need to be the final deal, but at least enough to show that it actually works with cpython. If you can provide a poc, your case will be much more convincing.

For me it easier to do with C++, I know C, but with C++ it is more maintainable and easier to add new feature and also easier to refactor ... I am not sure if CPython maintainers team will appropriate that I add C++ in code base ... But I can try, I will try to find time to create small proof of concept in C++

you really want to look at other Python JIT methods, including ones based on LLVM: numba: https://numba.pydata.org/ and the old unladen swallow project: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html and of course, not LLVM based, but PyPy is worth a look. In short: a lot of very smart people have put a lot of work into these efforts -- "m not saying you don't have good ideas, but it's clearly a very hard problem. -CHB On Mon, Jan 4, 2021 at 10:24 AM <redradist@gmail.com> wrote:
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
participants (9)
-
Antoine Pitrou
-
Brett Cannon
-
Christopher Barker
-
David Mertz
-
edwin@211mainstreet.net
-
Guido van Rossum
-
Ned Batchelder
-
Paul Sokolovsky
-
redradist@gmail.com