Python JIT Compilation Thoughts
Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
On Mon, May 25, 2020 at 3:05 AM <redradist@gmail.com> wrote:
I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
see numba: http://numba.pydata.org/ Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe? -CHB
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/A3VF6D... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Mon, May 25, 2020, 5:33 PM Christopher Barker
see numba: http://numba.pydata.org/
Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe?
Numba is great. Very easy and very fast. Pypy experimented with an LLVM target but decided it's own JIT was better for various reasons.
It was Unladen Swallow that used LLVM, but it was early on in LLVM's JIT life and they unfortunately had to spend a bunch of time fixing LLVM which ate up the time they had to do the experiment. On Mon, May 25, 2020 at 2:36 PM Christopher Barker <pythonchb@gmail.com> wrote:
On Mon, May 25, 2020 at 3:05 AM <redradist@gmail.com> wrote:
I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
see numba: http://numba.pydata.org/
Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe?
-CHB
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/A3VF6D... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JJQNOX... Code of Conduct: http://python.org/psf/codeofconduct/
There was also Pyston. https://blog.pyston.org/ On Tue, May 26, 2020 at 11:42 Brett Cannon <brett@python.org> wrote:
It was Unladen Swallow that used LLVM, but it was early on in LLVM's JIT life and they unfortunately had to spend a bunch of time fixing LLVM which ate up the time they had to do the experiment.
On Mon, May 25, 2020 at 2:36 PM Christopher Barker <pythonchb@gmail.com> wrote:
On Mon, May 25, 2020 at 3:05 AM <redradist@gmail.com> wrote:
I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
see numba: http://numba.pydata.org/
Also, I"m pretty sure one of the efforts to make a faster python was based on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe?
-CHB
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/A3VF6D... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JJQNOX... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JY53RH... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido (mobile)
Thanks Guido !! I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project
Hello, On Wed, 27 May 2020 08:18:11 -0000 redradist@gmail.com wrote:
Thanks Guido !!
I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project
A recent attempt to collect known Python JITs/compilers: https://github.com/pfalcon/awesome-python-compilers (well, definitely more recent than https://wiki.python.org/moin/PythonImplementations#Compilers). And even that is not complete, I'm still too lazy to add mypyc there, as hopefully everyone knows about it anyway. -- Best regards, Paul mailto:pmiscml@gmail.com
On Wed, 27 May 2020 21:24:08 +0300 Paul Sokolovsky <pmiscml@gmail.com> wrote:
Hello,
On Wed, 27 May 2020 08:18:11 -0000 redradist@gmail.com wrote:
Thanks Guido !!
I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project
A recent attempt to collect known Python JITs/compilers: https://github.com/pfalcon/awesome-python-compilers (well, definitely more recent than https://wiki.python.org/moin/PythonImplementations#Compilers). And even that is not complete, I'm still too lazy to add mypyc there, as hopefully everyone knows about it anyway.
I'm quite sure that not everyone knows about mypyc... far from it. But mypyc doesn't seem to be a JIT compiler anyway. Regards Antoine.
Hello, On Wed, 27 May 2020 20:45:15 +0200 Antoine Pitrou <solipsis@pitrou.net> wrote: []
A recent attempt to collect known Python JITs/compilers: https://github.com/pfalcon/awesome-python-compilers (well, definitely more recent than https://wiki.python.org/moin/PythonImplementations#Compilers). And even that is not complete, I'm still too lazy to add mypyc there, as hopefully everyone knows about it anyway.
I'm quite sure that not everyone knows about mypyc... far from it.
So, it's added to that humble list.
But mypyc doesn't seem to be a JIT compiler anyway.
I don't think there's a big difference between a JIT and AOT compiler, even from end-users' point of view. Today they want their programs to run faster, tomorrow they want to send a binary to their grandma...
Regards
Antoine.
-- Best regards, Paul mailto:pmiscml@gmail.com
On 5/25/20 6:02 AM, redradist@gmail.com wrote:
Hi all,
I do not know maybe it was already discussed ...
It's been extensively discussed and attempted.
but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
Be careful with "will": nothing is simple in this arena. --Ned.
Ned Batchelder wrote:
On 5/25/20 6:02 AM, redradist@gmail.com wrote:
Hi all, I do not know maybe it was already discussed ... It's been extensively discussed and attempted. but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !! Be careful with "will": nothing is simple in this arena. --Ned.
And what was the result of discussions ? There are also Cranelift toolchain that is written in Rust and it provides as for very good API ...
On Mon, Jun 29, 2020 at 2:50 AM <redradist@gmail.com> wrote:
Ned Batchelder wrote:
On 5/25/20 6:02 AM, redradist@gmail.com wrote:
Hi all, I do not know maybe it was already discussed ... It's been extensively discussed and attempted. but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !! Be careful with "will": nothing is simple in this arena. --Ned.
And what was the result of discussions ?
That it's very hard and PyPy is the only one that has pulled off a JIT and kept it going with many other attempts that didn't make it as far. -Brett
There are also Cranelift toolchain that is written in Rust and it provides as for very good API ... _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AJJWKG... Code of Conduct: http://python.org/psf/codeofconduct/
I have found very nice attempt of JIT for CPython https://github.com/tonybaloney/Pyjion It is working under .NET Runtime, but if it is possible in this project I believe it is possible in CPython as well Also to integrate JIT compilcation now it not so hard because we can use llvm ability for JIT Also I will be less hard if CPython started switching it code-base to modern C++ at least C++11 that will make more maintainable code than raw C
January 4, 2021 12:55 PM, redradist@gmail.com wrote:
I have found very nice attempt of JIT for CPython https://github.com/tonybaloney/Pyjion
It is working under .NET Runtime, but if it is possible in this project I believe it is possible in CPython as well
I suggest that you fork cpython on github and provide a proof of concept. Doesn't need to be the final deal, but at least enough to show that it actually works with cpython. If you can provide a poc, your case will be much more convincing.
For me it easier to do with C++, I know C, but with C++ it is more maintainable and easier to add new feature and also easier to refactor ... I am not sure if CPython maintainers team will appropriate that I add C++ in code base ... But I can try, I will try to find time to create small proof of concept in C++
you really want to look at other Python JIT methods, including ones based on LLVM: numba: https://numba.pydata.org/ and the old unladen swallow project: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html and of course, not LLVM based, but PyPy is worth a look. In short: a lot of very smart people have put a lot of work into these efforts -- "m not saying you don't have good ideas, but it's clearly a very hard problem. -CHB On Mon, Jan 4, 2021 at 10:24 AM <redradist@gmail.com> wrote:
For me it easier to do with C++, I know C, but with C++ it is more maintainable and easier to add new feature and also easier to refactor ...
I am not sure if CPython maintainers team will appropriate that I add C++ in code base ...
But I can try, I will try to find time to create small proof of concept in C++ _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6UM65... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
participants (9)
-
Antoine Pitrou
-
Brett Cannon
-
Christopher Barker
-
David Mertz
-
edwin@211mainstreet.net
-
Guido van Rossum
-
Ned Batchelder
-
Paul Sokolovsky
-
redradist@gmail.com