"Micro-optimisations can speed up CPython"
Hi, I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython Regards Antoine.
Hi,
Thank you for the links.
- PyObject *value = GETLOCAL(oparg);
+ PyObject *value = GETLOCAL((unsigned)oparg);
Oh, I remember that I proposed to change oparg type to unsigned when
Demur wrote the WORDCODE change. I even wrote a patch, but I got a
segfault and was unable to understand why. It seems like I should try
again, it seems worth it ;-)
Victor
2017-05-29 14:13 GMT+02:00 Antoine Pitrou
Hi,
I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.co...
29.05.17 15:13, Antoine Pitrou пише:
I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-) And the following article should be especially interesting for Victor: https://www.corsix.org/content/why-are-slots-so-slow The part of optimizations already are applied in 3.6 and 3.7, but `a + b` still is slower than `a.__add__(b)`.
On Mon, May 29, 2017 at 05:15:43PM +0300, Serhiy Storchaka wrote:
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-)
the author's answer: https://twitter.com/corsix/status/869200284261789696 😉 Cheers, -- zmo
On 5/29/2017 10:36 AM, Guyzmo via Python-Dev wrote:
On Mon, May 29, 2017 at 05:15:43PM +0300, Serhiy Storchaka wrote:
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-)
the author's answer: https://twitter.com/corsix/status/869200284261789696
"Because getting patches merged is much effort", which is sometimes true. Peter Cawley is already bpo user 'Peter Cawley' with linked github account 'corsix', but no CLA. If one or more core developers were to commit to reviewing his patches, one of them (or someone else) could email him (see bpo user profile) conveying our interest and such commitment(s), and request that he sign the CLA, open an issue for one patch, and open a git PR for the patch. -- Terry Jan Reedy
That's very interesting -- thanks for sharing, Serhiy and Antoine. Very
relevant!
On Mon, May 29, 2017 at 10:15 AM, Serhiy Storchaka
29.05.17 15:13, Antoine Pitrou пише:
I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-)
And the following article should be especially interesting for Victor:
https://www.corsix.org/content/why-are-slots-so-slow
The part of optimizations already are applied in 3.6 and 3.7, but `a + b` still is slower than `a.__add__(b)`.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/benhoyt% 40gmail.com
On Mon, May 29, 2017 at 7:15 AM, Serhiy Storchaka
29.05.17 15:13, Antoine Pitrou пише:
I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-)
And the following article should be especially interesting for Victor:
Is the author of that article using non-standard terminology? The article doesn't appear to be about __slots__ at all.
The part of optimizations already are applied in 3.6 and 3.7, but `a + b` still is slower than `a.__add__(b)`.
-- --Guido van Rossum (python.org/~guido)
On Mon, May 29, 2017 at 07:27:05PM -0700, Guido van Rossum wrote:
Is the author of that article using non-standard terminology? The article doesn't appear to be about __slots__ at all.
Quoting Armin Ronacher: By far my biggest problem with the language is the stupid slot system. I do not mean the __slots__ but the internal type slots for special methods. http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/ Armin shows the history of these "slots" (or however they're called) back to Python in 1990! static number_methods int_as_number = { intadd, /*tp_add*/ intsub, /*tp_subtract*/ intmul, /*tp_multiply*/ ... etc I don't know where the name "slot" comes for from the various tp_* members (fields? attributes? slots?) but I'm pretty sure I've heard it before. I don't normally pay attention to what happens in the C implementation, but I'm fairly sure he's referring to these tp_* thingies. Oh yes, here you go: https://docs.python.org/2/extending/newtypes.html#type-methods refers to "tp_* slot" regularly. What does "tp" stand for? Type something, I guess. -- Steve
<Blush>
tp_ stands for "type". I wrote all that in 1990 and indeed made up the term
slot -- it's like an entry in a C++ vtable, where apparently they are also
called slots by some folks. I also did a major update to the machinery here
in the early 2000s (when typeobject.c grew from 50 lines to 5000) but
didn't update the terminology.
I feel a bit offended that Armin calls it "the stupid slot system" -- this
architecture an important part of what makes C extensions so flexible. I
will happily see it replaced by something better, if it can be found, but I
don't think it's fair to call it "stupid". However I no longer am
interested in maintaining that code myself, and I can't bear to read those
blog posts (they're rambling and sound like someone's research notes).
On Mon, May 29, 2017 at 7:47 PM, Steven D'Aprano
On Mon, May 29, 2017 at 07:27:05PM -0700, Guido van Rossum wrote:
Is the author of that article using non-standard terminology? The article doesn't appear to be about __slots__ at all.
Quoting Armin Ronacher:
By far my biggest problem with the language is the stupid slot system. I do not mean the __slots__ but the internal type slots for special methods.
http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/
Armin shows the history of these "slots" (or however they're called) back to Python in 1990!
static number_methods int_as_number = { intadd, /*tp_add*/ intsub, /*tp_subtract*/ intmul, /*tp_multiply*/ ... etc
I don't know where the name "slot" comes for from the various tp_* members (fields? attributes? slots?) but I'm pretty sure I've heard it before. I don't normally pay attention to what happens in the C implementation, but I'm fairly sure he's referring to these tp_* thingies.
Oh yes, here you go:
https://docs.python.org/2/extending/newtypes.html#type-methods
refers to "tp_* slot" regularly. What does "tp" stand for? Type something, I guess.
-- Steve _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ guido%40python.org
-- --Guido van Rossum (python.org/~guido)
Steven D'Aprano wrote:
What does "tp" stand for? Type something, I guess.
I think it's just short for "type". There's an old tradition in C of giving member names a short prefix reminiscent of the type they belong to. Not sure why, maybe someone thought it helped readability. -- Greg
30.05.17 09:06, Greg Ewing пише:
Steven D'Aprano wrote:
What does "tp" stand for? Type something, I guess.
I think it's just short for "type". There's an old tradition in C of giving member names a short prefix reminiscent of the type they belong to. Not sure why, maybe someone thought it helped readability.
In early ages of C structures didn't create namespaces, and member names were globals.
On Mon, May 29, 2017 at 11:16 PM, Serhiy Storchaka
30.05.17 09:06, Greg Ewing пише:
Steven D'Aprano wrote:
What does "tp" stand for? Type something, I guess.
I think it's just short for "type". There's an old tradition in C of giving member names a short prefix reminiscent of the type they belong to. Not sure why, maybe someone thought it helped readability.
In early ages of C structures didn't create namespaces, and member names were globals.
That's nonsense. The reason is greppability. -- --Guido van Rossum (python.org/~guido)
30.05.17 18:38, Guido van Rossum пише:
On Mon, May 29, 2017 at 11:16 PM, Serhiy Storchaka
mailto:storchaka@gmail.com> wrote: 30.05.17 09:06, Greg Ewing пише:
Steven D'Aprano wrote:
What does "tp" stand for? Type something, I guess.
I think it's just short for "type". There's an old tradition in C of giving member names a short prefix reminiscent of the type they belong to. Not sure why, maybe someone thought it helped readability.
In early ages of C structures didn't create namespaces, and member names were globals.
That's nonsense. The reason is greppability.
Good reason!
On Tue, May 30, 2017 at 09:38:55PM +0300, Serhiy Storchaka wrote:
In early ages of C structures didn't create namespaces, and member names were globals.
That's nonsense. The reason is greppability.
Good reason!
The first time I heard about prefixing struct members was to allow tricks like this: x86_64-linux-gnu/bits/stat.h 94:# define st_atime st_atim.tv_sec /* Backward compatibility. */ 95:# define st_mtime st_mtim.tv_sec 96:# define st_ctime st_ctim.tv_sec Which is relatively safe thanks to the prefix. David
On Tue, May 30, 2017 at 11:49 AM, David Wilson
On Tue, May 30, 2017 at 09:38:55PM +0300, Serhiy Storchaka wrote:
In early ages of C structures didn't create namespaces, and member names were globals.
That's nonsense. The reason is greppability.
Good reason!
The first time I heard about prefixing struct members was to allow tricks like this:
x86_64-linux-gnu/bits/stat.h 94:# define st_atime st_atim.tv_sec /* Backward compatibility. */ 95:# define st_mtime st_mtim.tv_sec 96:# define st_ctime st_ctim.tv_sec
Which is relatively safe thanks to the prefix.
Believe me that is not why the prefix convention was introduced. -- --Guido van Rossum (python.org/~guido)
On 5/30/2017 2:18 PM, Guido van Rossum wrote:
On Tue, May 30, 2017 at 11:49 AM, David Wilson
mailto:dw+python-dev@hmmz.org> wrote: On Tue, May 30, 2017 at 09:38:55PM +0300, Serhiy Storchaka wrote:
> > In early ages of C structures didn't create namespaces, and member > > names were globals.
> >That's nonsense. The reason is greppability.
> Good reason!
The first time I heard about prefixing struct members was to allow tricks like this:
x86_64-linux-gnu/bits/stat.h 94:# define st_atime st_atim.tv_sec /* Backward compatibility. */ 95:# define st_mtime st_mtim.tv_sec 96:# define st_ctime st_ctim.tv_sec
Which is relatively safe thanks to the prefix.
Believe me that is not why the prefix convention was introduced.
Sure, but grepping was not the reason either. Serhiy had the right explanation.
On 30/05/2017 16:38, Guido van Rossum wrote:
On Mon, May 29, 2017 at 11:16 PM, Serhiy Storchaka
mailto:storchaka@gmail.com> wrote: 30.05.17 09:06, Greg Ewing пише:
Steven D'Aprano wrote:
What does "tp" stand for? Type something, I guess.
I think it's just short for "type". There's an old tradition in C of giving member names a short prefix reminiscent of the type they belong to. Not sure why, maybe someone thought it helped readability.
In early ages of C structures didn't create namespaces, and member names were globals.
That's nonsense. The reason is greppability.
It does seem that far enough back, struct member names were all one space, standing for little more than their offset and type: "Two structures may share a common initial sequence of members; that is, the same member may appear in two different structures if it has the same type in both and if all previous members are the same in both. (Actually, the compiler checks only that a name in two different structures has the same type and offset in both, ... )" -- The C Programming Language, K&R 1978 (p197). With these Python name spaces, you're really spoiling us, Mr BDFL. Jeff Allen
Serhiy Storchaka wrote:
In early ages of C structures didn't create namespaces, and member names were globals.
That would certainly explain the origins of it, but I'm pretty sure it wasn't the case by the time Python was invented. So Guido must have liked it for other reasons. -- Greg
But I do stand corrected. I had forgotten that.
On May 30, 2017 3:49 PM, "Greg Ewing"
Serhiy Storchaka wrote:
In early ages of C structures didn't create namespaces, and member names were globals.
That would certainly explain the origins of it, but I'm pretty sure it wasn't the case by the time Python was invented. So Guido must have liked it for other reasons.
-- Greg _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% 40python.org
29.05.17 17:15, Serhiy Storchaka пише:
29.05.17 15:13, Antoine Pitrou пише:
I hope readers won't get bothered by what is mostly a link to blogpost (well, two of them :-)), but I suspect there at least 2 or 3 people who might be interested in the following analysis: https://www.corsix.org/content/compilers-cpython-interpreter-main-loop http://www.corsix.org/content/micro-optimisations-can-speed-up-cpython
Interesting articles, thank you. I wonder why the author doesn't propose his patches for CPython. Does he fear that CPython can become faster than Lua? ;-)
And the following article should be especially interesting for Victor:
https://www.corsix.org/content/why-are-slots-so-slow
The part of optimizations already are applied in 3.6 and 3.7, but `a + b` still is slower than `a.__add__(b)`.
See https://bugs.python.org/issue30509. All optimizations already are applied in 3.7 by Victor. In these microbenchmarks 3.7 is much faster than 2.7 and 3.5. But still there is some overhead due to excess intermediate levels.
participants (14)
-
Antoine Pitrou
-
Ben Hoyt
-
David Wilson
-
Glenn Linderman
-
Greg Ewing
-
Guido van Rossum
-
Guido van Rossum
-
Guyzmo
-
Jeff Allen
-
Jonathan Cronin
-
Serhiy Storchaka
-
Steven D'Aprano
-
Terry Reedy
-
Victor Stinner