[Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

[Wes Turner]

On Friday, January 26, 2018, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Hi,
>
> Well, I wrote https://faster-cpython.readthedocs.io/ website to answer
> to such question.
>
> See for example https://faster-cpython.readthedocs.io/mutable.html
> "Everything in Python is mutable".


Thanks!

"""

Contents:

   - Projects to optimize CPython 3.7
   <https://faster-cpython.readthedocs.io/cpython37.html>
   - Projects to optimize CPython 3.6
   <https://faster-cpython.readthedocs.io/cpython36.html>
   - Notes on Python and CPython performance, 2017
   <https://faster-cpython.readthedocs.io/notes_2017.html>
   - FAT Python <https://faster-cpython.readthedocs.io/fat_python.html>
   - Everything in Python is mutable
   <https://faster-cpython.readthedocs.io/mutable.html>
   - Optimizations
   <https://faster-cpython.readthedocs.io/optimizations.html>
   - Python bytecode <https://faster-cpython.readthedocs.io/bytecode.html>
   - Python C API <https://faster-cpython.readthedocs.io/c_api.html>
   - AST Optimizers
   <https://faster-cpython.readthedocs.io/ast_optimizer.html>
   - Register-based Virtual Machine for Python
   <https://faster-cpython.readthedocs.io/registervm.html>
   - Read-only Python <https://faster-cpython.readthedocs.io/readonly.html>
   - History of Python optimizations
   <https://faster-cpython.readthedocs.io/history.html>
   - Misc <https://faster-cpython.readthedocs.io/misc.html>
   - Kill the GIL? <https://faster-cpython.readthedocs.io/gil.html>
   - Implementations of Python
   <https://faster-cpython.readthedocs.io/implementations.html>
   - Benchmarks <https://faster-cpython.readthedocs.io/benchmarks.html>
   - Random notes about PyPy
   <https://faster-cpython.readthedocs.io/pypy.html>
   - Talks <https://faster-cpython.readthedocs.io/talks.html>
   - Links <https://faster-cpython.readthedocs.io/links.html>

 """

Pandas & Cython
https://pandas.pydata.org/pandas-docs/stable/enhancingperf.html

"Vaex uses memory mapping, zero memory copy policy and lazy computations
for best performance (no memory wasted)."
https://github.com/maartenbreddels/vaex



> Victor
>
> 2018-01-26 22:35 GMT+01:00 Pau Freixes <pfreixes at gmail.com>:
> > Hi,
> >
> > This mail is the consequence of a true story, a story where CPython
> > got defeated by Javascript, Java, C# and Go.
> >
> > One of the teams of the company where Im working had a kind of
> > benchmark to compare the different languages on top of their
> > respective "official" web servers such as Node.js, Aiohttp, Dropwizard
> > and so on.  The test by itself was pretty simple and tried to test the
> > happy path of the logic, a piece of code that fetches N rules from
> > another system and then apply them to X whatevers also fetched from
> > another system, something like that
> >
> > def filter(rule, whatever):
> >     if rule.x in whatever.x:
> >         return True
> >
> > rules = get_rules()
> > whatevers = get_whatevers()
> > for rule in rules:
> >     for whatever in whatevers:
> >         if filter(rule, whatever):
> >             cnt = cnt + 1
> >
> > return cnt
> >
> >
> > The performance of Python compared with the other languages was almost
> > x10 times slower. It's true that they didn't optimize the code, but
> > they did not for any language having for all of them the same cost in
> > terms of iterations.
> >
> > Once I saw the code I proposed a pair of changes, remove the call to
> > the filter function making it "inline" and caching the rule's
> > attributes, something like that
> >
> > for rule in rules:
> >     x = rule.x
> >     for whatever in whatevers:
> >         if x in whatever.x:
> >             cnt += 1
> >
> > The performance of the CPython boosted x3/x4 just doing these "silly"
> things.
> >
> > The case of the rule cache IMHO is very striking, we have plenty
> > examples in many repositories where the caching of none local
> > variables is a widely used pattern, why hasn't been considered a way
> > to do it implicitly and by default?
> >
> > The case of the slowness to call functions in CPython is quite
> > recurrent and looks like its an unsolved problem at all.
> >
> > Sure I'm missing many things, and I do not have all of the
> > information. This mail wants to get all of this information that might
> > help me to understand why we are here - CPython - regarding this two
> > slow patterns.
> >
> > This could be considered an unimportant thing, but its more relevant
> > than someone could expect, at least IMHO. If the default code that you
> > can write in a language is by default slow and exists an alternative
> > to make it faster, this language is doing something wrong.
> >
> > BTW: pypy looks like is immunized [1]
> >
> > [1] https://gist.github.com/pfreixes/d60d00761093c3bdaf29da025a004582
> > --
> > --pau
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180130/abd45439/attachment.html>