Awesome ;-)
--
Ludovic Gasc (GMLudo)
Lead Developer Architect at ALLOcloud
https://be.linkedin.com/in/ludovicgasc
2017-02-19 7:45 GMT+01:00 INADA Naoki <songofacandy(a)gmail.com>:
> On Wed, Feb 8, 2017 at 12:59 AM, Victor Stinner
> <victor.stinner(a)gmail.com> wrote:
> > Hi,
> >
> > FYI CPython 3.7 is now faster than CPython 2.7 on most benchmarks. It
> > would be interesting to investigate why 12 benchmarks are still slower
> > to check if we missed something obvious.
> >
> > Comparison, ignoring differences smaller than 10%:
> >
> > haypo@speed-python$ python3 -m perf compare_to
> > 2017-01-03_11-17-2.7-5988caffbff9.json
> > 2017-02-06_07-15-default-e06af4027546.json -G --min-speed=10
> > Slower (12):
> > - python_startup_no_site: 3.11 ms +- 0.02 ms -> 8.33 ms +- 0.04 ms:
> > 2.68x slower (+168%)
> > - python_startup: 6.42 ms +- 0.04 ms -> 14.1 ms +- 0.1 ms: 2.20x slower
> (+120%)
>
>
> The top two slower benchmark is startup. There are some unavoidable
> reasons (io, importlib,
> and more rich types based on abc).
> But I sent two pull requests reduce cost of site.py.
>
> * http://bugs.python.org/issue29585
> * http://bugs.python.org/issue29592
>
> If they are merged, (python_startup - python_startup_no_site) will be
> very close to Python 2.7.
>
> For more optimization, I think we need to write some function in C for
> importlib.
> _______________________________________________
> Speed mailing list
> Speed(a)python.org
> https://mail.python.org/mailman/listinfo/speed
>
Hi Victor,
Codespeed supports both git and Github. When you configure a project in the
admin, you should see a dropdown for the type of revision control. Github
is probably best so there is no need for locally cloning the git repository.
Anything else that might be a blocker?
Miquel
On Mon, Feb 20, 2017 at 1:58 PM Victor Stinner <victor.stinner(a)gmail.com>
wrote:
> Hi,
>
> The website speed.python.org is made of different tools which were
> written for Mercurial, but CPython moved to Git. These tools should be
> updated.
>
> Tools:
>
> * The Django application "codespeed"
> * scripts/bench_cpython.py and scripts/bench_revisions.py of
> https://github.com/python/performance/
> * my private tool to benchmark a patch -- short bash script, sorry I
> should put it in python/performance/scripts/ as well
> * maybe others
>
> Moreover, all benchmark results was generated using Mercurial commit
> identifiers. We must remove all of them and recompute all results
> using Git commit identifiers. It's not a big deal. It's just that
> someone has to do it.
>
> I spent almost a year to fix all these minor things to get a working
> speed.python.org, so I'm not ready to do it right now. I may do the
> required changes later ;-)
>
> If codespeed doesn't support Git, I suggest to drop the list of
> changes between two commits on the https://speed.python.org/changes/
> tab. I'm now trying to only compute benchmarks manually something like
> once per week, because the UI doesn't handle well too many points, and
> because performance don't change every day ;-) The changes/ tab is
> very slow when there is a long list of commits, whereas it's very easy
> to get this list using Git directly, GitHub, or whatever. I don't
> think that it's very useful on the website.
>
> Victor
> _______________________________________________
> Speed mailing list
> Speed(a)python.org
> https://mail.python.org/mailman/listinfo/speed
>
Hi,
The website speed.python.org is made of different tools which were
written for Mercurial, but CPython moved to Git. These tools should be
updated.
Tools:
* The Django application "codespeed"
* scripts/bench_cpython.py and scripts/bench_revisions.py of
https://github.com/python/performance/
* my private tool to benchmark a patch -- short bash script, sorry I
should put it in python/performance/scripts/ as well
* maybe others
Moreover, all benchmark results was generated using Mercurial commit
identifiers. We must remove all of them and recompute all results
using Git commit identifiers. It's not a big deal. It's just that
someone has to do it.
I spent almost a year to fix all these minor things to get a working
speed.python.org, so I'm not ready to do it right now. I may do the
required changes later ;-)
If codespeed doesn't support Git, I suggest to drop the list of
changes between two commits on the https://speed.python.org/changes/
tab. I'm now trying to only compute benchmarks manually something like
once per week, because the UI doesn't handle well too many points, and
because performance don't change every day ;-) The changes/ tab is
very slow when there is a long list of commits, whereas it's very easy
to get this list using Git directly, GitHub, or whatever. I don't
think that it's very useful on the website.
Victor
On Wed, Feb 8, 2017 at 12:59 AM, Victor Stinner
<victor.stinner(a)gmail.com> wrote:
> Hi,
>
> FYI CPython 3.7 is now faster than CPython 2.7 on most benchmarks. It
> would be interesting to investigate why 12 benchmarks are still slower
> to check if we missed something obvious.
>
> Comparison, ignoring differences smaller than 10%:
>
> haypo@speed-python$ python3 -m perf compare_to
> 2017-01-03_11-17-2.7-5988caffbff9.json
> 2017-02-06_07-15-default-e06af4027546.json -G --min-speed=10
> Slower (12):
> - python_startup_no_site: 3.11 ms +- 0.02 ms -> 8.33 ms +- 0.04 ms:
> 2.68x slower (+168%)
> - python_startup: 6.42 ms +- 0.04 ms -> 14.1 ms +- 0.1 ms: 2.20x slower (+120%)
The top two slower benchmark is startup. There are some unavoidable
reasons (io, importlib,
and more rich types based on abc).
But I sent two pull requests reduce cost of site.py.
* http://bugs.python.org/issue29585
* http://bugs.python.org/issue29592
If they are merged, (python_startup - python_startup_no_site) will be
very close to Python 2.7.
For more optimization, I think we need to write some function in C for
importlib.
Hi,
I annoyed many of you on this mailing list with all my WTF performance
issues. I consider that I now succeeded to get benchmarks stable
enough to become "usable" (reliable). Two days ago, I gave a talk "How
to run a stable benchmark" at FOSDEM (Brussels, Belgium) on my
findings.
Slides and video of my talk are now online, enjoy!
https://fosdem.org/2017/schedule/event/python_stable_benchmark/
"Takeaway":
* Tune system to run benchmarks: python3 -m perf system tune
* Stop using timeit!: python3 -m timeit STMT ⇒ python3 -m perf timeit STMT
* Use perf and its documentation! http://perf.rtfd.io/
Victor
