Speed August 2016

speed@python.org

9 participants
29 discussions

Re: [Speed] performance 0.1 (and 0.1.1) release

by Victor Stinner

Release early, release often: performance 0.1.2 has been released! The first version supporting Windows. I renamed the GitHub project from python/benchmarks to python/performance. All changes: * Windows is now supported * Add a new ``venv`` command to show, create, recrete or remove the virtual environment. * Fix pybench benchmark (update to perf 0.7.4 API) * performance now tries to install the ``psutil`` module on CPython for better system metrics in metadata and CPU pinning on Python 2. * The creation of the virtual environment now also tries ``virtualenv`` and ``venv`` Python modules, not only the virtualenv command. * The development version of performance now installs performance with "pip install -e <path_to_performance>" * The GitHub project was renamed from ``python/benchmarks`` to ``python/performance``. Victor

3 years, 10 months

Re: [Speed] Rename python/benchmarks GitHub project to python/performance?

by Brett Cannon

On Thu, 25 Aug 2016 at 15:08 Victor Stinner <victor.stinner(a)gmail.com> wrote: > Hi, > > For the first release of the "new" benchmark suite, I chose the name > "performance", since "benchmark" and "benchmarks" names were already > reserved on PyPI. It's the name of the Python module, but also of the > command line tool: "pyperformance". > > Since there is an "old" benchmark suite > (https://hg.python.org/benchmarks), PyPy has its benchmark suite, etc. > I propose to rename the GitHub project to "performance" to avoid > confusion. > > What do you think? > If you want to do then go ahead, but I don't think it will be a big issue in the grand scheme of things. > > Note: I'm not a big fan of the "performance" name, but I don't think > it matters much. The name only needs to be unique and available on > PyPI :-D > > By the way, I don't know if it's worth it to have a "pyperformance" > command line tool. You can already use "python3 -m performance ..." > syntax. But you have to recall the Python version used to install the > module. "python2 -m performance ..." doesn't work if you only > installed performance for Python 3! > As Antoine pointed out, if it doesn't matter what interpreter has the script installed to run the benchmarks against another interpreter than a script makes sense (but do keep it available through -m).

3 years, 10 months

Re: [Speed] Rename python/benchmarks GitHub project to python/performance?

by Antoine Pitrou

On Fri, 26 Aug 2016 00:07:53 +0200 Victor Stinner <victor.stinner(a)gmail.com> wrote: > > By the way, I don't know if it's worth it to have a "pyperformance" > command line tool. You can already use "python3 -m performance ..." > syntax. But you have to recall the Python version used to install the > module. "python2 -m performance ..." doesn't work if you only > installed performance for Python 3! Also, you may have several Python 3s installed (the system 3.4, a custom 3.4, a custom 3.5, a custom 3.6...) so a CLI script is much easier to use. Regards Antoine.

3 years, 10 months

Rename python/benchmarks GitHub project to python/performance?

by Victor Stinner

Hi, For the first release of the "new" benchmark suite, I chose the name "performance", since "benchmark" and "benchmarks" names were already reserved on PyPI. It's the name of the Python module, but also of the command line tool: "pyperformance". Since there is an "old" benchmark suite (https://hg.python.org/benchmarks), PyPy has its benchmark suite, etc. I propose to rename the GitHub project to "performance" to avoid confusion. What do you think? Note: I'm not a big fan of the "performance" name, but I don't think it matters much. The name only needs to be unique and available on PyPI :-D By the way, I don't know if it's worth it to have a "pyperformance" command line tool. You can already use "python3 -m performance ..." syntax. But you have to recall the Python version used to install the module. "python2 -m performance ..." doesn't work if you only installed performance for Python 3! Victor

3 years, 10 months

Re: [Speed] performance 0.1 (and 0.1.1) release

by Victor Stinner

2016-08-24 17:38 GMT+02:00 Victor Stinner <victor.stinner(a)gmail.com>: > Now the development version always install performance 0.1.1 (see > performance/requirements.txt). I should fix this to install the > development version of performance/ when it is run from the source > code (when setup.py is available in the parent directory?). FYI it's now fixed: the development version of the benchmark suite now installs the performance module using "pip install -e <path>" in development mode, but "pip install performance==x.y.z" in release mode. Victor

3 years, 10 months

performance 0.1 (and 0.1.1) release

by Victor Stinner

Hi, I released a first version of the Python benchmark suite. (Quickly followed by a 0.1.1 bugfix ;-)) It is now possible to install it using pip: python3 -m pip install performance And run it using: pyperformance run --python=python2 --rigorous -b all -o py2.json pyperformance run --python=python3 --rigorous -b all -o py3.json pyperformance compare py2.json py3.json Note: the "python3 -m performance ..." syntax works too. It creates virtual environments in ./venv/ subdirectory. (I may add an option to choose where to create them.) performance 0.1.1 version works well on Linux with CPython. There are some know issues on Windows: https://github.com/python/benchmarks/issues/5 I don't consider the PyPy support as stable yet. I used the "performance" name on PyPI, because "benchmark" and "benchmarks" are already reserved. The Python module is also named "performance" and comes with "pyperformance" script. I made a suble bugfix: requirements.txt now uses fixed version rather than ">=min_version". For example, "perf>=0.7.4" became "perf==0.7.4". I expect to get more reproductible benchmark results with fixed versions. Before a release, we should not forget to update dependencies to test the most recent versions of Python modules and applications. Now the development version always install performance 0.1.1 (see performance/requirements.txt). I should fix this to install the development version of performance/ when it is run from the source code (when setup.py is available in the parent directory?). Victor

3 years, 10 months

Re: [Speed] New benchmark suite for Python

by Victor Stinner

Done: I renamed "django" benchmark" to "django_template": https://github.com/python/benchmarks/commit/d674a99e3a9a10a29c44349b2916740… Victor 2016-08-21 19:36 GMT+02:00 Victor Stinner <victor.stinner(a)gmail.com>: > Le 21 août 2016 11:02 AM, "Maciej Fijalkowski" <fijall(a)gmail.com> a écrit : >> Let's not have it called "bzr" though because it gives the wrong >> impression. > > The benchmark was called "bzr_startup", bit I agree that "django" mus be > renamed to "django_template". There is an HTTP benchmark, tornado_http, > using the local link (server and client run in the same process if I recall > correctly). > > Victor

3 years, 10 months

Re: [Speed] New benchmark suite for Python

by Victor Stinner

Le 21 août 2016 11:02 AM, "Maciej Fijalkowski" <fijall(a)gmail.com> a écrit : > Let's not have it called "bzr" though because it gives the wrong > impression. The benchmark was called "bzr_startup", bit I agree that "django" mus be renamed to "django_template". There is an HTTP benchmark, tornado_http, using the local link (server and client run in the same process if I recall correctly). Victor

3 years, 10 months

Re: [Speed] New benchmark suite for Python

by Nick Coghlan

On 21 August 2016 at 19:02, Maciej Fijalkowski <fijall(a)gmail.com> wrote: > Let's not have it called "bzr" though because it gives the wrong > impression. The same way unladen swallow added a benchmark and called > it "django" while not representing django very well. That said, likely > not very many people use bzr, but still, would be good if it's called > bzr-pyc or simpler - have a benchmark that imports a whole bunch of > pyc from a big project (e.g. pypy :-) Yeah, I assume the use of bzr for this purpose is mainly an accident of history - adopting a non-trivial cross-platform Python command line application, rather than putting together a synthetic benchmark like Tools/importbench that may not be particular representative of real workloads (which can use techniques like lazy module imports to defer startup costs until those features are actually needed). Perhaps call the benchmark "bzr-startup" to emphasize the aim is to measure how long it takes bzr to start in general, moreso than how long it takes to print the help message? Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 years, 10 months

Re: [Speed] New benchmark suite for Python

by Maciej Fijalkowski

On Sun, Aug 21, 2016 at 7:38 AM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On 20 August 2016 at 02:50, Maciej Fijalkowski <fijall(a)gmail.com> wrote: >> Very likely just pyc import time > > As one of the import system maintainers, that's a number I consider > quite interesting and worth benchmarking :) > > It's also one of the key numbers for Linux distro Python usage, since > it impacts how responsive the system shell feels to developers and > administrators - an end user can't readily tell the difference between > "this shell is slow" and "this particular command I am running is > using a language interpreter with a long startup time", but an > interpreter benchmark suite can. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia Fair point, let's have such a benchmark. Let's not have it called "bzr" though because it gives the wrong impression. The same way unladen swallow added a benchmark and called it "django" while not representing django very well. That said, likely not very many people use bzr, but still, would be good if it's called bzr-pyc or simpler - have a benchmark that imports a whole bunch of pyc from a big project (e.g. pypy :-) Cheers, fijal

3 years, 10 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Speed August 2016