Re: [Speed] Do we want to stop vendoring source of third-party libraries with the benchmarks?
On Thu, 11 Feb 2016 18:35:29 +0000 Brett Cannon <brett@python.org> wrote:
Maybe we should just have a requirements.txt file for Python 2 and another for Python 3 that are pegged to specific versions? We could even install things into a venv for isolation.
How does this impact interaction with the benchmarks suite? E.g. does it increase the time of running a couple of benchmarks? Does it make it easier or harder to benchmark a work-in-progress patch for whatever interpreter?
If we go this route then we could make the benchmark suite a package on PyPI and have people install the benchmark suite and then have instructions to run pip on the requirements files that we embed in the package.
I'm not fond of encouraging random users to run the benchmarks suite without understanding what they're doing, and starting throwing around pointless numbers and misconceptions about performance (which are then very hard to fight since people tend to be irrationally captivated by "performance numbers"). The benchmarks suite is mostly a tool for developers of Python implementations, not the greater public.
Having the benchmarks suite only available through hg or git kind of discourages those tendencies.
Regards
Antoine.
On Fri, Feb 12, 2016, 04:26 Antoine Pitrou <solipsis@pitrou.net> wrote:
On Thu, 11 Feb 2016 18:35:29 +0000 Brett Cannon <brett@python.org> wrote:
Maybe we should just have a requirements.txt file for Python 2 and another for Python 3 that are pegged to specific versions? We could even install things into a venv for isolation.
How does this impact interaction with the benchmarks suite?
Upon installation you would need to run pip install -r requirements 3.txt
to get the dependencies.
E.g. does it increase the time of running a couple of benchmarks?
No
Does
it make it easier or harder to benchmark a work-in-progress patch for whatever interpreter?
I think only on Windows because of the lack of symlink support.
If we go this route then we could make the benchmark suite a package on PyPI and have people install the benchmark suite and then have instructions to run pip on the requirements files that we embed in the package.
I'm not fond of encouraging random users to run the benchmarks suite without understanding what they're doing, and starting throwing around pointless numbers and misconceptions about performance (which are then very hard to fight since people tend to be irrationally captivated by "performance numbers"). The benchmarks suite is mostly a tool for developers of Python implementations, not the greater public.
Having the benchmarks suite only available through hg or git kind of discourages those tendencies.
That's fine, but then I would still want requirements files so we stop vendoring.
Brett
Regards
Antoine.
Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed
participants (2)
-
Antoine Pitrou -
Brett Cannon