On Thu, 11 Feb 2016 18:35:29 +0000
Brett Cannon <brett@python.org> wrote:
> Maybe we should just have a requirements.txt file for Python 2 and another
> for Python 3 that are pegged to specific versions? We could even install
> things into a venv for isolation.
How does this impact interaction with the benchmarks suite?
Upon installation you would need to run `pip install -r requirements 3.txt` to get the dependencies.
E.g. does it increase the time of running a couple of benchmarks?
No
Does
it make it easier or harder to benchmark a work-in-progress patch for
whatever interpreter?
I think only on Windows because of the lack of symlink support.
> If we go this route then we could make
> the benchmark suite a package on PyPI and have people install the benchmark
> suite and then have instructions to run pip on the requirements files that
> we embed in the package.
I'm not fond of encouraging random users to run the benchmarks suite
without understanding what they're doing, and starting throwing around
pointless numbers and misconceptions about performance (which are then
very hard to fight since people tend to be irrationally captivated by
"performance numbers"). The benchmarks suite is mostly a tool for
developers of Python implementations, not the greater public.
Having the benchmarks suite only available through hg or git kind of
discourages those tendencies.
That's fine, but then I would still want requirements files so we stop vendoring.
Brett