[Speed] Do we want to stop vendoring source of third-party libraries with the benchmarks?

[Brett Cannon]

On Fri, Feb 12, 2016, 04:26 Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Thu, 11 Feb 2016 18:35:29 +0000
> Brett Cannon <brett at python.org> wrote:
> > Maybe we should just have a requirements.txt file for Python 2 and
> another
> > for Python 3 that are pegged to specific versions? We could even install
> > things into a venv for isolation.
>
> How does this impact interaction with the benchmarks suite?
>

Upon installation you would need to run `pip install -r requirements 3.txt`
to get the dependencies.

E.g. does it increase the time of running a couple of benchmarks?


No

Does
> it make it easier or harder to benchmark a work-in-progress patch for
> whatever interpreter?
>

I think only on Windows because of the lack of symlink support.


> > If we go this route then we could make
> > the benchmark suite a package on PyPI and have people install the
> benchmark
> > suite and then have instructions to run pip on the requirements files
> that
> > we embed in the package.
>
> I'm not fond of encouraging random users to run the benchmarks suite
> without understanding what they're doing, and starting throwing around
> pointless numbers and misconceptions about performance (which are then
> very hard to fight since people tend to be irrationally captivated by
> "performance numbers").  The benchmarks suite is mostly a tool for
> developers of Python implementations, not the greater public.
>
> Having the benchmarks suite only available through hg or git kind of
> discourages those tendencies.
>

That's fine, but then I would still want requirements files so we stop
vendoring.

Brett


> Regards
>
> Antoine.
>
>
> _______________________________________________
> Speed mailing list
> Speed at python.org
> https://mail.python.org/mailman/listinfo/speed
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/speed/attachments/20160213/ab60a240/attachment-0001.html>