Hi All,
I have a couple of input on the benchmark suite.
I am currently using the Python Benchmark Suite https://github.com/python/performance for a customer and running into some issues. Due to specific circumstance, the net speed/bandwidth must be limited. As a result, the real time package checking and pulling step prior to the actual benchmark run was never able to finish (always time out somewhere during the bit pulling).
My request: is there a way to get a zipped (or git cloned version) of this benchmark, and stop the package pulling in real time prior to every run? In addition to net speed limit, we have to run the benchmark on multiple machines. Trying to wait for the same source codes being downloaded repeatedly for every run is really painfully, and not very efficient. The current implementation does not guarantee the exact same benchmarks are used for all runs. I feel git tag should be used to enforce consistency.
We used The Grand Unified Python Benchmark Suite https://hg.python.org/benchmarks in the past, and found that one was very easy to use, with far less dependency, and can be simply zipped and deployed easily.
Thanks,
Peter