[Pandas-dev] Fast py2/py3 testing, fast vbench

[Wes McKinney]

On Wed, Mar 20, 2013 at 4:05 PM, yoval p. <yoval at gmx.com> wrote:
> I've made some improvement to the tooling we have
> for development, just making sure everyone is
> aware of what's available.
>
> - closed GH3099, caching cython build artifacts
> when running setup.py.
> - setup.py now checks for BUILD_CACHE_DIR envar
> so you can enable it without touch the source code
> - Once enabled, with a warm cache testing py26/27/32/33
> takes only a couple of minutes compares with travis' ~15
> on a quad core machine
> - if caching is enabled (for future commits, the envar is sufficiant)
> test_perf.sh will run much faster.
> - i've added an option to filter vbench by regex when running
> test_perf.sh.
>
> Quick iteration makes everything easier, I hope these
> changes do that.
>
> Here's an example of all of the above, comparing two adjacent
> commits on a reduced set of vbenches in 1min flat:
>
> λ export BUILD_CACHE_DIR="/tmp/.pandas_build_cache/"
> λ time ./test_perf.sh -b 18c7e6c -t 18c7e6c^ -r reindex
> ...
> Results:
>                                     t_head  t_baseline      ratio
> name
> dataframe_reindex                   0.3726      0.3726     1.0000
> reindex_fillna_backfill_float32     0.0961      0.0961     1.0000
> reindex_fillna_pad_float32          0.0959      0.0959     1.0000
> frame_reindex_upcast               17.7334     17.7334     1.0000
> reindex_daterange_backfill          0.1649      0.1649     1.0000
> reindex_fillna_pad                  0.1052      0.1052     1.0000
> reindex_daterange_pad               0.1757      0.1757     1.0000
> reindex_frame_level_align           1.0109      1.0109     1.0000
> reindex_fillna_backfill             0.1035      0.1035     1.0000
> reindex_frame_level_reindex         0.9586      0.9586     1.0000
> frame_reindex_columns               0.3101      0.3101     1.0000
> reindex_multiindex                  1.1427      1.1427     1.0000
>
> Columns: test_name | target_duration [ms] | baseline_duration [ms] | ratio
>
> - a Ratio of 1.30 means the target commit is 30% slower then the baseline.
>
> Target [18c7e6c] : BLD: check for BUILD_CACHE_DIR envar in setup.py
> Baseline [18c7e6c] : BLD: check for BUILD_CACHE_DIR envar in setup.py
>
>
> *** Results were also written to the logfile at
> '/home/user1/src/pandas/vb_suite.log'
>
>
> real    0m58.561s
> user    0m52.699s
> sys    0m1.645s
> _______________________________________________
> Pandas-dev mailing list
> Pandas-dev at python.org
> http://mail.python.org/mailman/listinfo/pandas-dev
>

This is really great. Caching of builds is a no-brainer and the vbench
suite has gotten quite large (surprised it's not more popular? we are
avant garde). Thanks y-p!