[Numpy-discussion] Numpy 1.11.0b2 released

[Pauli Virtanen]

04.02.2016, 07:56, Nathaniel Smith kirjoitti:
[clip]
> Whoops, got distracted talking about the results and forgot to say --
> I guess we should think about how to combine these? I like the
> information on warnings, because it helps gauge the impact of
> deprecations, which is a thing that takes a lot of our attention. But
> your approach is clearly fancier in terms of how it parses the test
> results. (Do you think the fanciness is worth it? I can see an
> argument for crude and simple if the fanciness ends up being fragile,
> but I haven't read the code -- mostly I was just being crude and
> simple because I'm lazy :-).)

The fanciness is essentially a question of implementation language and
ease of writing the reporting code. At 640 SLOC it's probably not so bad.

I guess it's reasonably robust --- the test report formats are unlikely
to change, and pip/virtualenv will probably continue to work esp. with
pinned pip version.

It should be simple to extract also the warnings from the test stdout.

I'm not sure if the order of test results is deterministic in
nose/py.test, so I don't know if just diffing the outputs always works.

Building downstream from source avoids future binary compatibility issues.

[clip]
> Maybe it should be uploading the reports somewhere? So there'd be a
> readable "what's currently broken by 1.x" page, plus with persistent
> storage we could get travis to flag if new additions to the release
> branch causes any new failures to appear? (That way we only have to
> remember to look at the report manually once per release, instead of
> constantly throughout the process.)

This is probably possible to implement. Although, I'm not sure how much
added value this is compared to travis matrix, eg.
https://travis-ci.org/pv/testrig/

Of course, if the suggestion is that the results are generated on
somewhere else than on travis, then that's a different matter.

-- 
Pauli Virtanen