[Numpy-discussion] Release vs development testing.

[Ralf Gommers]

On Sun, May 12, 2019 at 4:41 PM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Sun, May 12, 2019 at 7:27 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>>
>> On Sun, May 12, 2019 at 3:56 PM Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>>
>>>
>>> On Sun, May 12, 2019 at 6:33 AM Julian Taylor <
>>> jtaylor.debian at googlemail.com> wrote:
>>>
>>>> On 12.05.19 14:58, Charles R Harris wrote:
>>>> > Hi All,
>>>> >
>>>> > NumPy currently distinguishes between release and development versions
>>>> > when running tests. Is there a good reason to continue this practice?
>>>> I
>>>> > ask, because with the last pytest release it would be convenient to
>>>> > always include `pytest.ini ` so that we can register markers. The
>>>> > presence of `pytest.ini` is how we distinguish betweendevelopment from
>>>> > release for testing purposes.
>>>> >
>>>>
>>>> One difference between development and release builds was that in
>>>> development releases numpy.testing throws errors on floating point
>>>> exceptions while the release version it did not.
>>>>
>>>
>> I'd prefer to keep this behavior. It's not clear to me if the proposal is
>> to change the behavior or not.
>>
>> If that is still the case removing the distinction could require a lot
>>>> of changes in upstream test suites that are not regularly run against
>>>> development builds.
>>>>
>>>> The motivation is not quite clear to me, can you please elaborate on
>>>> what you want to do.
>>>>
>>>
>>> NumPy pytest testing is NumPy specific and not used downstream like our
>>> nose testing framework was, so I don't see why that should affect other
>>> projects. What motivates this question is that the new version of pytest
>>> released yesterday raises warnings for non-registered markers,
>>> `pytest.mark.slow` in particular, and that was causing CI failures. The
>>> easiest way to register a mark is using `pytest.ini`, but we currently
>>> don't include that in released wheels, only in source releases.
>>>
>>
>> Adding a pytest.ini file in wheels should be perfectly fine I think,
>>
>>
> It is the absence of pytest.ini that makes it a release, for that is the
> file that turns warnings into errors.
>

Why don't we just always keep pytest.ini, and move the settings for
warnings-to-errors to runtests.py or tools/travis_test.sh?

It's not important how we check, all we need is some mechanism to prevent
new warnings creeping in. Same as we set -Wall for building in CI.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190513/e9eb0120/attachment.html>