[Numpy-discussion] Release vs development testing.

Charles R Harris charlesr.harris at gmail.com
Mon May 13 13:25:57 EDT 2019


On Mon, May 13, 2019 at 3:02 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sun, May 12, 2019 at 4:41 PM Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Sun, May 12, 2019 at 7:27 AM Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Sun, May 12, 2019 at 3:56 PM Charles R Harris <
>>> charlesr.harris at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Sun, May 12, 2019 at 6:33 AM Julian Taylor <
>>>> jtaylor.debian at googlemail.com> wrote:
>>>>
>>>>> On 12.05.19 14:58, Charles R Harris wrote:
>>>>> > Hi All,
>>>>> >
>>>>> > NumPy currently distinguishes between release and development
>>>>> versions
>>>>> > when running tests. Is there a good reason to continue this
>>>>> practice? I
>>>>> > ask, because with the last pytest release it would be convenient to
>>>>> > always include `pytest.ini ` so that we can register markers. The
>>>>> > presence of `pytest.ini` is how we distinguish betweendevelopment
>>>>> from
>>>>> > release for testing purposes.
>>>>> >
>>>>>
>>>>> One difference between development and release builds was that in
>>>>> development releases numpy.testing throws errors on floating point
>>>>> exceptions while the release version it did not.
>>>>>
>>>>
>>> I'd prefer to keep this behavior. It's not clear to me if the proposal
>>> is to change the behavior or not.
>>>
>>> If that is still the case removing the distinction could require a lot
>>>>> of changes in upstream test suites that are not regularly run against
>>>>> development builds.
>>>>>
>>>>> The motivation is not quite clear to me, can you please elaborate on
>>>>> what you want to do.
>>>>>
>>>>
>>>> NumPy pytest testing is NumPy specific and not used downstream like our
>>>> nose testing framework was, so I don't see why that should affect other
>>>> projects. What motivates this question is that the new version of pytest
>>>> released yesterday raises warnings for non-registered markers,
>>>> `pytest.mark.slow` in particular, and that was causing CI failures. The
>>>> easiest way to register a mark is using `pytest.ini`, but we currently
>>>> don't include that in released wheels, only in source releases.
>>>>
>>>
>>> Adding a pytest.ini file in wheels should be perfectly fine I think,
>>>
>>>
>> It is the absence of pytest.ini that makes it a release, for that is the
>> file that turns warnings into errors.
>>
>
> Why don't we just always keep pytest.ini, and move the settings for
> warnings-to-errors to runtests.py or tools/travis_test.sh?
>
> It's not important how we check, all we need is some mechanism to prevent
> new warnings creeping in. Same as we set -Wall for building in CI.
>
>
I just checked that current wheels show the warnings when `numpy.test()` is
run with latest pytest.  However, moving the `pytest.ini` file into the
`numpy` directory is tricky, as we need to tell pytest where to find the
installed file (-c option). The simplest short time solution is to ignore
the warning, but long term I'm worried that the warning will become an
error as pytest is doing this because they want to clean up the their
implementation. Ideally there would be a better way to register the marks.
Note that we currently deal with the missing `pytest.ini` in wheels by
duplicating the warnings filters in `_pytesttester.py` using the pytest
command line.

Adding the `error` filter in the command line worries me, as I'm not sure
how the priorities work out, the command line is now appended to the
contents of `pytest.ini`.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190513/85e1f967/attachment.html>


More information about the NumPy-Discussion mailing list