[Numpy-discussion] known failure test decorator not treated as expected by nose

[Jonathan March]

On Fri, Oct 8, 2010 at 6:46 PM, Ralf Gommers <ralf.gommers at googlemail.com>wrote:

>
>
> On Fri, Oct 8, 2010 at 1:09 AM, Jonathan March <jmarch at enthought.com>wrote:
>
>> On Wed, Oct 6, 2010 at 6:17 PM,  <josef.pktd at gmail.com> wrote:
>> > On Wed, Oct 6, 2010 at 7:08 PM, Jonathan March <jmarch at enthought.com>
>> wrote:
>> >> It appears that the numpy testing decorators for skipping and for
>> >> known failure should behave similarly to each other, at least from
>> >> their descriptions here:
>> >>
>> http://projects.scipy.org/numpy/wiki/TestingGuidelines#known-failures-skipping-tests
>> >>
>> >> Yet in the following example, run under nose, they behave quite
>> differently:
>> >>
>> >> ====
>> >> from numpy.testing import  dec
>> >>
>> >> @dec.knownfailureif( True, "known failure test message")
>> >> def test_will_fail():
>> >>    pass
>> >>
>> >> @dec.skipif( True, "skip test message")
>> >> def test_to_skip():
>> >>    pass
>> >> ====
>> >>
>> >> The skipped test is marked as "S" as expected, but when the
>> >> KnownFailureTest exception is raised, nose handles it as an error
>> >> (marked "E" rather than "K" as expected, and printing the stack
>> >> trace.)
>> >>
>> >> It's also interesting that nose has a --no-skip option but no
>> >> corresponding option for ignoring known failure decorators.
>> >>
>> >> Why the discrepancy? Who is out of step with whom? This is with numpy
>> >> 1.4.0 and nosetests 0.11.3
>> >
>> > just as additional information
>> >
>> > This only happens on the command line, with nosetests packagename
>> > but not with the function, packagename.test()
>> >
>>
>
> Knownfailure is a numpy-specific plugin, which needs to be loaded for tests
> decorated with @dec.knownfailureif to be marked as K instead of E. This
> plugin loading is done by the NoseTester class (defined in
> numpy/testing/nosetester.py) which is used when tests are run through
> numpy.test().
>
> Using 'nosetests' on the command line skips this plugin loading, hence the
> error. Nose does have a "skip" plugin so that never gives an error.
>
>
>> > Josef
>>
>>
>> Numpy issue, nose issue, or user issue?
>>
>
> User issue.
>

Thank you, Ralf. Very clear. I've opened a ticket on the corresponding
documentation:
http://projects.scipy.org/numpy/ticket/1630

Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101009/0e15ff47/attachment.html>