[SciPy-User] bugs in scipy.stats

Wed Jun 12 01:26:57 EDT 2013

On Wed, Jun 12, 2013 at 12:37 AM, Oleksandr Huziy <guziy.sasha at gmail.com> wrote:
> Thank you Josef,
>
> for the detailed description. Though I have one more question concerning the
> workflow.
>
> After I've cloned the project locally can I develop and run tests from
> inside the directory without reinstalling it? (I suppose build is necessary
> when fortran code changes), but I would like to be able to edit and run the
> code from the same place.. (this would permit for example to use it as a
> project in my favourite IDE). But when I try to run test_morestats.py from
> inside the project I get the following exception:
>
> Traceback (most recent call last):
>   File
> "/home/san/.IntelliJIdea11/config/plugins/python/helpers/pycharm/utrunner.py",
> line 113, in <module>
>     modules = [loadSource(a[0])]
>   File
> "/home/san/.IntelliJIdea11/config/plugins/python/helpers/pycharm/utrunner.py",
> line 44, in loadSource
>     module = imp.load_source(moduleName, fileName)
>   File "/home/san/Python/scipy_project/scipy/stats/tests/test_morestats.py",
> line 14, in <module>
>     import scipy.stats as stats
>   File "/home/san/Python/scipy_project/scipy/__init__.py", line 120, in
> <module>
>     raise ImportError(msg)
> ImportError: Error importing scipy: you cannot import scipy while
>         being in scipy source directory; please exit the scipy source
>         tree first, and relaunch your python intepreter.
>
> How do you work on it? I understand that installing it (using virtualenv,
> which I am currently using for scipy testing) and then running tests in a
> different place would work, but is it the only way, it does not seem
> efficient.

I usually run nosetests directly in a separate shell window, which
makes it easy to run just specific test modules or individual tests

nosetests path_to_test_module
nosetests path_to_test_module:testfunction_name    (IIRC)

It uses whichever scipy is on the python path. So if you build
inplace, then you should be able to edit, run nosetests, commit for
pure python code. And the test_module can be, but does not need to be
inside scipy, for example run a test module that is in a source tree
when the build/installed scipy is somewhere else.

The alternative using scipy.stats.tests in a separate python
interpreter is much slower, since it's running all the tests for
scipy.stats. Full test without skipping the slow tests takes 5 minutes
or so.

For me working on scipy is more complicated, since I cannot build it
on Windows, but that's what I do for statsmodels.
I usually use Eclipse for editing the source with inplace build of
extensions, spyder to run some examples, and nosetests in a shell for
the tests.

There are some utility scripts for compiling and testing scipy, but
for pure python code it's a detour that's not necessary.

Cheers,

Josef

>
> Cheers
> --
> Sasha
>
>
>
>
> 2013/6/11 <josef.pktd at gmail.com>
>
>> On Tue, Jun 11, 2013 at 1:58 PM,  <josef.pktd at gmail.com> wrote:
>> > On Tue, Jun 11, 2013 at 1:24 PM, Oleksandr Huziy <guziy.sasha at gmail.com>
>> > wrote:
>> >> Hi Josef,
>> >>
>> >> could you, please, list the functions which need to be tested?
>> >> And the link to the testing approach that you'd prefer me to use
>> >> unittest,
>> >> nose, doctest? I am not experienced tester but really want to help.
>> >
>> > Hi Sasha,
>> >
>> > I didn't run the test coverage on scipy.stats in a long time
>> > This was my old list (2009) which is very outdated
>> > https://github.com/scipy/scipy/issues/1554
>> >
>> > All tests are run with nose, scipy doesn't have doctests. The testing
>> > guidelines are at
>> > https://github.com/numpy/numpy/blob/master/doc/TESTS.rst.txt
>> >
>> > The pattern for the tests can be seen in the test suite
>> > https://github.com/scipy/scipy/tree/master/scipy/stats/tests
>> > especially test_stats.py and test_morestats.py, and those for mstats
>> >
>> > One check that would also be very helpful is to try out different
>> > kinds of arguments.
>> > For example, I think there might still be problems with 2d arrays in
>> > some functions. Some will raise ValueErrors if they cannot handle 2d
>> > arrays, but some might just return incorrect numbers.
>> >
>> > Example: I never looked closely at `mood` which has unit tests, so a
>> > quick try:
>> >>>> stats.mood(np.random.randn(10,2), np.random.randn(15,2))
>> > (26.664783935766987, 1.2060935978310698e-156)
>> >>>> stats.mood(np.random.randn(10), np.random.randn(15))
>> > (-0.46553454010068451, 0.64154870791874163)
>> >
>> > the first result looks pretty weird
>> >
>> > In these cases we should add a `raise ValueError` or try to enhance it
>> > to 2d.
>>
>> If you check a function and it works as advertised, then this would
>> also be good to know
>> We still have an old milestone for the stats review, where we can also
>> note that everything is fine
>> https://github.com/scipy/scipy/issues?milestone=4&state=open
>>
>> or open a new issue and note the functions that you checked there, so
>> we have a record.
>>
>> Other hypothesis test that I never looked at in detail and tried out
>> only with "nice" numbers are
>> fligner, ansari, bartlett, ...
>>
>> >>> x, y = np.random.randn(10,2), np.random.randn(15,2)
>> >>> stats.bartlett(x, y)
>> (4.7695839486013287e-05, 0.99448967952524281)
>> >>> stats.bartlett(x.ravel(), y.ravel())
>> (0.00010231554134127248, 0.99192944393408666)
>>
>> looks also wrong
>>
>> (ttests are vectorized, ks tests raise an exception somewhere in the
>> code with 2d)
>>
>> Josef
>>
>>
>> >
>> > Thank you,
>> >
>> > Josef
>> >
>> >>
>> >> Cheers
>> >> --
>> >> Oleksandr (Sasha) Huziy
>> >>
>> >> _______________________________________________
>> >> SciPy-User mailing list
>> >> SciPy-User at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/scipy-user
>> >>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>