[Numpy-discussion] Questions about fixes for 1.9.0rc2

[Nathaniel Smith]

On 5 Jul 2014 00:07, "Charles R Harris" <charlesr.harris at gmail.com> wrote:
>
>
>
>
> On Fri, Jul 4, 2014 at 3:33 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Fri, Jul 4, 2014 at 10:31 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> > On Fri, Jul 4, 2014 at 3:15 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> >>
>> >> On Fri, Jul 4, 2014 at 9:48 PM, Charles R Harris
>> >> <charlesr.harris at gmail.com> wrote:
>> >> >
>> >> > On Fri, Jul 4, 2014 at 2:41 PM, Nathaniel Smith <njs at pobox.com>
wrote:
>> >> >>
>> >> >> On Fri, Jul 4, 2014 at 9:33 PM, Charles R Harris
>> >> >> <charlesr.harris at gmail.com> wrote:
>> >> >> >
>> >> >> > On Fri, Jul 4, 2014 at 2:09 PM, Nathaniel Smith <njs at pobox.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> On Fri, Jul 4, 2014 at 9:02 PM, Ralf Gommers
>> >> >> >> <ralf.gommers at gmail.com>
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> > On Fri, Jul 4, 2014 at 10:00 PM, Charles R Harris
>> >> >> >> > <charlesr.harris at gmail.com> wrote:
>> >> >> >> >>
>> >> >> >> >> On Fri, Jul 4, 2014 at 1:42 PM, Charles R Harris
>> >> >> >> >> <charlesr.harris at gmail.com> wrote:
>> >> >> >> >>>
>> >> >> >> >>> Sebastian Seberg has fixed one class of test failures due
to the
>> >> >> >> >>> indexing
>> >> >> >> >>> changes in numpy 1.9.0b1.  There are some remaining errors,
and
>> >> >> >> >>> in
>> >> >> >> >>> the
>> >> >> >> >>> case
>> >> >> >> >>> of the Matplotlib failures, they look to me to be Matplotlib
>> >> >> >> >>> bugs.
>> >> >> >> >>> The
>> >> >> >> >>> 2-d
>> >> >> >> >>> arrays that cause the error are returned by the overloaded
>> >> >> >> >>> _interpolate_single_key function in CubicTriInterpolator
that is
>> >> >> >> >>> documented
>> >> >> >> >>> in the base class to return a 1-d array, whereas the actual
>> >> >> >> >>> dimensions
>> >> >> >> >>> are
>> >> >> >> >>> of the form (n, 1). The question is, what is the best work
>> >> >> >> >>> around
>> >> >> >> >>> here
>> >> >> >> >>> for
>> >> >> >> >>> these sorts errors? Can we afford to break Matplotlib and
other
>> >> >> >> >>> packages on
>> >> >> >> >>> account of a bug that was previously accepted by Numpy?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > It depends how bad the break is, but in principle I'd say that
>> >> >> >> > breaking
>> >> >> >> > Matplotlib is not OK.
>> >> >> >>
>> >> >> >> I agree. If it's easy to hack around it and issue a warning for
now,
>> >> >> >> and doesn't have other negative consequences, then IMO we should
>> >> >> >> give
>> >> >> >> matplotlib a release or so worth of grace period to fix things.
>> >> >> >
>> >> >> >
>> >> >> > Here is another example, from skimage.
>> >> >> >
>> >> >> >
>> >> >> >
======================================================================
>> >> >> > ERROR: test_join.test_relabel_sequential_offset1
>> >> >> >
>> >> >> >
----------------------------------------------------------------------
>> >> >> > Traceback (most recent call last):
>> >> >> >   File "X:\Python27-x64\lib\site-packages\nose\case.py", line
197, in
>> >> >> > runTest
>> >> >> >     self.test(*self.arg)
>> >> >> >   File
>> >> >> >
>> >> >> >
>> >> >> >
"X:\Python27-x64\lib\site-packages\skimage\segmentation\tests\test_join.py",
>> >> >> > line 30, in test_relabel_sequential_offset1
>> >> >> >     ar_relab, fw, inv = relabel_sequential(ar)
>> >> >> >   File
>> >> >> >
"X:\Python27-x64\lib\site-packages\skimage\segmentation\_join.py",
>> >> >> > line 127, in relabel_sequential
>> >> >> >     forward_map[labels0] = np.arange(offset, offset +
len(labels0) +
>> >> >> > 1)
>> >> >> > ValueError: shape mismatch: value array of shape (6,) could not
be
>> >> >> > broadcast
>> >> >> > to indexing result of shape (5,)
>> >> >> >
>> >> >> > Which is pretty clearly a coding error. Unfortunately, the error
is
>> >> >> > in
>> >> >> > the
>> >> >> > package rather than the test.
>> >> >> >
>> >> >> > The only easy way to fix all of these sorts of things is to
revert
>> >> >> > the
>> >> >> > indexing changes, and I'm loathe to do that. Grrr...
>> >> >>
>> >> >> Ugh, that's pretty bad :-/. Do you really think we can't use a
>> >> >> band-aid over the new indexing code, though?
>> >> >
>> >> >
>> >> > Yeah, we can. But Sebastian doesn't have time and I'm unfamiliar
with
>> >> > the
>> >> > code, so it may take a while...
>> >>
>> >> Fair enough!
>> >>
>> >> I guess that if what are (arguably) bugs in matplotlib and
>> >> scikit-image are holding up the numpy release, then it's worth CC'ing
>> >> their mailing lists in case someone feels like volunteering to fix
>> >> it... ;-).
>> >
>> > I can do that ;) Doesn't help with the release though unless we want to
>> > document the errors in the release notes and tell folks to wait on the
next
>> > release of the packages.
>>
>> Oh, I meant, in case they want to fix numpy so that their packages
>> don't break :-).
>>
>
> I've filed issues with all the affected projects. Here is the current
status.
>
> matplotlib -- Reported, being fixed, should be in 1.4 in a few days.
> skimage -- Reported.
> scikit-learn -- Reported.
> tables -- Reported.
> statsmodels -- Reported, fixed in master.
> bottleneck -- Reported. IIRC, kwgoodman already knew of the changes.
> pyfits -- Reported to astropy.
> milk -- Reported.
> pandas -- Reportedly fixed in master.

That is a massive pile of affected projects :-(.

My worry is that if all these projects we know about are broken, then how
many other codebases that we aren't testing are also broken?

> If the issues are fixed in matplotlib and pandas I'd be inclined to
release as is with a mention of versions in the release notes.

Even if it's fixed in pandas master, how long until it's in user's hands?

-n

> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140705/31304396/attachment.html>