[Numpy-discussion] Created NumPy 1.7.x branch

[Ondřej Čertík]

On Mon, Jun 25, 2012 at 7:38 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Mon, Jun 25, 2012 at 6:39 PM, Travis Oliphant <travis at continuum.io> wrote:
>>
>> On Jun 25, 2012, at 7:21 PM, Fernando Perez wrote:
>
>>
>> For context, consider that for many years, the word "gratuitous" has been used in a non-derogatory way in the Python ecosystem to describe changes to semantics and syntax that don't have benefits significant enough to offset the pain it will cause to existing users.    That's why I used the word.   I am not trying to be derogatory.   I am trying to be clear that we need to respect existing users of NumPy more than we have done from 1.5 to 1.7 in the enthusiasm to make changes.
>>
>
> For reference, here's the (long) thread where this came to be:
>
> http://mail.scipy.org/pipermail/scipy-dev/2009-October/012958.html
>
> It's worth noting that at the time, the discussion was for an addition
> to *scipy*, not to numpy.  I don't know when things were moved over to
> numpy.
>
>
>> Working on the NumPy code base implies respecting the conventions that are already in place --- not just disregarding them and doing whatever we want.     I'm not really sure why I have to argue the existing users point of view so much recently.    I would hope that all of us would have the perspective that the people who have adopted NumPy deserve to be treated with respect.    The changes that grate on me are the ones that seem to take lightly existing users of NumPy.
>>
>
> I certainly appreciate the need to not break user habits/code, as we
> struggle with the very same issue in IPython all the time.  And
> obviously at this point numpy is 'core infrastructure' enough that
> breaking backwards compatibility in any way should be very strongly
> discouraged (things were probably a bit different back in 2009).
>
>>> I know that this particular issue grates you quite a bit, but I urge
>>> you to be fair in your appreciation of how it came to be: through the
>>> work of well-intentioned and thoughtful (but not omniscient) people
>>> when you weren't participating actively in numpy development.
>>
>> I'm trying very hard to be fair --- especially to changes like this.  What grates me are changes that affect our user base in a negative way --- specifically by causing code that used to work to no longer work or create alterations to real conventions.  This kind of change is just not acceptable if we can avoid it.   I'm really trying to understand why others do not feel so strongly about this, but I'm not persuaded by what I've heard so far.
>
> I just want to note that I'm not advocating for *any*
> backwards-compatibility breakage in numpy at this point... I was just
> providing context for a discussion that happened back in 2009, and in
> the scipy list.  I certainly feel pretty strongly at this point about
> the importance of preserving working code *today*, given the role of
> numpy at the 'root node' of the scipy ecosystem tree and the size of
> said tree.

I think that everybody strongly agrees that backward incompatible
changes should not be made.

Sometimes it can be more subtle,
see for example this numpy bug report in Debian:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589835

and read the dozens of emails that it generated, e.g.
http://lists.debian.org/debian-python/2010/07/msg00048.html, and so
on. I've been hit by this problem too, that's why I remember it --
suddenly many packages that depend on NumPy stopped working in a
subtle way and I had to spent hours figuring out what went wrong and
that the problem is not in h5py, but actually that NumPy has changed
its ABI, or more precisely the problem is described here (some new
members were added to a C datastructure):
http://lists.debian.org/debian-python/2010/07/msg00045.html
I am sure that this ABI change had to be done and there were good
reasons for it and this particular change probably even couldn't have
been avoided. But nevertheless it has caused headaches to a lot of
people downstream. I just looked into the release notes for NumPy
1.4.0 and didn't find this change nor how to fix it in there. I am
just posting this as a particular, concrete, real life example of
consequences for the end users.

My understanding is that Travis is simply trying to stress "We have to
think about the implications of our changes on existing users." and
also that little changes (with the best intentions!) that however mean
either a breakage or confusion for users (due to historical reasons)
should be avoided if possible. And I very strongly feel the same way.
And I think that most people on this list do as well.

But sometimes I guess mistakes are made anyway. What can be done to
avoid similar issues like with the polynomial order in the future?

Ondrej