[Numpy-discussion] Created NumPy 1.7.x branch

[Travis Oliphant]

>> 
>> I just want to note that I'm not advocating for *any*
>> backwards-compatibility breakage in numpy at this point... I was just
>> providing context for a discussion that happened back in 2009, and in
>> the scipy list.  I certainly feel pretty strongly at this point about
>> the importance of preserving working code *today*, given the role of
>> numpy at the 'root node' of the scipy ecosystem tree and the size of
>> said tree.
> 
> I think that everybody strongly agrees that backward incompatible
> changes should not be made.

> 
> Sometimes it can be more subtle,
> see for example this numpy bug report in Debian:

There are a lot of subtleties and different users and different expectations.    It does make it difficult to know the best course of action.  I appreciate the perspective of as many people as possible --- especially those who have managed code bases with a large number of users.   

What should have happened in this case, in my mind, is that NumPy 1.4.0 should have been 1.5.0 and advertised that there was a break in the ABI and that all extensions would have to be re-built against the new version.    This would have been some pain for one class of users (primarily package maintainers) and no pain for another class.   There was no API breakage.  We just needed to communicate clearly.     Because we guessed wrongly that the changes made did not change the ABI, we did not communicate clearly during the release.    This was a mistake.  I was a large part of that mistake.    I also understand the impact that the unsolved packaging problem in the Python community has created (at least for non-academic users and HPC users). 

Some take this example as you can't change the ABI.   That's not quite my perspective for what it's worth.   I don't think you should have a habit of changing the ABI (because it does create some hassle for downstream users), but especially today when there are many pre-packaged distributions of Python, occassional changes that require a re-compile of downstream dependencies does not constitute the kind of breakage I'm talking about.   The kind of breakage I'm talking about is the kind that causes code that used to work to stop working (either because it won't compile against the new headers) or because the behavior of operations changes in subtle ways.   Both kinds of changes have happened between 1.5.x and 1.7.x.    Some believe these changes are inconsequential.  I hope they are right.  I don't believe we have enough data to make that call, and there is some evidence I am aware of from people in other organizations that their are changes that will make upgrading difficult for people --- much more difficult than an ABI breakage would have been. 

You can change things.  You just have to be cautious and more careful.   It's definitely more painful.   Changes that will require *any* work by a user of NumPy other than a re-compile of their code should only be on major version numbers, preferably have a backward-compatible header to use, and a document that describes all the changes that must be made to move the code forward. 
 
I'm not trying to throw stones.  My glass house and my own sins would not justify such behavior.   I apologize if it has come off that way at any time.   

Best, 

-Travis