[Numpy-discussion] Created NumPy 1.7.x branch

[Travis Oliphant]

> 
> The usual practice is to announce a schedule first.

I just did announce the schedule.   

>  
> There is time before the first Release candidate to make changes on the 1.7.x branch.   If you want to make the changes on master, and just indicate the Pull requests, Ondrej can make sure they are added to the 1.7.x. branch by Monday.    We can also delay the first Release Candidate by a few days to next Wednesday and then bump everything 3 days if that will help.     There will be a follow-on 1.8 release before the end of the year --- so there is time to make changes for that release as well.    The next release will not take a year to get out, so we shouldn't feel pressured to get *everything* in this release.
> 
> What are we going to do for 1.8?

Let's get 1.7 out the door first. 

> 
> Yes, the functions will give warnings otherwise.

I think this needs to be revisited.  I don't think these changes are necessary for *every* use of macros.   It can cause a lot of effort for people downstream without concrete benefit. 

>  
>   That's not as nice to type.
> 
> So? The point is to have correctness, not ease of typing.

I'm not sure if a pun was intended there or not.    C is not a safe and fully-typed system.    That is one of its weaknesses according to many.  But, I would submit that not being forced to give everything a "type" (and recognizing the tradeoffs that implies) is also one reason it gets used.


>  
>  Is that assuming that PyArray_NDIM will become a function and need a specific object type for its argument (and everything else cast....).   That's one clear disadvantage of inline functions versus macros in my mind:  no automatic polymorphism.
> 
> That's a disadvantage of Python. The virtue of inline functions is precisely type checking.

Right, but we need to be more conscientious about this.   Not every use of Macros should be replaced by inline function calls and the requisite *forced* type-checking.   type-chekcing is not *universally* a virtue --- if it were, nobody would use Python. 

>  
> I don't think type safety is a big win for macros like these.     We need to be more judicious about which macros are scheduled for function inlining.  Some just don't benefit from the type-safety implications as much as others do, and you end up requiring everyone to change their code downstream for no real reason.  
> 
> These sorts of changes really feel to me like unnecessary spelling changes that require work from extension writers who now have to modify their code with no real gain.   There seems to be a lot of that going on in the code base and I'm not really convinced that it's useful for end-users.
> 
> Good style and type checking are useful. Numpy needs more of both.

You can assert it, but it doesn't make it so.   "Good style" depends on what you are trying to accomplish and on your point of view.  NumPy's style is not the product of one person, it's been adapted from multiple styles and inherits quite a bit from Python's style.   I don't make any claims for it other than it allowed me to write it with the time and experience I had 7 years ago.    We obviously disagree about this point.  I'm sorry about that.  I'm pretty flexible usually --- that's probably one of your big criticisms of my "style".  

But, one of the things I feel quite strongly about is how hard we make it for NumPy users to upgrade.    There are two specific things I disagree with pretty strongly: 

	1) Changing defined macros that should work the same on PyArrayObjects or PyObjects to now *require* types --- if we want to introduce new macros that require types than we can --- as long as it just provides warnings but still compiles then I suppose I could find this acceptable.

	2) Changing MACROS to require semicolons when they were previously not needed.    I'm going to be very hard-nosed about this one. 

>   
> I'm going to be a lot more resistant to that sort of change in the code base when I see it.
> 
> Numpy is a team effort. There are people out there who write better code than you do, you should learn from them.

Exactly!  It's a team effort.   I'm part of that team as well, and while I don't always have strong opinions about things.  When I do, I'm going to voice it.    

I've learned long ago there are people that write better code than me.    There are people that write better code than you.   That is not the question here at all.     The question here is not requiring a *re-write* of code in order to get their extensions to compile using NumPy headers.    We should not be making people change their code to get their extensions to compile in NumPy 1.X

>  
> 
> One particularly glaring example to my lens on the world:   I think it would have been better to define new macros which require semicolons than changing the macros that don't require semicolons to now require semicolons:  
> 
>     NPY_BEGIN_THREADS_DEF
>     NPY_BEGIN_THREADS
>     NPY_ALLOW_C_API
>     NPY_ALLOW_C_API_DEF
>     NPY_DISABLE_C_API
> 
> That feels like a gratuitous style change that will force users of those macros to re-write their code.
> 
> It doesn't seem to be much of a problem.

Unfortunately, I don't trust your judgment on that.   My experience and understanding tells a much different story.   I'm sorry if you disagree with me. 

>  
> Sure, it's a simple change, but it's a simple change that doesn't do anything for you as an end user.   I think I'm going to back this change out, in fact.   I can't see requiring people to change their C-code like this will require without a clear benefit to them.    I'm quite sure there is code out there that uses these documented APIs (without the semicolon).   If we want to define new macros that require colons, then we do that, but we can't get rid of the old ones --- especially in a 1.x release. 
> 
> Our policy should not be to allow gratuitous style changes just because we think something is prettier another way.   The NumPy code base has come from multiple sources and reflects several styles.   It also follows an older style of C-programming (that is quite common in the Python code base).    It can be changed, but those changes shouldn't be painful for a library user without some specific gain for them that the change allows. 
> 
> 
> You use that word 'gratuitous' a lot, I don't think it means what you think it means. For instance, the new polynomial coefficient order wasn't gratuitous, it was doing things in a way many found more intuitive and generalized better to different polynomial basis. People
> have different ideas, that doesn't make them gratuitous.

That's a slightly different issue.    At least you created a new object and api which is a *little* better.     My complaint about the choice there is now there *must* be two interfaces and added confusion as people will have to figure out which assumption is being used.   I don't really care about the coefficient order --- really I don't.   Either one is fine in my mind.   I recognize the reasons.    The problem is *changing* it without a *really* good reason.   Now, we have to have two different APIs.   I would much preferred to have poly1d disappear and just use your much nicer polynomial classes.    Now, it can't and we are faced with a user-story that is either difficult for someone transitioning from MATLAB or a "why did you do that?" puzzled look from a new user as to why we support both coefficient orders.  Of course, that could be our story --- hey we support all kinds of orders, it doesn't really matter, you just have to tell us what you mean when passing in an unadorned array of coefficients.   But, this is a different issue. 

I'm using the word 'gratuitous' to mean that it is "uncalled for and lacks a good reason".     There needs to be much better reasons given for code changes that require someone to re-write working code than "it's better style" or even "it will help new programmers avoid errors".   Let's write another interface that new programmers can use that fits the world the way you see it, don't change what's already working just because you don't like it or wish a different choice had been made.  

>  
> There are significant users of NumPy out there still on 1.4.    Even the policy of deprecation that has been discussed will not help people trying to upgrade from 1.4 to 1.8.   They will be forced to upgrade multiple times.    The easier we can make this process for users the better.    I remain convinced that it's better and am much more comfortable with making a release that requires a re-compile (that will succeed without further code changes --- because of backward compatibility efforts) than to have supposed ABI compatibility with subtle semantic changes and required C-code changes when you do happen to re-compile.  
> 
> 
> Cleanups need to be made bit by bit. I don't think we have done anything that will cause undo trouble.

I disagree substantially on the impact of these changes.  You can disagree about my awareness of NumPy users, but I think I understand a large number of them and why NumPy has been successful in getting users.    I agree that we have been unsuccessful at getting serious developers and I'm convinced by you and Mark as to why that is.    But, we can't sacrifice users for the sake of getting developers who will spend their free time trying to get around the organic pile that NumPy is at this point.  

Because of this viewpoint, I think there is some adaptation and cleanup right now, needed, so that significant users of NumPy can upgrade based on the changes that have occurred without causing them annoying errors (even simple changes can be a pain in the neck to fix).  

I do agree changes can be made.    I realize you've worked hard to keep the code-base in a state that you find more adequate.   I think you go overboard on that front, but I acknowledge that there are people that appreciate this.    I do feel very strongly that we should not require users to have to re-write working C-code in order to use a new minor version number in NumPy, regardless of how the code "looks" or how much "better" it is according to some idealized standard.  

The macro changes are border-line (at least I believe code will still compile --- just raise warnings, but I need to be sure about this).    The changes that require semi-colons are not acceptable at all.   

Look Charles, I believe we can continue to work productively together and our differences can be a strength to the community.  I hope you feel the same way.  I will continue to respect and listen to your perspective --- especially when I disagree with it. 

-Travis





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120622/9588829e/attachment.html>