[Numpy-discussion] Created NumPy 1.7.x branch

[Dag Sverre Seljebotn]

On 06/23/2012 09:32 AM, Dag Sverre Seljebotn wrote:
> On 06/23/2012 05:14 AM, Charles R Harris wrote:
>>
>>
>> On Fri, Jun 22, 2012 at 2:42 PM, Travis Oliphant<travis at continuum.io
>> <mailto:travis at continuum.io>>  wrote:
>>
>>>
>>>      The usual practice is to announce a schedule first.
>>
>>      I just did announce the schedule.
>>
>>
>> What has been done in the past is that an intent to fork is announced
>> some two weeks in advance so that people can weigh in on what needs to
>> be done before the fork. The immediate fork was a bit hasty. Likewise,
>> when I suggested going to the github issue tracking, I opened a
>> discussion on needed tags, but voila, there it was with an incomplete
>> set and no discussion. That to seemed hasty.
>>
>>>
>>>          There is time before the first Release candidate to make
>>>          changes on the 1.7.x branch.   If you want to make the changes
>>>          on master, and just indicate the Pull requests, Ondrej can
>>>          make sure they are added to the 1.7.x. branch by Monday.    We
>>>          can also delay the first Release Candidate by a few days to
>>>          next Wednesday and then bump everything 3 days if that will
>>>          help.     There will be a follow-on 1.8 release before the end
>>>          of the year --- so there is time to make changes for that
>>>          release as well.    The next release will not take a year to
>>>          get out, so we shouldn't feel pressured to get *everything* in
>>>          this release.
>>>
>>>
>>>      What are we going to do for 1.8?
>>
>>      Let's get 1.7 out the door first.
>>
>>
>> Mark proposed a schedule for the next several releases, I'd like to know
>> if we are going to follow it.
>>
>>
>>>
>>>      Yes, the functions will give warnings otherwise.
>>
>>      I think this needs to be revisited.  I don't think these changes are
>>      necessary for *every* use of macros.   It can cause a lot of effort
>>      for people downstream without concrete benefit.
>>
>>
>> The idea is to slowly move towards hiding the innards of the array type.
>> This has been under discussion since 1.3 came out. It is certainly the
>> case that not all macros need to go away.
>>
>>
>>>
>>>            That's not as nice to type.
>>>
>>>
>>>      So? The point is to have correctness, not ease of typing.
>>
>>      I'm not sure if a pun was intended there or not.    C is not a safe
>>      and fully-typed system.    That is one of its weaknesses according
>>      to many.  But, I would submit that not being forced to give
>>      everything a "type" (and recognizing the tradeoffs that implies) is
>>      also one reason it gets used.
>>
>>
>> C was famous for bugs due to the lack of function prototypes. This was
>> fixed with C99 and the stricter typing was a great help.
>>
>>
>>
>>>
>>>           Is that assuming that PyArray_NDIM will become a function and
>>>          need a specific object type for its argument (and everything
>>>          else cast....).   That's one clear disadvantage of inline
>>>          functions versus macros in my mind:  no automatic polymorphism.
>>>
>>>
>>>      That's a disadvantage of Python. The virtue of inline functions is
>>>      precisely type checking.
>>
>>      Right, but we need to be more conscientious about this.   Not every
>>      use of Macros should be replaced by inline function calls and the
>>      requisite *forced* type-checking.   type-chekcing is not
>>      *universally* a virtue --- if it were, nobody would use Python.
>>
>>>
>>>          I don't think type safety is a big win for macros like these.
>>>              We need to be more judicious about which macros are
>>>          scheduled for function inlining.  Some just don't benefit from
>>>          the type-safety implications as much as others do, and you end
>>>          up requiring everyone to change their code downstream for no
>>>          real reason.
>>>
>>>          These sorts of changes really feel to me like unnecessary
>>>          spelling changes that require work from extension writers who
>>>          now have to modify their code with no real gain.   There seems
>>>          to be a lot of that going on in the code base and I'm not
>>>          really convinced that it's useful for end-users.
>>>
>>>
>>>      Good style and type checking are useful. Numpy needs more of both.
>>
>>      You can assert it, but it doesn't make it so. "Good style" depends
>>      on what you are trying to accomplish and on your point of view.
>>        NumPy's style is not the product of one person, it's been adapted
>>      from multiple styles and inherits quite a bit from Python's style.
>>      I don't make any claims for it other than it allowed me to write it
>>      with the time and experience I had 7 years ago.    We obviously
>>      disagree about this point.  I'm sorry about that.  I'm pretty
>>      flexible usually --- that's probably one of your big criticisms of
>>      my "style".
>>
>>
>> Curiously, my criticism would be more that you are inflexible, slow to
>> change old habits.
>>
>>
>>      But, one of the things I feel quite strongly about is how hard we
>>      make it for NumPy users to upgrade.    There are two specific things
>>      I disagree with pretty strongly:
>>
>>      1) Changing defined macros that should work the same on
>>      PyArrayObjects or PyObjects to now *require* types --- if we want to
>>      introduce new macros that require types than we can --- as long as
>>      it just provides warnings but still compiles then I suppose I could
>>      find this acceptable.
>>
>>      2) Changing MACROS to require semicolons when they were previously
>>      not needed.    I'm going to be very hard-nosed about this one.
>>
>>>
>>>          I'm going to be a lot more resistant to that sort of change in
>>>          the code base when I see it.
>>>
>>>
>>>      Numpy is a team effort. There are people out there who write
>>>      better code than you do, you should learn from them.
>>
>>      Exactly!  It's a team effort.   I'm part of that team as well, and
>>      while I don't always have strong opinions about things.  When I do,
>>      I'm going to voice it.
>>
>>      I've learned long ago there are people that write better code than
>>      me.    There are people that write better code than you.
>>
>>
>> Of course. Writing code is not my profession, and even if it were, there
>> are people out there who would be immeasurable better. I have tried to
>> improve my style over the years by reading books and browsing code by
>> people who are better than me. I also recognize common bad habits naive
>> coders tend to pick up when they start out, not least because I have at
>> one time or another had many of the same bad habits.
>>
>>      That is not the question here at all.     The question here is not
>>      requiring a *re-write* of code in order to get their extensions to
>>      compile using NumPy headers.    We should not be making people
>>      change their code to get their extensions to compile in NumPy 1.X
>>
>>
>> I think a bit of rewrite here and there along the way is more palatable
>> than a big change coming in as one big lump, especially if the changes
>> are done with a long term goal in mind. We are working towards a Numpy
>> 2, but we can't just go off for a year or two and write it, we have to
>> get there step by step. And that requires a plan.
>
> To me you sound like you expect that people just need to change, say,
>
> PyArray_SHAPE(obj)
>
> to
>
> PyArray_SHAPE((PyArrayObject*)obj)
>
> But that's not the reality. The reality is that most users of the NumPy
> C API are required to do:
>
> #if WHATEVERNUMPYVERSIONDEFINE>  0x...
> PyArray_SHAPE(obj)
> #else
> PyArray_SHAPE((PyArrayObject*)obj)
> #endif
>
> or, perhaps, PyArray_SHAPE(CAST_IF_NEW_NUMPY obj).

Whoops. Terribly sorry, bad example -- I guess fixes to the users code 
would make it work with any NumPy version.

And I guess an extra semicolon never hurts for the macros either?

So by now I wish I could retract that post. Realized it five seconds too 
late :-)

Dag

>
> Or perhaps write a shim wrapper to insulate themselves from the NumPy API.
>
> At least if you want to cleanly compile against all the last ~3 versions
> of NumPy cleanly without warnings -- which any good developer wishes
> (unless there are *features* in newer versions that make a hard
> dependency on the newest version logical). Thus, cleaning up the NumPy
> API makes users' code much more ugly and difficult to read.
>
> "Gradual changes along the way" means there will be lots of different
> #if tests like that, which is at least harder to remember and work with
> than a single #if test for 1.x vs 2.x.
>
> Dag
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion