[Numpy-discussion] Removing datetime support for 1.4.x series ?

[Charles R Harris]

On Thu, Feb 4, 2010 at 12:37 AM, Travis Oliphant <oliphant at enthought.com>wrote:

>
> On Feb 4, 2010, at 12:59 AM, Charles R Harris wrote:
>
>
>
> On Wed, Feb 3, 2010 at 11:48 PM, Travis Oliphant <oliphant at enthought.com>wrote:
>
>>
>> On Feb 2, 2010, at 11:59 PM, David Cournapeau wrote:
>>
>> Travis Oliphant wrote:
>>
>>
>> On Feb 2, 2010, at 11:46 PM, David Cournapeau wrote:
>>
>>
>> On Wed, Feb 3, 2010 at 12:23 PM, David Cournapeau
>>
>> <david at silveregg.co.jp <mailto:david at silveregg.co.jp<david at silveregg.co.jp>>>
>> wrote:
>>
>>
>>
>> Sorry, my question was badly worded: besides the metadata pointer, is
>>
>> there any other change related to the metadata infratructure which may
>>
>> potentially change changes the publicly exported structures ? I wonder
>>
>> whereas the metadata infrastructure can be kept in 1.4.x independently
>>
>> of the datetime support without breaking the ABI
>>
>>
>> FWIW, keeping the metadata pointer, and only removing datetime-related
>>
>> things makes numpy 1.4.x backward compatible, at least as far as scipy
>>
>> is concerned. So it seems the PyArray_Funcs change is the only
>>
>> ABI-incompatible change.
>>
>>
>> What do you mean by the "PyArray_Funcs change"?
>>
>>
>> The change that broke the ABI is in the PyArray_Funcs structure
>> (ndarrayobject.h):
>>
>> struct {
>>         PyArray_VectorUnaryFunc *cast[NPY_NTYPES];
>>         ....
>>
>> Because NPY_NTYPES is bigger after the datetime change.
>>
>> If there is a way to have the datetime not expanding NPY_NTYPES, then I
>> think we can keep the ABI. I tried something with datetimes considered
>> as user types, but did not go very far (most certainly because I have
>> never used this part of the code before).
>>
>>
>> Thanks for reminding me what the ABI problem is.  Yes, that will break it
>> (I was very suspicious that we could change the number of basic types
>> without ABI consequence but didn't have time to think about the real
>> problem).
>>
>> My intention in adding the datetime data-type was not to try and preserve
>> ABI in the process.
>>
>>
> If so, then it would have been better to have been upfront about that when
> it went in. I know I pushed for inclusion, but I was told that the ABI could
> be preserved. We've all been surprised by unforeseen bugs, accidents happen.
> The question is what is the most graceful way out. I think we should follow
> David's lead here as he is the current release guy.
>
>
> Yes, it would have been better.  But, I wasn't trying to hide anything.
>  There were suggestions that the ABI could be preserved, and I didn't see
> the argument to resisting those claims very clearly, and so couldn't refute
> them quickly.
>
> Why the versioning matters is that we have a release with the needed ABI
> changes to support date-time.   The date-time data-type is useful in its
> current state (it's not complete but what is there is useable for storing
> date-time information).  I think giving it time for people to use it will
> help continue to improve what is there and encourage someone to finish the
> rest of the implementation (it's just not that much more work for someone
> with about 40-80 hours to spare).
>
> Perhaps one way to articulate my perspective is the following:
>
> There are currently 2 groups of NumPy users:
>
>  1)  those who have re-compiled all of their code for 1.4.0
>  2)  those who haven't
>
>
I think David has a better grip on that. There really are a lot of people
who depend on binaries, and those binaries in turn depend on numpy. I would
even say those folks are a majority, they are those who download the Mac and
Windows versions of numpy.


> Group 1) will have to re-compile again no matter what we do (because we are
> either going to have to bump the ABI number or back-pedal).
> Group 2) will not have to re-compile once the new release comes out.
>
> I don't want to make Group 1) have to re-compile yet a third time when
> date-time support finally comes out.  If they have bitten the bullet now,
> they will be rewarded with a stable ABI (that will eventually have the
> benefit of better ufunc support for record arrays as well as the date-time
> features).
>
>
I feel that a latter release date for datetime would be a benefit to
yourself also, as you would have the time to get the code into shape. As I
recall you were even reluctant to commit it in the first place. What has
changed?

Chuck

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100204/62a82ab2/attachment.html>