[Numpy-discussion] 1.9.x branch

Sat Apr 26 13:36:26 EDT 2014

On Sat, Apr 26, 2014 at 1:12 PM, Sebastian Berg
<sebastian at sipsolutions.net>wrote:

> On Mi, 2014-04-23 at 10:11 -0400, josef.pktd at gmail.com wrote:
> > On Wed, Apr 23, 2014 at 5:32 AM, Sebastian Berg
> > <sebastian at sipsolutions.net> wrote:
> > > On Di, 2014-04-22 at 15:35 -0600, Charles R Harris wrote:
> > >> Hi All,
> > >>
> > >>
> > >> I'd like to branch 1.9.x at the end of the month. There are a couple
> > >> of reasons for the timing. First, we have a lot of new stuff in the
> > >> development branch. Second, there is work ongoing in masked arrays
> > >> that I'd like to keep out of the release so that it has more time to
> > >> settle. Third, it's past time ;)
> > >
> > > Sounds good.
> > >
> > >> There are currently a number of 1.9.0 blockers, which can be seen
> > >> here.
> > >>
> > >> Datetime timezone handling broken in 1.7.x
> > >>
> > >> I don't think there is time to get this done for 1.9.0 and it needs to
> > >> be pushed off to 1.10.0.
> > >>
> > >> Return multiple field selection as ro view
> > >>
> > >> I have a branch for this, but because the returned type differs from a
> > >> copy by alignment spacing there was a test failure. Merging that
> > >> branch might cause some incompatibilities.
> > >>
> > >
> > > I am a bit worried here that comparisons might make trouble.
> > >
> > >> Object array creation new conversion to int
> > >>
> > >>
> > >> This one needs a decision. Julian, Sebastian, thoughts?
> > >>
> > >
> > > Maybe for all to consider this is about what happens for object arrays
> > > if you do things like:
> > >
> > > # Array cast to object array (np.array(arr) would be identical):
> > > a = np.arange(10).astype(object)
> > > # Array passed into new array creation (not just *one* array):
> > > b = np.array([np.arange(10)], dtype=object)
> > > # Numerical array is assigned to object array:
> > > c = np.empty(10, dtype=object)
> > > c[...] = np.arange(10)
> > >
> > > Before this change, the result was:
> > > type(a[0]) is int
> > > type(b[0,0]) is np.int_  # Note the numpy type
> > > type(c[0]) is int
> > >
> > > After this change, they are all `int`. Though note that the numpy type
> > > is preserved for example for long double. On the one hand preserving
> the
> > > numpy type might be nice, but on the other hand we don't care much
> about
> > > the dtypes of scalars and in practice the python types are probably
> more
> > > often wanted.
> >
> > what if I don't like python?
> >
>
> Fair point :). I also think it is more consistent if we use the numpy
> types (and the user has to cast to the python type explicitly). Could
> argue that if someone casts to object, they might like python objects,
> but if you don't want them that is tricky, too.
>
> There is the option that the numerical array -> object array cast always
> returns an array of numpy scalars. Making it consistent (opposite to
> what is currently in numpy master).
>
> This would mean that `numerical_arr.astype(object)` would give an array
> of numpy scalars always. Getting python scalars would only be possible
> using `arr.item()` (or `tolist`). I guess that change is less likely to
> cause problems, because the numpy types can do more things normally
> though they are slower.
>
> So a (still a bit unsure) +1 from me for making numeric -> object casts
> return arrays of numpy scalars unless we have reason to expect that to
> cause problems. Not sure how easy that would be to change, it wasn't a
> one line change when I tried, so there is something slightly tricky to
> clear out, but probably not too hard either.
>

More general background question.

Why is there casting involved in object arrays?

I thought object arrays will just take and return whatever you put in.
If I use python ints, I might want python ints.
If I use numpy int_s, I might want numpy ints.

b1 = np.array([np.arange(10)], dtype=object)
versus
b2 = np.array([list(range(10))], dtype=object)

>>> b1 = np.array([np.arange(10)], dtype=object)
>>> type(b1[0,2])
<type 'numpy.int32'>
>>> type(b1[0][2])
<type 'numpy.int32'>
>>>

>>> b2 = np.array([list(range(10))], dtype=object)
>>> type(b2[0,2])
<type 'int'>
>>> type(b2[0][2])
<type 'int'>

another version

>>> type(np.array(np.arange(10).tolist(), dtype=object)[2])
<type 'int'>
>>> type(np.array(list(np.arange(10)), dtype=object)[2])
<type 'numpy.int32'>

Josef

>
> - Sebastian
>
> > >>> np.int_(0)**(-1)
> > inf
> > >>> 0**-1
> > Traceback (most recent call last):
> >   File "<pyshell#28>", line 1, in <module>
> >     0**-1
> > ZeroDivisionError: 0.0 cannot be raised to a negative power
> >
> >
> > >>> type(np.arange(5)[0])
> > <type 'numpy.int32'>
> > >>> np.arange(5)[0]**-1
> > inf
> >
> > >>> type(np.arange(5)[0].item())
> > <type 'int'>
> > >>> np.arange(5)[0].item()**-1
> > Traceback (most recent call last):
> >   File "<pyshell#40>", line 1, in <module>
> >     np.arange(5)[0].item()**-1
> > ZeroDivisionError: 0.0 cannot be raised to a negative power
> >
> > >>> np.__version__
> > '1.6.1'
> >
> >
> > I remember struggling through this (avoiding python operations) quite
> > a bit in my early bugfixes to scipy.stats.distributions.
> >
> > (IIRC I ended up avoiding most ints.)
> >
> > Josef
> >
> > >
> > > Since I just realized that things are safe (float128 does not cast to
> > > float after all), I changed my mind and am tempted to keep the new
> > > behaviour. That is, if it does not create any problems (there was some
> > > issue in scipy, not sure how bad).
> > >
> > > - Sebastian
> > >
> > >> Median of np.matrix is broken(
> > >>
> > >>
> > >> Not sure what the status of this one is.
> > >>
> > >> 1.8 deprecations: Follow-up ticket
> > >>
> > >>
> > >> Things that might should be removed.
> > >>
> > >> ERROR: test_big_arrays (test_io.TestSavezLoad) on OS X + Python 3.3
> > >>
> > >>
> > >> I believe this one was fixed. For general problems reading/writing big
> > >> files on OS X, I believe they were fixed in Maverick and I'm inclined
> > >> to recommend an OS upgrade rather than work to chunk all the io.
> > >>
> > >> Deprecate NPY_CHAR
> > >> This one is waiting on a fix from Pearu to make f2py use numpy
> > >> strings. I think we will need to do this ourselves if we want to carry
> > >> through the deprecation. In any case it probably needs to be pushed
> > >> off to 1.10.
> > >>
> > >> 1.7 deprecations: Follow-up ticket
> > >> Some of these changes have been made, ro diagonal view for instance,
> > >> some still remain.
> > >>
> > >>
> > >>
> > >> Additions, updates, and thoughts welcome.
> > >>
> > >>
> > >> Chuck
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> NumPy-Discussion mailing list
> > >> NumPy-Discussion at scipy.org
> > >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> > >
> > >
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140426/a198d3e6/attachment.html>