[Numpy-discussion] Views of a different dtype

Jaime Fernández del Río jaime.frio at gmail.com
Wed Feb 4 10:13:15 EST 2015


On Tue, Feb 3, 2015 at 1:52 PM, Ian Henriksen <
insertinterestingnamehere at gmail.com> wrote:

> On Tue Feb 03 2015 at 1:47:34 PM Jaime Fernández del Río <
> jaime.frio at gmail.com> wrote:
>
>> On Tue, Feb 3, 2015 at 8:59 AM, Sebastian Berg <
>> sebastian at sipsolutions.net> wrote:
>>
>>> On Di, 2015-02-03 at 07:18 -0800, Jaime Fernández del Río wrote:
>>> >
>>> <snip>
>>> >
>>> >
>>> >
>>> > Do you have a concrete example of what a non (1, 1) array that fails
>>> > with relaxed strides would look like?
>>> >
>>> >
>>> > If we used, as right now, the array flags as a first choice point, and
>>> > only if none is set try to determine it from the strides/dimensions
>>> > information, I fail to imagine any situation where the end result
>>> > would be worse than now. I don't think that a little bit of
>>> > predictable surprising in an advanced functionality is too bad. We
>>> > could start raising "on the face of ambiguity, we refuse to guess"
>>> > errors, even for the current behavior you show above, but that is more
>>> > likely to trip people by not giving them any simple workaround, that
>>> > it seems to me would be "add a .T if all dimensions are 1" in some
>>> > particular situations. Or are you thinking of something more serious
>>> > than a shape mismatch when you write about "breaking current code"?
>>> >
>>>
>>> Yes, I am talking only about wrongly shaped results for some fortran
>>> order arrays. A (20, 1) fortran order complex array being viewed as
>>> float, will with relaxed strides become a (20, 2) array instead of a
>>> (40, 1) one.
>>>
>>
>> That is a limitation of the current implementation too, and happens
>> already whenever relaxed strides are in place. Which is the default for
>> 1.10, right?
>>
>> Perhaps giving 'view' an 'order' or 'axis' kwarg could make sense after
>> all? It should probably be more of a hint of what to do (fortran vs c) when
>> in doubt. "C" would prioritize last axis, "F" the first, and we could even
>> add a "raise" option to have it fail if the axis cannot be inferred from
>> the strides and shape. Current behavior would is equivalent to what "C"
>> would do.
>>
>> Jaime
>>
>
>
> IMHO, the best option would be something like this:
>
> - When changing to a type with smaller itemsize, add a new axis after the
> others so the resulting array is C contiguous (unless a different axis is
> specified by a keyword argument). The advantage here is that if you index
> the new view using the old indices for an entry, you get an array showing
> its representation in the new type.
> - No shape change for views with the same itemsize
> - When changing to a type with a larger itemsize, collapse along the last
> axis unless a different axis is specified, throwing an error if the axis
> specified does not match the axis specified.
>

My only concern with adding a new axis, backwards compatibility aside, is
that you would not know wether to keep or discard the resulting size 1
dimension when taking a view of the view. We could reuse the keepdims
terminology from ufuncs, though.

So the current behavior would remain unchanged if you set axis=None,
keepdims=True, and we could transition to a more reasonable default with
axis=-1, keepdims=False over a few releases with adequate warnings.

In my mind, when expanding, the axis would not indicate where to place the
new axis, but which axis to collapse over, that is the hard part to figure
out!If you want something else, it is only a call to rollaxis away. Perhaps
we could also give keepdims a meaning when expanding, as to whether to add
a new axis at the end, or change the size of the chosen dimension.

I don't know, there may be an actual interface hidden somewhere here, but
still needs some  cooking to fully figure it out.

Jaime


>
> The last point essentially is just adding an axis argument. I like that
> idea because it gives users the ability to do all that the array's memory
> layout allows in a clear and concise way. Throwing an error if the default
> axis doesn't work would be a good way to prevent strange bugs from
> happening when the default behavior is expected.
>
> The first point would be a break in backwards compatibility, so I'm not
> sure if it's feasible at this point. The advantage would be that all all
> arrays returned when using this functionality would be contiguous along the
> last axis. The shape of the new array would be independent of the memory
> layout of the original one. This would also be a much cleaner way to ensure
> that views of a different type are always reversible while still allowing
> for relaxed strides.
>
> Either way, thanks for looking into this. It's a great feature to have
> available.
>
> -Ian Henriksen
>
>
>>
>>
>>
>>> >
>>> > If there are any real loopholes in expanding this functionality, then
>>> > lets not do it, but we know we have at least one user unsatisfied with
>>> > the current performance, so I really think it is worth trying. Plus,
>>> > I'll admit to that, messing around with some of these stuff deep
>>> > inside the guts of the beast is lots of fun! ;)
>>> >
>>>
>>> I do not think there are loopholes with expanding this functionality. I
>>> think there have regressions when we put relaxed strides to on, because
>>> suddenly the fortran order array might be expanded along a 1-sized axis,
>>> because it is also C order. So I wonder if we can fix these regressions
>>> and at the same time maybe provide a more intuitive approach then using
>>> the memory order blindly....
>>>
>>
>>> - Sebastian
>>>
>>> >
>>> > Jaime
>>> >
>>> >
>>> >         - Sebastian
>>> >
>>> >
>>> >         >
>>> >         > Jaime
>>> >         >
>>> >         >
>>> >         >
>>> >         >         - Sebastian
>>> >         >
>>> >         >         >
>>> >         >         >         I guess the main consideration for this is
>>> >         that we
>>> >         >         may be
>>> >         >         >         stuck with
>>> >         >         >         stuff b/c of backwards compatibility. Can
>>> >         you maybe
>>> >         >         say a
>>> >         >         >         little bit
>>> >         >         >         about what is allowed now, and what
>>> >         constraints that
>>> >         >         puts on
>>> >         >         >         things?
>>> >         >         >         E.g. are we already grovelling around in
>>> >         strides and
>>> >         >         picking
>>> >         >         >         random
>>> >         >         >         dimensions in some cases?
>>> >         >         >
>>> >         >         >
>>> >         >         > Just to restate it: right now we only allow new
>>> >         views if the
>>> >         >         array is
>>> >         >         > globally contiguous, so either along the first or
>>> >         last
>>> >         >         dimension.
>>> >         >         >
>>> >         >         >
>>> >         >         > Jaime
>>> >         >         >
>>> >         >         >
>>> >         >         >         -n
>>> >         >         >
>>> >         >         >         --
>>> >         >         >         Nathaniel J. Smith
>>> >         >         >         Postdoctoral researcher - Informatics -
>>> >         University
>>> >         >         of
>>> >         >         >         Edinburgh
>>> >         >         >         http://vorpus.org
>>> >         >         >
>>> >          _______________________________________________
>>> >         >         >         NumPy-Discussion mailing list
>>> >         >         >         NumPy-Discussion at scipy.org
>>> >         >         >
>>> >         >
>>> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> >         >         >
>>> >         >         >
>>> >         >         >
>>> >         >         >
>>> >         >         >
>>> >         >         > --
>>> >         >         > (\__/)
>>> >         >         > ( O.o)
>>> >         >         > ( > <) Este es Conejo. Copia a Conejo en tu firma
>>> >         y ayúdale
>>> >         >         en sus
>>> >         >         > planes de dominación mundial.
>>> >         >         > _______________________________________________
>>> >         >         > NumPy-Discussion mailing list
>>> >         >         > NumPy-Discussion at scipy.org
>>> >         >         >
>>> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> >         >
>>> >         >
>>> >         >
>>> >         >         _______________________________________________
>>> >         >         NumPy-Discussion mailing list
>>> >         >         NumPy-Discussion at scipy.org
>>> >         >
>>> >          http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> >         >
>>> >         >
>>> >         >
>>> >         >
>>> >         >
>>> >         > --
>>> >         > (\__/)
>>> >         > ( O.o)
>>> >         > ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale
>>> >         en sus
>>> >         > planes de dominación mundial.
>>> >         > _______________________________________________
>>> >         > NumPy-Discussion mailing list
>>> >         > NumPy-Discussion at scipy.org
>>> >         > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> >
>>> >
>>> >
>>> >         _______________________________________________
>>> >         NumPy-Discussion mailing list
>>> >         NumPy-Discussion at scipy.org
>>> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > (\__/)
>>> > ( O.o)
>>> > ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus
>>> > planes de dominación mundial.
>>> > _______________________________________________
>>> > NumPy-Discussion mailing list
>>> > NumPy-Discussion at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>
>>
>> --
>> (\__/)
>> ( O.o)
>> ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
>> de dominación mundial.
>>  _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150204/3d35c67c/attachment.html>


More information about the NumPy-Discussion mailing list