[Numpy-discussion] ENH: Proposal to add atleast_nd function

[Eric Wieser]

> There might be some linear algebraic reason why those axis positions make
sense, but I’m not aware of it...

My guess is that the historical motivation was to allow grayscale `(H, W)`
images to be converted into `(H, W, 1)` images so that they can be
broadcast against `(H, W, 3)` RGB images.

Eric

On Fri, 12 Feb 2021 at 02:32, Juan Nunez-Iglesias <jni at fastmail.com> wrote:

> both napari and scikit-image use atleast_ a few times. I don’t have many
> examples of where I used nd because it didn’t exist. But I have the very
> distinct impression of needing it repeatedly. In some places, I’ve used
> `np.broadcast_to` to signal the same intention, where `atleast_nd` would
> have been the more readable solution.
>
> I don’t buy the argument that it’s just a way to mask errors. NumPy
> broadcasting also has that same potential but I hope no one would seriously
> consider deprecating it. Indeed, even if we accept that we (library
> authors) should force users to provide an array of the right
> dimensionality, that still argues for making it convenient for users to do
> that!
>
> I don’t feel super strongly about this. But I think atleast_nd is a move
> in a positive direction and I’d prefer  it to what’s there now:
>
> In [1]: import numpy as np
> In [2]: np.atleast_3d(np.ones(4)).shape
> Out[2]: (1, 4, 1)
>
> There might be some linear algebraic reason why those axis positions make
> sense, but I’m not aware of it...
>
> Juan.
>
> On 12 Feb 2021, at 5:32 am, Eric Wieser <wieser.eric+numpy at gmail.com>
> wrote:
>
> I did a quick search of matplotlib, and found a few uses of all three
> functions:
>
> *
> https://github.com/matplotlib/matplotlib/blob/fed55c63a314351cd39a12783f385009782c06e1/lib/matplotlib/_layoutgrid.py#L441-L446
>   This one isn't really numpy at all, and is really just a shorthand for
> normalizing an argument `x=n` to `x=[n, n]`
> *
> https://github.com/matplotlib/matplotlib/blob/dd249744270f6abe3f540f81b7a77c0cb728ddbb/lib/matplotlib/mlab.py#L888
>    This one is the classic "either multivariate or single-variable data"
> thing endemic to the SciPy ecosystem.
> *
> https://github.com/matplotlib/matplotlib/blob/1eef019109b64ee4085732544cb5e310e69451ab/lib/matplotlib/cbook/__init__.py#L1325-L1326
>   Matplotlib has their own `_check_1d` function for input sanitization,
> although github says it's only used to parse the arguments to `plot`, which
> at this point are fairly established as being flexible.
> *
> https://github.com/matplotlib/matplotlib/blob/f72adc49092fe0233a8cd21aa0f317918dafb18d/lib/matplotlib/transforms.py#L631
>   This just looks like "defensive programming", and if the argument isn't
> already 3d then something is probably wrong.
>
> This isn't an exhaustive list, just a handful of different situations the
> functions were used.
>
> Eric
>
>
>
> On Thu, 11 Feb 2021 at 18:15, Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Thu, Feb 11, 2021 at 9:42 AM Benjamin Root <ben.v.root at gmail.com>
>> wrote:
>>
>>> for me, I find that the at_least{1,2,3}d functions are useful for
>>> sanitizing inputs. Having an at_leastnd() function can be viewed as a step
>>> towards cleaning up the API, not cluttering it (although, deprecations of
>>> the existing functions probably should be long given how long they have
>>> existed).
>>>
>>
>> I would love to see examples of this -- perhaps in matplotlib?
>>
>> My thinking is that in most cases it's probably a better idea to keep the
>> interface simpler, and raise an error for lower-dimensional arrays.
>> Automatic conversion is convenient (and endemic within the SciPy
>> ecosystem), but is also a common source of bugs.
>>
>> On Thu, Feb 11, 2021 at 1:56 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>>>
>>>> On Wed, Feb 10, 2021 at 9:48 PM Juan Nunez-Iglesias <jni at fastmail.com>
>>>> wrote:
>>>>
>>>>> I totally agree with the namespace clutter concern, but honestly, I
>>>>> would use `atleast_nd` with its `pos` argument (I might rename it to
>>>>> `position`, `axis`, or `axis_position`) any day over `at_least{1,2,3}d`,
>>>>> for which I had no idea where the new axes would end up.
>>>>>
>>>>> So, I’m in favour of including it, and optionally deprecating
>>>>> `atleast_{1,2,3}d`.
>>>>>
>>>>>
>>>> I appreciate that `atleast_nd` feels more sensible than
>>>> `at_least{1,2,3}d`, but I don't think "better" than a pattern we would not
>>>> recommend is a good enough reason for inclusion in NumPy. It needs to stand
>>>> on its own.
>>>>
>>>> What would be the recommended use-cases for this new function?
>>>> Have any libraries building on top of NumPy implemented a version of
>>>> this?
>>>>
>>>>
>>>>> Juan.
>>>>>
>>>>> On 11 Feb 2021, at 9:48 am, Sebastian Berg <sebastian at sipsolutions.net>
>>>>> wrote:
>>>>>
>>>>> On Wed, 2021-02-10 at 17:31 -0500, Joseph Fox-Rabinovitz wrote:
>>>>>
>>>>> I've created PR#18386 to add a function called atleast_nd to numpy and
>>>>> numpy.ma. This would generalize the existing atleast_1d, atleast_2d,
>>>>> and
>>>>> atleast_3d functions.
>>>>>
>>>>> I proposed a similar idea about four and a half years ago:
>>>>>
>>>>> https://mail.python.org/pipermail/numpy-discussion/2016-July/075722.html
>>>>> ,
>>>>> PR#7804. The reception was ambivalent, but a couple of folks have
>>>>> asked me
>>>>> about this, so I'm bringing it back.
>>>>>
>>>>> Some pros:
>>>>>
>>>>> - This closes issue #12336
>>>>> - There are a couple of Stack Overflow questions that would benefit
>>>>> - Been asked about this a couple of times
>>>>> - Implementation of three existing atleast_*d functions gets easier
>>>>> - Looks nicer that the equivalent broadcasting and reshaping
>>>>>
>>>>> Some cons:
>>>>>
>>>>> - Cluttering up the API
>>>>> - Maintenance burden (but not a big one)
>>>>> - This is just a utility function, which can be achieved through
>>>>> broadcasting and reshaping
>>>>>
>>>>>
>>>>> My main concern would be the namespace cluttering. I can't say I use
>>>>> even the `atleast_2d` etc. functions personally, so I would tend to be
>>>>> slightly against the addition. But if others land on the "useful" side here
>>>>> (and it seemed a bit at least on github), I am also not opposed.  It is a
>>>>> clean name that lines up with existing ones, so it doesn't seem like a big
>>>>> "mental load" with respect to namespace cluttering.
>>>>>
>>>>> Bike shedding the API is probably a good idea in any case.
>>>>>
>>>>> I have pasted the current PR documentation (as html) below for quick
>>>>> reference. I wonder a bit about the reasoning for having `pos` specify a
>>>>> value rather than just a side?
>>>>>
>>>>>
>>>>>
>>>>> numpy.atleast_nd(*ary*, *ndim*, *pos=0*)
>>>>> View input as array with at least ndim dimensions.
>>>>> New unit dimensions are inserted at the index given by *pos* if
>>>>> necessary.
>>>>> Parameters*ary  *array_like
>>>>> The input array. Non-array inputs are converted to arrays. Arrays that
>>>>> already have ndim or more dimensions are preserved.
>>>>> *ndim  *int
>>>>> The minimum number of dimensions required.
>>>>> *pos  *int, optional
>>>>> The index to insert the new dimensions. May range from -ary.ndim - 1
>>>>> to +ary.ndim (inclusive). Non-negative indices indicate locations
>>>>> before the corresponding axis: pos=0 means to insert at the very
>>>>> beginning. Negative indices indicate locations after the corresponding axis:
>>>>>  pos=-1 means to insert at the very end. 0 and -1 are always
>>>>> guaranteed to work. Any other number will depend on the dimensions of the
>>>>> existing array. Default is 0.
>>>>> Returns*res  *ndarray
>>>>> An array with res.ndim >= ndim. A view is returned for array inputs.
>>>>> Dimensions are prepended if *pos* is 0, so for example, a 1-D array
>>>>> of shape (N,) with ndim=4becomes a view of shape (1, 1, 1, N).
>>>>> Dimensions are appended if *pos* is -1, so for example a 2-D array of
>>>>> shape (M, N) becomes a view of shape (M, N, 1, 1)when ndim=4.
>>>>> *See also*
>>>>> atleast_1d
>>>>> <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_1d.html#numpy.atleast_1d>
>>>>> , atleast_2d
>>>>> <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_2d.html#numpy.atleast_2d>
>>>>> , atleast_3d
>>>>> <https://18298-908607-gh.circle-artifacts.com/0/doc/build/html/reference/generated/numpy.atleast_3d.html#numpy.atleast_3d>
>>>>> *Notes*
>>>>> This function does not follow the convention of the other atleast_*d functions
>>>>> in numpy in that it only accepts a single array argument. To process
>>>>> multiple arrays, use a comprehension or loop around the function call. See
>>>>> examples below.
>>>>> Setting pos=0 is equivalent to how the array would be interpreted by
>>>>> numpy’s broadcasting rules. There is no need to call this function for
>>>>> simple broadcasting. This is also roughly (but not exactly) equivalent to
>>>>>  np.array(ary, copy=False, subok=True, ndmin=ndim).
>>>>> It is easy to create functions for specific dimensions similar to the
>>>>> other atleast_*d functions using Python’s functools.partial
>>>>> <https://docs.python.org/dev/library/functools.html#functools.partial>
>>>>>  function. An example is shown below.
>>>>> *Examples*
>>>>>
>>>>> >>> np.atleast_nd(3.0, 4)array([[[[ 3.]]]])
>>>>>
>>>>> >>> x = np.arange(3.0)>>> np.atleast_nd(x, 2).shape(1, 3)
>>>>>
>>>>> >>> x = np.arange(12.0).reshape(4, 3)>>> np.atleast_nd(x, 5).shape(1, 1, 1, 4, 3)>>> np.atleast_nd(x, 5).base is x.baseTrue
>>>>>
>>>>> >>> [np.atleast_nd(x) for x in ((1, 2), [[1, 2]], [[[1, 2]]])]:[array([[1, 2]]), array([[1, 2]]), array([[[1, 2]]])]
>>>>>
>>>>> >>> np.atleast_nd((1, 2), 5, pos=0).shape(1, 1, 1, 1, 2)>>> np.atleast_nd((1, 2), 5, pos=-1).shape(2, 1, 1, 1, 1)
>>>>>
>>>>> >>> from functools import partial>>> atleast_4d = partial(np.atleast_nd, ndim=4)>>> atleast_4d([1, 2, 3])[[[[1, 2, 3]]]]
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at python.org
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210212/2348e076/attachment-0001.html>