<br><br>On Wednesday, July 6, 2016, <<a href="mailto:numpy-discussion-request@scipy.org">numpy-discussion-request@scipy.org</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send NumPy-Discussion mailing list submissions to<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion@scipy.org')">numpy-discussion@scipy.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion-request@scipy.org')">numpy-discussion-request@scipy.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion-owner@scipy.org')">numpy-discussion-owner@scipy.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of NumPy-Discussion digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Added atleast_nd, request for clarification/cleanup of<br>
atleast_3d (Joseph Fox-Rabinovitz)<br>
2. Re: Added atleast_nd, request for clarification/cleanup of<br>
atleast_3d (Benjamin Root)<br>
3. Re: Added atleast_nd, request for clarification/cleanup of<br>
atleast_3d (Marten van Kerkwijk)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Wed, 6 Jul 2016 09:12:32 -0400<br>
From: Joseph Fox-Rabinovitz <<a href="javascript:;" onclick="_e(event, 'cvml', 'jfoxrabinovitz@gmail.com')">jfoxrabinovitz@gmail.com</a>><br>
To: Discussion of Numerical Python <<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion@scipy.org')">numpy-discussion@scipy.org</a>><br>
Subject: Re: [Numpy-discussion] Added atleast_nd, request for<br>
clarification/cleanup of atleast_3d<br>
Message-ID:<br>
<CAAa1KPa0RXYH3cCSyRGqpS_hzx9=K=<a href="javascript:;" onclick="_e(event, 'cvml', 'sp4dQJRbXoGzkEgBdGPw@mail.gmail.com')">sp4dQJRbXoGzkEgBdGPw@mail.gmail.com</a>><br>
Content-Type: text/plain; charset=UTF-8<br>
<br>
I can add a keyword-only argument that lets you put the new dims<br>
before or after the existing ones. I am not sure how to specify<br>
arbitrary patterns for the new dimensions, but that should take care<br>
of most use cases.<br>
<br>
The use case that motivated this function in the first place is that I<br>
am doing some processing on 4D arrays and I need to reduce them but<br>
return a result with the original dimensionality (but not shape).<br>
atleast_nd seemed like a better solution than atleast_4d.<br>
<br>
-Joe<br>
<br>
<br>
On Wed, Jul 6, 2016 at 3:41 AM, <<a href="javascript:;" onclick="_e(event, 'cvml', 'josef.pktd@gmail.com')">josef.pktd@gmail.com</a>> wrote:<br>
><br>
><br>
> On Wed, Jul 6, 2016 at 3:29 AM, <<a href="javascript:;" onclick="_e(event, 'cvml', 'josef.pktd@gmail.com')">josef.pktd@gmail.com</a>> wrote:<br>
>><br>
>><br>
>><br>
>> On Wed, Jul 6, 2016 at 2:21 AM, Ralf Gommers <<a href="javascript:;" onclick="_e(event, 'cvml', 'ralf.gommers@gmail.com')">ralf.gommers@gmail.com</a>><br>
>> wrote:<br>
>>><br>
>>><br>
>>><br>
>>> On Wed, Jul 6, 2016 at 7:06 AM, Nathaniel Smith <<a href="javascript:;" onclick="_e(event, 'cvml', 'njs@pobox.com')">njs@pobox.com</a>> wrote:<br>
>>><br>
>>>> On Jul 5, 2016 9:09 PM, "Joseph Fox-Rabinovitz"<br>
>>>> <<a href="javascript:;" onclick="_e(event, 'cvml', 'jfoxrabinovitz@gmail.com')">jfoxrabinovitz@gmail.com</a>> wrote:<br>
>>>> ><br>
>>>> > Hi,<br>
>>>> ><br>
>>>> > I have generalized np.atleast_1d, np.atleast_2d, np.atleast_3d with a<br>
>>>> > function np.atleast_nd in PR#7804<br>
>>>> > (<a href="https://github.com/numpy/numpy/pull/7804" target="_blank">https://github.com/numpy/numpy/pull/7804</a>).<br>
>>>> ><br>
>>>> > As a result of this PR, I have a couple of questions about<br>
>>>> > `np.atleast_3d`. `np.atleast_3d` appears to do something weird with<br>
>>>> > the dimensions: If the input is 1D, it prepends and appends a size-1<br>
>>>> > dimension. If the input is 2D, it appends a size-1 dimension. This is<br>
>>>> > inconsistent with `np.atleast_2d`, which always prepends (as does<br>
>>>> > `np.atleast_nd`).<br>
>>>> ><br>
>>>> > - Is there any reason for this behavior?<br>
>>>> > - Can it be cleaned up (e.g., by reimplementing `np.atleast_3d` in<br>
>>>> > terms of `np.atleast_nd`, which is actually much simpler)? This would<br>
>>>> > be a slight API change since the output would not be exactly the same.<br>
>>>><br>
>>>> Changing atleast_3d seems likely to break a bunch of stuff...<br>
>>>><br>
>>>> Beyond that, I find it hard to have an opinion about the best design for<br>
>>>> these functions, because I don't think I've ever encountered a situation<br>
>>>> where they were actually what I wanted. I'm not a big fan of coercing<br>
>>>> dimensions in the first place, for the usual "refuse to guess" reasons. And<br>
>>>> then generally if I do want to coerce an array to another dimension, then I<br>
>>>> have some opinion about where the new dimensions should go, and/or I have<br>
>>>> some opinion about the minimum acceptable starting dimension, and/or I have<br>
>>>> a maximum dimension in mind. (E.g. "coerce 1d inputs into a column matrix;<br>
>>>> 0d or 3d inputs are an error" -- atleast_2d is zero-for-three on that<br>
>>>> requirements list.)<br>
>>>><br>
>>>> I don't know how typical I am in this. But it does make me wonder if the<br>
>>>> atleast_* functions act as an attractive nuisance, where new users take<br>
>>>> their presence as an implicit recommendation that they are actually a useful<br>
>>>> thing to reach for, even though they... aren't that. And maybe we should be<br>
>>>> recommending folk move away from them rather than trying to extend them<br>
>>>> further?<br>
>>>><br>
>>>> Or maybe they're totally useful and I'm just missing it. What's your use<br>
>>>> case that motivates atleast_nd?<br>
>>><br>
>>> I think you're just missing it:) atleast_1d/2d are used quite a bit in<br>
>>> Scipy and Statsmodels (those are the only ones I checked), and in the large<br>
>>> majority of cases it's the best thing to use there. There's a bunch of<br>
>>> atleast_2d calls with a transpose appended because the input needs to be<br>
>>> treated as columns instead of rows, but that's still efficient and readable<br>
>>> enough.<br>
>><br>
>><br>
>><br>
>> As Ralph pointed out its usage in statsmodels. I do find them useful as<br>
>> replacement for several lines of ifs and reshapes<br>
>><br>
>> We stilll need in many cases the atleast_2d_cols, that appends the newaxis<br>
>> if necessary.<br>
>><br>
>> roughly the equivalent of<br>
>><br>
>> if x.ndim == 1:<br>
>> x = x[:, None]<br>
>> else:<br>
>> x = np.atleast_2d(x)<br>
>><br>
>> Josef<br>
>><br>
>>><br>
>>><br>
>>> For 3D/nD I can see that you'd need more control over where the<br>
>>> dimensions go, but 1D/2D are fine.<br>
><br>
><br>
><br>
> statsmodels has currently very little code with ndim >2, so I have no<br>
> overview of possible use cases, but it would be necessary to have full<br>
> control over the added axis since axis have a strict meaning and stats still<br>
> prefer Fortran order to default numpy/C ordering.<br>
><br>
> Josef<br>
><br>
><br>
>>><br>
>>><br>
>>><br>
>>> Ralf<br>
>>><br>
>>><br>
>>> _______________________________________________<br>
>>> NumPy-Discussion mailing list<br>
>>> <a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
>>> <a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
>>><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> NumPy-Discussion mailing list<br>
> <a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
> <a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
><br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Wed, 6 Jul 2016 09:51:16 -0400<br>
From: Benjamin Root <<a href="javascript:;" onclick="_e(event, 'cvml', 'ben.v.root@gmail.com')">ben.v.root@gmail.com</a>><br>
To: Discussion of Numerical Python <<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion@scipy.org')">numpy-discussion@scipy.org</a>><br>
Subject: Re: [Numpy-discussion] Added atleast_nd, request for<br>
clarification/cleanup of atleast_3d<br>
Message-ID:<br>
<<a href="javascript:;" onclick="_e(event, 'cvml', 'CANNq6FkrEmDM3DYW2ztHxxELW+wBeq2uj+viF3VNq4KsFEu_dQ@mail.gmail.com')">CANNq6FkrEmDM3DYW2ztHxxELW+wBeq2uj+viF3VNq4KsFEu_dQ@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
While atleast_1d/2d/3d predates my involvement in numpy, I am probably<br>
partly to blame for popularizing them as I helped to fix them up a fair<br>
amount. I wouldn't call its use "guessing". Rather, I would treat them as<br>
useful input sanitizers. If your function is going to be doing 2d indexing<br>
on an input, then it is very convenient to have atleast_2d() at the top of<br>
your function, not only to sanitize the input, but to make it clear that<br>
your code expects at least two dimensions.<br>
<br>
One place where it is used is in np.loadtxt(..., ndmin=N) to protect<br>
against the situation of a single row of data becoming a 1-D array rather<br>
than a 2-D array (or an empty text file returning something completely<br>
useless).<br>
<br>
I have previously pointed out the oddity with atleast_3d(). I can't<br>
remember the explanation I got though. Maybe someone can find the old<br>
thread that has the explanation, if any?<br>
<br>
I think the keyword argument approach for controlling the behavior might be<br>
a good approach, provided that a suitable design could be devised. 1 & 2<br>
dimensions is fairly trivial to control, but 3+ dimensions has too many<br>
degrees of freedom for me to consider.<br>
<br>
Cheers!<br>
Ben Root<br>
<br>
<br>
On Wed, Jul 6, 2016 at 9:12 AM, Joseph Fox-Rabinovitz <<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'jfoxrabinovitz@gmail.com')">jfoxrabinovitz@gmail.com</a>> wrote:<br>
<br>
> I can add a keyword-only argument that lets you put the new dims<br>
> before or after the existing ones. I am not sure how to specify<br>
> arbitrary patterns for the new dimensions, but that should take care<br>
> of most use cases.<br>
><br>
> The use case that motivated this function in the first place is that I<br>
> am doing some processing on 4D arrays and I need to reduce them but<br>
> return a result with the original dimensionality (but not shape).<br>
> atleast_nd seemed like a better solution than atleast_4d.<br>
><br>
> -Joe<br>
><br>
><br>
> On Wed, Jul 6, 2016 at 3:41 AM, <<a href="javascript:;" onclick="_e(event, 'cvml', 'josef.pktd@gmail.com')">josef.pktd@gmail.com</a>> wrote:<br>
> ><br>
> ><br>
> > On Wed, Jul 6, 2016 at 3:29 AM, <<a href="javascript:;" onclick="_e(event, 'cvml', 'josef.pktd@gmail.com')">josef.pktd@gmail.com</a>> wrote:<br>
> >><br>
> >><br>
> >><br>
> >> On Wed, Jul 6, 2016 at 2:21 AM, Ralf Gommers <<a href="javascript:;" onclick="_e(event, 'cvml', 'ralf.gommers@gmail.com')">ralf.gommers@gmail.com</a>><br>
> >> wrote:<br>
> >>><br>
> >>><br>
> >>><br>
> >>> On Wed, Jul 6, 2016 at 7:06 AM, Nathaniel Smith <<a href="javascript:;" onclick="_e(event, 'cvml', 'njs@pobox.com')">njs@pobox.com</a>> wrote:<br>
> >>><br>
> >>>> On Jul 5, 2016 9:09 PM, "Joseph Fox-Rabinovitz"<br>
> >>>> <<a href="javascript:;" onclick="_e(event, 'cvml', 'jfoxrabinovitz@gmail.com')">jfoxrabinovitz@gmail.com</a>> wrote:<br>
> >>>> ><br>
> >>>> > Hi,<br>
> >>>> ><br>
> >>>> > I have generalized np.atleast_1d, np.atleast_2d, np.atleast_3d with<br>
> a<br>
> >>>> > function np.atleast_nd in PR#7804<br>
> >>>> > (<a href="https://github.com/numpy/numpy/pull/7804" target="_blank">https://github.com/numpy/numpy/pull/7804</a>).<br>
> >>>> ><br>
> >>>> > As a result of this PR, I have a couple of questions about<br>
> >>>> > `np.atleast_3d`. `np.atleast_3d` appears to do something weird with<br>
> >>>> > the dimensions: If the input is 1D, it prepends and appends a size-1<br>
> >>>> > dimension. If the input is 2D, it appends a size-1 dimension. This<br>
> is<br>
> >>>> > inconsistent with `np.atleast_2d`, which always prepends (as does<br>
> >>>> > `np.atleast_nd`).<br>
> >>>> ><br>
> >>>> > - Is there any reason for this behavior?<br>
> >>>> > - Can it be cleaned up (e.g., by reimplementing `np.atleast_3d` in<br>
> >>>> > terms of `np.atleast_nd`, which is actually much simpler)? This<br>
> would<br>
> >>>> > be a slight API change since the output would not be exactly the<br>
> same.<br>
> >>>><br>
> >>>> Changing atleast_3d seems likely to break a bunch of stuff...<br>
> >>>><br>
> >>>> Beyond that, I find it hard to have an opinion about the best design<br>
> for<br>
> >>>> these functions, because I don't think I've ever encountered a<br>
> situation<br>
> >>>> where they were actually what I wanted. I'm not a big fan of coercing<br>
> >>>> dimensions in the first place, for the usual "refuse to guess"<br>
> reasons. And<br>
> >>>> then generally if I do want to coerce an array to another dimension,<br>
> then I<br>
> >>>> have some opinion about where the new dimensions should go, and/or I<br>
> have<br>
> >>>> some opinion about the minimum acceptable starting dimension, and/or<br>
> I have<br>
> >>>> a maximum dimension in mind. (E.g. "coerce 1d inputs into a column<br>
> matrix;<br>
> >>>> 0d or 3d inputs are an error" -- atleast_2d is zero-for-three on that<br>
> >>>> requirements list.)<br>
> >>>><br>
> >>>> I don't know how typical I am in this. But it does make me wonder if<br>
> the<br>
> >>>> atleast_* functions act as an attractive nuisance, where new users<br>
> take<br>
> >>>> their presence as an implicit recommendation that they are actually a<br>
> useful<br>
> >>>> thing to reach for, even though they... aren't that. And maybe we<br>
> should be<br>
> >>>> recommending folk move away from them rather than trying to extend<br>
> them<br>
> >>>> further?<br>
> >>>><br>
> >>>> Or maybe they're totally useful and I'm just missing it. What's your<br>
> use<br>
> >>>> case that motivates atleast_nd?<br>
> >>><br>
> >>> I think you're just missing it:) atleast_1d/2d are used quite a bit in<br>
> >>> Scipy and Statsmodels (those are the only ones I checked), and in the<br>
> large<br>
> >>> majority of cases it's the best thing to use there. There's a bunch of<br>
> >>> atleast_2d calls with a transpose appended because the input needs to<br>
> be<br>
> >>> treated as columns instead of rows, but that's still efficient and<br>
> readable<br>
> >>> enough.<br>
> >><br>
> >><br>
> >><br>
> >> As Ralph pointed out its usage in statsmodels. I do find them useful as<br>
> >> replacement for several lines of ifs and reshapes<br>
> >><br>
> >> We stilll need in many cases the atleast_2d_cols, that appends the<br>
> newaxis<br>
> >> if necessary.<br>
> >><br>
> >> roughly the equivalent of<br>
> >><br>
> >> if x.ndim == 1:<br>
> >> x = x[:, None]<br>
> >> else:<br>
> >> x = np.atleast_2d(x)<br>
> >><br>
> >> Josef<br>
> >><br>
> >>><br>
> >>><br>
> >>> For 3D/nD I can see that you'd need more control over where the<br>
> >>> dimensions go, but 1D/2D are fine.<br>
> ><br>
> ><br>
> ><br>
> > statsmodels has currently very little code with ndim >2, so I have no<br>
> > overview of possible use cases, but it would be necessary to have full<br>
> > control over the added axis since axis have a strict meaning and stats<br>
> still<br>
> > prefer Fortran order to default numpy/C ordering.<br>
> ><br>
> > Josef<br>
> ><br>
> ><br>
> >>><br>
> >>><br>
> >>><br>
> >>> Ralf<br>
> >>><br>
> >>><br>
> >>> _______________________________________________<br>
> >>> NumPy-Discussion mailing list<br>
> >>> <a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
> >>> <a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
> >>><br>
> >><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > NumPy-Discussion mailing list<br>
> > <a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
> > <a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
> ><br>
> _______________________________________________<br>
> NumPy-Discussion mailing list<br>
> <a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
> <a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="https://mail.scipy.org/pipermail/numpy-discussion/attachments/20160706/4241522b/attachment-0001.html" target="_blank">https://mail.scipy.org/pipermail/numpy-discussion/attachments/20160706/4241522b/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Wed, 6 Jul 2016 10:22:05 -0400<br>
From: Marten van Kerkwijk <<a href="javascript:;" onclick="_e(event, 'cvml', 'm.h.vankerkwijk@gmail.com')">m.h.vankerkwijk@gmail.com</a>><br>
To: Discussion of Numerical Python <<a href="javascript:;" onclick="_e(event, 'cvml', 'numpy-discussion@scipy.org')">numpy-discussion@scipy.org</a>><br>
Subject: Re: [Numpy-discussion] Added atleast_nd, request for<br>
clarification/cleanup of atleast_3d<br>
Message-ID:<br>
<<a href="javascript:;" onclick="_e(event, 'cvml', 'CAJNV+9vZX4xN6FvjUyn_h1zw+0a6qtt4nG3_NfOy7-cTL+RFMA@mail.gmail.com')">CAJNV+9vZX4xN6FvjUyn_h1zw+0a6qtt4nG3_NfOy7-cTL+RFMA@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi All,<br>
<br>
I'm with Nathaniel here, in that I don't really see the point of these<br>
routines in the first place: broadcasting takes care of many of the initial<br>
use cases one might think of, and others are generally not all that well<br>
served by them: the examples from scipy to me do not really support<br>
`at_least?d`, but rather suggest that little thought has been put into<br>
higher-dimensional objects which should be treated as stacks of row or<br>
column vectors. My sense is that we're better off developing the direction<br>
started with `matmul`, perhaps adding `matvecmul` etc.<br>
<br>
More to the point of the initial inquiry: what is the advantage of having a<br>
general `np.at_leastnd` routine over doing<br>
```<br>
np.array(a, copy=False, ndim=n)<br>
```<br>
or, for a list of inputs,<br>
```<br>
[np.array(a, copy=False, ndim=n) for a in input_list]<br>
```<br>
<br>
All the best,<br>
<br>
Marten<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="https://mail.scipy.org/pipermail/numpy-discussion/attachments/20160706/75f87f2d/attachment.html" target="_blank">https://mail.scipy.org/pipermail/numpy-discussion/attachments/20160706/75f87f2d/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
NumPy-Discussion mailing list<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'NumPy-Discussion@scipy.org')">NumPy-Discussion@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">https://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
<br>
<br>
------------------------------<br>
<br>
End of NumPy-Discussion Digest, Vol 118, Issue 9<br>
************************************************<br>
</blockquote>