[Numpy-discussion] Experimental `like=` attribute for array creation functions

Wed Aug 12 21:44:33 EDT 2020

I’ve generally been on the “let the NumPy devs worry about it” side of things, but I do agree with Ilhan that `like=` is confusing and `typeof=` would be a much more appropriate name for that parameter.

I do think library writers are NumPy users and so I wouldn’t really make that distinction, though. Users writing their own analysis code could very well be interested in writing code using numpy functions that will transparently work when the input is a CuPy array or whatever.

I also share Ilhan’s concern (and I mentioned this in a previous NEP discussion) that NEPs are getting pretty inaccessible. In a sense these are difficult topics and readers should be expected to have *some* familiarity with the topics being discussed, but perhaps more effort should be put into the context/motivation/background of a NEP before accepting it. One way to ensure this might be to require a final proofreading step by someone who has not been involved at all in the discussions, like peer review does for papers.

Food for thought.

Juan.

> On 13 Aug 2020, at 9:24 am, Ilhan Polat <ilhanpolat at gmail.com> wrote:
> 
> For what is worth, as a potential consumer in SciPy, it really doesn't say anything (both in NEP and the PR) about how the regular users of NumPy will benefit from this. If only and only 3rd parties are going to benefit from it, I am not sure adding a new keyword to an already confusing function is the right thing to do. 
> 
> Let me clarify,
> 
> - This is already a very (I mean extremely very) easy keyword name to confuse with ones_like, zeros_like and by its nature any other interpretation. It is not signalling anything about the functionality that is being discussed. I would seriously consider reserving such obvious names for really obvious tasks. Because you would also expect the shape and ndim would be mimicked by the "like"d argument but it turns out it is acting more like "typeof=" and not "like=" at all. Because if we follow the semantics it reads as "make your argument asarray like the other thing" but it is actually doing, "make your argument an array with the other thing's type" which might not be an array after all. 
> 
> - Again, if this is meant for downstream libraries (because that's what I got out of the PR discussion, cupy, dask, and JAX were the only examples I could read) then hiding it in another function and writing with capital letters "this is not meant for numpy users" would be a much more convenient way to separate the target audience and regular users. numpy.astypedarray([[some data], [...]], type_of=x) or whatever else it may be would be quite clean and to the point with no ambiguous keywords.
> 
> I think, arriving to an agreement would be much faster if there is an executive summary of who this is intended for and what the regular usage is. Because with no offense, all I see is "dispatch", "_array_function_" and a lot of technical details of which I am absolutely ignorant. 
> 
> Finally as a minor point, I know we are mostly (ex-)academics but this necessity of formal language on NEPs is self-imposed (probably PEPs are to blame) and not quite helping. It can be a bit more descriptive in my external opinion.
> 
> best,
> ilhan
> 
> 
> 
> 
> 
> 
> 
> On Tue, Aug 11, 2020 at 12:18 AM Ralf Gommers <ralf.gommers at gmail.com <mailto:ralf.gommers at gmail.com>> wrote:
> 
> 
> On Mon, Aug 10, 2020 at 8:37 PM Sebastian Berg <sebastian at sipsolutions.net <mailto:sebastian at sipsolutions.net>> wrote:
> On Mon, 2020-08-10 at 17:35 +0200, Hameer Abbasi wrote:
> > Hi,
> > 
> > We should have a higher-bandwidth meeting/communication for all
> > stakeholders, and particularly some library authors, to see what
> > would be good for them.
> 
> I'm not sure that helps. At this point there's little progress since the last meeting, I think the plan is unchanged: we need implementations of all the options on offer, and then try them out in PRs for scikit-learn, SciPy and perhaps another package who's maintainers are interested, to test like=, __array_module__ in realistic situations.
> 
> 
> > 
> > We should definitely have language in the NEP that says it won’t be
> > in a release unless the NEP is accepted.
> 
> In that case, I think the important part is to have language right now
> in the implementation, although that can refer to the NEP itself of
> course.
> You can't expect everyone who may be tempted to use it to actually read
> the NEP draft, at least not without pointing it out.
> 
> Agreed, I think the decision is on this list not in the NEP, and to make sure we won't forget we need an issue opened with the 1.20 milestone.
> 
> Cheers,
> Ralf
> 
> 
> I will say that I think it is not very high risk, because I think
> annoying or not, the argument could be deprecated again with a
> transition short phase. Admittedly, that argument only works if we have
> a replacement solution.
> 
> Cheers,
> 
> Sebastian
> 
> 
> > 
> > Best regards,
> > Hameer Abbasi
> > 
> > --
> > Sent from Canary (https://canarymail.io <https://canarymail.io/>)
> > 
> > > On Monday, Aug 10, 2020 at 5:31 PM, Sebastian Berg <
> > > sebastian at sipsolutions.net <mailto:sebastian at sipsolutions.net> (mailto:sebastian at sipsolutions.net <mailto:sebastian at sipsolutions.net>)>
> > > wrote:
> > > Hi all,
> > > 
> > > as a heads up that Peter Entschev has a PR open to add `like=` to
> > > most array creation functions, my current plan is to merge it soon
> > > as a preliminary API and bring it up again before the actual
> > > release (in a few months). This allows overriding for array-likes,
> > > e.g. it will allow:
> > > 
> > > 
> > > arr = np.asarray([3], like=dask_array)
> > > type(arr) is dask.array.Array
> > > 
> > > This was proposed in NEP 35:
> > > 
> > > https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function.html <https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function.html>
> > > 
> > > Although that has not been accepted as of now, the PR is:
> > > 
> > > https://github.com/numpy/numpy/pull/16935 <https://github.com/numpy/numpy/pull/16935>
> > > 
> > > 
> > > This was discussed in a smaller group, and is an attempt to see how
> > > we
> > > can make the array-function protocol viable to allow packages such
> > > as
> > > sklearn to work with non-NumPy arrays.
> > > 
> > > As of now, this would be experimental and can revisit it before the
> > > actual NumPy release. We should probably discuss accepting NEP 35
> > > more. At this time, I hope that we can put in the functionality to
> > > facilitate this discussion, rather the other way around.
> > > 
> > > If anyone feels nervous about this step, I would be happy to
> > > document
> > > that we will not include it in the next release unless the NEP is
> > > accepted first, or at least hide it behind an environment variable.
> > > 
> > > Cheers,
> > > 
> > > Sebastian
> > > 
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> > > https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> > https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org <mailto:NumPy-Discussion at python.org>
> https://mail.python.org/mailman/listinfo/numpy-discussion <https://mail.python.org/mailman/listinfo/numpy-discussion>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200813/9ba46186/attachment-0001.html>