From peter at entschev.com  Mon Aug  5 10:58:24 2019
From: peter at entschev.com (Peter Andreas Entschev)
Date: Mon, 5 Aug 2019 16:58:24 +0200
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
Message-ID: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>

Hi,

we have a new proposal for the implementation of NumPy array duck
typing [1] [2], following the high-level overview described in NEP-22
[3].

Would be great to get some comments on that.

[1] https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
[2] https://github.com/numpy/numpy/pull/14170
[3] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html

Best,
Peter

From ralf.gommers at gmail.com  Mon Aug  5 17:45:21 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Aug 2019 14:45:21 -0700
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
Message-ID: <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>

Hi Peter, thanks for writing that up!


On Mon, Aug 5, 2019 at 8:07 AM Peter Andreas Entschev <peter at entschev.com>
wrote:

> Hi,
>
> we have a new proposal for the implementation of NumPy array duck
> typing [1] [2], following the high-level overview described in NEP-22
> [3].
>

A couple of high level comments:

Having __array__ give a TypeError is fine for libraries that want to
prevent unintentional coercion with, e.g., `np.asarray(my_ducktype)`.
However that leaves the obvious question of what the right way is to do
this intentionally. Would be good to recommend something, for example a
`numpy()` or `to_numpy()` method. Also, the NEP should make it clearer that
this is not the obviously correct thing to do, it only makes sense in cases
where coercion is very expensive, like CuPy and Sparse. For Dask for
example, coercion to a numpy array is perfectly reasonable.

The NEP currently does not say who this is meant for. Would you expect
libraries like SciPy to adopt it for example?

The NEP also (understandably) punts on the question of when something is a
valid duck array. If you want this to be widely used, that will need an
answer or at least some rough guidance though. For example, we would expect
a duck array to have a mean() method, but probably not a ptp() method. A
library author who wants to use np.duckarray() needs to know, because she
can't test with all existing and future duck array implementations.

An alternative to introducing np.duckarray() would be to just modify
np.asarray(). Of course this has backwards compatibility impact, but if
you're going to be raising a TypeError from __array__ then that impact is
there anyway. Note: I don't think this is necessarily a better idea,
because it may lead to less clear errors, but it's worth putting in the
alternatives section at least.

Cheers,
Ralf




> Would be great to get some comments on that.
>
> [1]
> https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
> [2] https://github.com/numpy/numpy/pull/14170
> [3] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
>
> Best,
> Peter
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190805/bee2a3ee/attachment.html>

From ralf.gommers at gmail.com  Mon Aug  5 20:15:23 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Aug 2019 17:15:23 -0700
Subject: [Numpy-discussion] proposal to accept NEP 28 (numpy.org website
 redesign)
Message-ID: <CABL7CQhcn=emwz2Gk-H9h_V5BvePyinYaEG4xwBewuLGXewcUw@mail.gmail.com>

Hi all,

I propose to accept NEP 28 (https://github.com/numpy/numpy/pull/14032).
Discussion seems to have died down, and there have been no major objections
raised.

In addition to the discussion on the PR, we had some discussion on
translations in the community call 1.5 weeks ago. We decided that it was
low-risk/impact to simply try this out with one language, and Zijie already
volunteered to help so we can try this with Mandarin. Once that's done, we
can evaluate the process and the cost/benefit better.

If there are no substantive objections within 5 days from this email, then
the NEP will be accepted.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190805/93a2b81d/attachment.html>

From shoyer at gmail.com  Mon Aug  5 21:17:27 2019
From: shoyer at gmail.com (Stephan Hoyer)
Date: Mon, 5 Aug 2019 18:17:27 -0700
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
Message-ID: <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>

On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Having __array__ give a TypeError is fine for libraries that want to
> prevent unintentional coercion with, e.g., `np.asarray(my_ducktype)`.
> However that leaves the obvious question of what the right way is to do
> this intentionally. Would be good to recommend something, for example a
> `numpy()` or `to_numpy()` method. Also, the NEP should make it clearer that
> this is not the obviously correct thing to do, it only makes sense in cases
> where coercion is very expensive, like CuPy and Sparse. For Dask for
> example, coercion to a numpy array is perfectly reasonable.
>

I agree, we need another solution for explicit array conversion, either
from duck arrays to NumPy arrays or between duck arrays.

As has come-up on GitHub [1], think this should probably be another
protocol, to allow for third-party conversions like sparse <-> dask that in
principle could be implemented by either library.

To get discussion start, here's one possible proposal for what the NumPy
API(s) could look like:
np.coerce(sparse_array)  # by default, coerce to np.ndarray
np.coerce(sparse_array, dask.array.Array)  # coerces to dask
np.coerce_like(sparse_array, dask_array)  # coerce like the second array
type
np.coerce_arrays(list_of_arrays)  # coerce to first type that can handle
everything

The protocol itself should probably either use __array_function__ (e.g.,
for np.coerce_like, if all the dispatched on arguments are arrays) or a
custom protocol in the same style that allows for implementations on either
the array being converted or the type of the result [2].

[1] https://github.com/numpy/numpy/issues/13831
[2] https://github.com/numpy/numpy/pull/14170#issuecomment-517004293


> The NEP currently does not say who this is meant for. Would you expect
> libraries like SciPy to adopt it for example?
>
> The NEP also (understandably) punts on the question of when something is a
> valid duck array. If you want this to be widely used, that will need an
> answer or at least some rough guidance though. For example, we would expect
> a duck array to have a mean() method, but probably not a ptp() method. A
> library author who wants to use np.duckarray() needs to know, because she
> can't test with all existing and future duck array implementations.
>

I think this is covered in NEP-22 already. As discussed there, I don't
think NumPy is in a good position to pronounce decisive APIs at this time.
I would welcome efforts to try, but I don't think that's essential for now.

An alternative to introducing np.duckarray() would be to just modify
> np.asarray(). Of course this has backwards compatibility impact, but if
> you're going to be raising a TypeError from __array__ then that impact is
> there anyway. Note: I don't think this is necessarily a better idea,
> because it may lead to less clear errors, but it's worth putting in the
> alternatives section at least.
>
> Cheers,
> Ralf
>
>>
>> Would be great to get some comments on that.
>>
>> [1]
>> https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
>> [2] https://github.com/numpy/numpy/pull/14170
>> [3] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
>>
>> Best,
>> Peter
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190805/0196375f/attachment-0001.html>

From ralf.gommers at gmail.com  Tue Aug  6 01:24:28 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 5 Aug 2019 22:24:28 -0700
Subject: [Numpy-discussion] did anyone create numpy.slack.com?
Message-ID: <CABL7CQhPRXngq3feqiJeS15_a19gDOJQV4k50Bj471-XVxnnpw@mail.gmail.com>

Hi all,

I was trying to create a Slack workspace named numpy, but it turns out that
numpy.slack.com already exists. Does anyone have access to it or knows who
is administering it?

Thanks,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190805/20c37af3/attachment.html>

From peter at entschev.com  Tue Aug  6 04:24:44 2019
From: peter at entschev.com (Peter Andreas Entschev)
Date: Tue, 6 Aug 2019 10:24:44 +0200
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
Message-ID: <CANxROpziwqz4cpirJW4Gm=FnNmNfJitu8aZMzemMS+T8Pc2h9Q@mail.gmail.com>

Thanks for the concerns raised, and Stephan for promptly answering them.

> An alternative to introducing np.duckarray() would be to just modify np.asarray(). Of course this has backwards compatibility impact, but if you're going to be raising a TypeError from __array__ then that impact is there anyway. Note: I don't think this is necessarily a better idea, because it may lead to less clear errors, but it's worth putting in the alternatives section at least.

I don't know if mentioning alternatives in that way is good, it gives
me the impression that NumPy is encouraging them (unless it really
is). In that sense, I think the best is indeed going down the path of
finding a good coercion solution (as Stephan already mentioned) and
later we could just add a pointer to the new NEP in this one.

Best,
Peter


On Tue, Aug 6, 2019 at 3:17 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>
> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>> Having __array__ give a TypeError is fine for libraries that want to prevent unintentional coercion with, e.g., `np.asarray(my_ducktype)`. However that leaves the obvious question of what the right way is to do this intentionally. Would be good to recommend something, for example a `numpy()` or `to_numpy()` method. Also, the NEP should make it clearer that this is not the obviously correct thing to do, it only makes sense in cases where coercion is very expensive, like CuPy and Sparse. For Dask for example, coercion to a numpy array is perfectly reasonable.
>
>
> I agree, we need another solution for explicit array conversion, either from duck arrays to NumPy arrays or between duck arrays.
>
> As has come-up on GitHub [1], think this should probably be another protocol, to allow for third-party conversions like sparse <-> dask that in principle could be implemented by either library.
>
> To get discussion start, here's one possible proposal for what the NumPy API(s) could look like:
> np.coerce(sparse_array)  # by default, coerce to np.ndarray
> np.coerce(sparse_array, dask.array.Array)  # coerces to dask
> np.coerce_like(sparse_array, dask_array)  # coerce like the second array type
> np.coerce_arrays(list_of_arrays)  # coerce to first type that can handle everything
>
> The protocol itself should probably either use __array_function__ (e.g., for np.coerce_like, if all the dispatched on arguments are arrays) or a custom protocol in the same style that allows for implementations on either the array being converted or the type of the result [2].
>
> [1] https://github.com/numpy/numpy/issues/13831
> [2] https://github.com/numpy/numpy/pull/14170#issuecomment-517004293
>
>>
>> The NEP currently does not say who this is meant for. Would you expect libraries like SciPy to adopt it for example?
>>
>> The NEP also (understandably) punts on the question of when something is a valid duck array. If you want this to be widely used, that will need an answer or at least some rough guidance though. For example, we would expect a duck array to have a mean() method, but probably not a ptp() method. A library author who wants to use np.duckarray() needs to know, because she can't test with all existing and future duck array implementations.
>
>
> I think this is covered in NEP-22 already. As discussed there, I don't think NumPy is in a good position to pronounce decisive APIs at this time. I would welcome efforts to try, but I don't think that's essential for now.
>
>> An alternative to introducing np.duckarray() would be to just modify np.asarray(). Of course this has backwards compatibility impact, but if you're going to be raising a TypeError from __array__ then that impact is there anyway. Note: I don't think this is necessarily a better idea, because it may lead to less clear errors, but it's worth putting in the alternatives section at least.
>>
>> Cheers,
>> Ralf
>>>
>>>
>>> Would be great to get some comments on that.
>>>
>>> [1] https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
>>> [2] https://github.com/numpy/numpy/pull/14170
>>> [3] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
>>>
>>> Best,
>>> Peter
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From deak.andris at gmail.com  Tue Aug  6 04:48:36 2019
From: deak.andris at gmail.com (Andras Deak)
Date: Tue, 6 Aug 2019 10:48:36 +0200
Subject: [Numpy-discussion] 
	=?utf-8?q?NEP_28_=E2=80=94_A_standard_communi?=
	=?utf-8?q?ty_policy_for_dropping_support_of_old_Python_and_NumPy_v?=
	=?utf-8?q?ersions?=
In-Reply-To: <CAA48SF_BCV7rvekWgcNycbGrgrTvYXuVWzh7HwPrTOCfKByiBA@mail.gmail.com>
References: <CAA48SF_BCV7rvekWgcNycbGrgrTvYXuVWzh7HwPrTOCfKByiBA@mail.gmail.com>
Message-ID: <CAMEWA4Mi3PjuBsYgSpE7zk=aOuT11VHW0K0iZA0wuVqgFo1G=w@mail.gmail.com>

Hi,

This is just a reminder for others like myself who have too limited a
cognitive buffer: this NEP was renumbered and it's NEP 29 now
https://github.com/numpy/numpy/pull/14086/files (just to prevent
possible confusion).

Andr?s

On Wed, Jul 24, 2019 at 1:52 AM Thomas Caswell <tcaswell at gmail.com> wrote:
>
> Folks,
>
> This NEP is a proposing a standard policy for the community to determine when we age-out support for old  versions of Python. This came out of in-person discussions at SciPy earlier in July and scattered discussion across github. This is being proposed by maintainers from Matplotlib, scikit-learn,
> IPython, Jupyter, yt, SciPy, NumPy, and scikit-image.
>
> TL;DR:
>
> We propose only supporting versions of CPython initially released in the preceding 42 months of a major or minor release of any of our projects.
>
> Please see https://github.com/numpy/numpy/pull/14086 and keep the discussion there as there are many interested parties (from the other projects) that may not be subscribed to the numpy mailing list.
>
> Tom
>
> --
> Thomas Caswell
> tcaswell at gmail.com
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From pankaj.jangid at gmail.com  Tue Aug  6 05:38:19 2019
From: pankaj.jangid at gmail.com (Pankaj Jangid)
Date: Tue, 6 Aug 2019 15:08:19 +0530
Subject: [Numpy-discussion] did anyone create numpy.slack.com?
In-Reply-To: <CABL7CQhPRXngq3feqiJeS15_a19gDOJQV4k50Bj471-XVxnnpw@mail.gmail.com>
References: <CABL7CQhPRXngq3feqiJeS15_a19gDOJQV4k50Bj471-XVxnnpw@mail.gmail.com>
Message-ID: <20190806093819.GB21160@Pankajs-MacBook-Pro.local>

On Mon, Aug 05, 2019 at 10:24:28PM -0700, Ralf Gommers wrote:
> I was trying to create a Slack workspace named numpy, but it turns out that
> numpy.slack.com already exists. Does anyone have access to it or knows who
> is administering it?

A lot of users try to grab all possible IDs available. A friend of mine
had reserved gmail-IDs of almost all celebrities, as if they will
register on Gmail. :-)

I guess, it is one of those. Try using some combination - 'numpy-xxxx'.

Regards.

-- 
Pankaj
Planet Earth. 

""
You will be recognized and honored as a community leader.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/4b67e389/attachment.sig>

From jni at fastmail.com  Tue Aug  6 05:51:04 2019
From: jni at fastmail.com (Juan Nunez-Iglesias)
Date: Tue, 6 Aug 2019 19:51:04 +1000
Subject: [Numpy-discussion] did anyone create numpy.slack.com?
In-Reply-To: <20190806093819.GB21160@Pankajs-MacBook-Pro.local>
References: <CABL7CQhPRXngq3feqiJeS15_a19gDOJQV4k50Bj471-XVxnnpw@mail.gmail.com>
 <20190806093819.GB21160@Pankajs-MacBook-Pro.local>
Message-ID: <AE83FD68-3E9F-46E4-A693-4621EFA058DE@fastmail.com>

FWIW we use Zulip for scikit-image and we are very happy with it. See https://skimage.zulipchat.com <https://skimage.zulipchat.com/> . Advantages over Slack:

- free for FOSS, including unlimited archives (https://zulipchat.com/for/open-source/ <https://zulipchat.com/for/open-source/>)
- login with GitHub
- Zulipchat is itself FOSS (I have successfully set up my own instance before)
- (upcoming) public archive (read without login)

Juan.

> On 6 Aug 2019, at 7:38 pm, Pankaj Jangid <pankaj.jangid at gmail.com> wrote:
> 
> On Mon, Aug 05, 2019 at 10:24:28PM -0700, Ralf Gommers wrote:
>> I was trying to create a Slack workspace named numpy, but it turns out that
>> numpy.slack.com already exists. Does anyone have access to it or knows who
>> is administering it?
> 
> A lot of users try to grab all possible IDs available. A friend of mine
> had reserved gmail-IDs of almost all celebrities, as if they will
> register on Gmail. :-)
> 
> I guess, it is one of those. Try using some combination - 'numpy-xxxx'.
> 
> Regards.
> 
> -- 
> Pankaj
> Planet Earth. 
> 
> ""
> You will be recognized and honored as a community leader.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/66b7d5ee/attachment-0001.html>

From sebastian at sipsolutions.net  Tue Aug  6 10:23:42 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 06 Aug 2019 09:23:42 -0500
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CANxROpziwqz4cpirJW4Gm=FnNmNfJitu8aZMzemMS+T8Pc2h9Q@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CANxROpziwqz4cpirJW4Gm=FnNmNfJitu8aZMzemMS+T8Pc2h9Q@mail.gmail.com>
Message-ID: <1d4c2d74f6284bf2390d134224a86ed9d75991c0.camel@sipsolutions.net>

On Tue, 2019-08-06 at 10:24 +0200, Peter Andreas Entschev wrote:
> Thanks for the concerns raised, and Stephan for promptly answering
> them.
> 
> > An alternative to introducing np.duckarray() would be to just
> > modify np.asarray(). Of course this has backwards compatibility
> > impact, but if you're going to be raising a TypeError from
> > __array__ then that impact is there anyway. Note: I don't think
> > this is necessarily a better idea, because it may lead to less
> > clear errors, but it's worth putting in the alternatives section at
> > least.
> 
> I don't know if mentioning alternatives in that way is good, it gives
> me the impression that NumPy is encouraging them (unless it really
> is).

Well, if you think alternatives is too open to actually using them, I
think it would be fine to list them as "Rejected Alternatives"?

- Sebastian


>  In that sense, I think the best is indeed going down the path of
> finding a good coercion solution (as Stephan already mentioned) and
> later we could just add a pointer to the new NEP in this one.
> 
> Best,
> Peter
> 
> 
> On Tue, Aug 6, 2019 at 3:17 AM Stephan Hoyer <shoyer at gmail.com>
> wrote:
> > On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com
> > > wrote:
> > > Having __array__ give a TypeError is fine for libraries that want
> > > to prevent unintentional coercion with, e.g.,
> > > `np.asarray(my_ducktype)`. However that leaves the obvious
> > > question of what the right way is to do this intentionally. Would
> > > be good to recommend something, for example a `numpy()` or
> > > `to_numpy()` method. Also, the NEP should make it clearer that
> > > this is not the obviously correct thing to do, it only makes
> > > sense in cases where coercion is very expensive, like CuPy and
> > > Sparse. For Dask for example, coercion to a numpy array is
> > > perfectly reasonable.
> > 
> > I agree, we need another solution for explicit array conversion,
> > either from duck arrays to NumPy arrays or between duck arrays.
> > 
> > As has come-up on GitHub [1], think this should probably be another
> > protocol, to allow for third-party conversions like sparse <-> dask
> > that in principle could be implemented by either library.
> > 
> > To get discussion start, here's one possible proposal for what the
> > NumPy API(s) could look like:
> > np.coerce(sparse_array)  # by default, coerce to np.ndarray
> > np.coerce(sparse_array, dask.array.Array)  # coerces to dask
> > np.coerce_like(sparse_array, dask_array)  # coerce like the second
> > array type
> > np.coerce_arrays(list_of_arrays)  # coerce to first type that can
> > handle everything
> > 
> > The protocol itself should probably either use __array_function__
> > (e.g., for np.coerce_like, if all the dispatched on arguments are
> > arrays) or a custom protocol in the same style that allows for
> > implementations on either the array being converted or the type of
> > the result [2].
> > 
> > [1] https://github.com/numpy/numpy/issues/13831
> > [2] 
> > https://github.com/numpy/numpy/pull/14170#issuecomment-517004293
> > 
> > > The NEP currently does not say who this is meant for. Would you
> > > expect libraries like SciPy to adopt it for example?
> > > 
> > > The NEP also (understandably) punts on the question of when
> > > something is a valid duck array. If you want this to be widely
> > > used, that will need an answer or at least some rough guidance
> > > though. For example, we would expect a duck array to have a
> > > mean() method, but probably not a ptp() method. A library author
> > > who wants to use np.duckarray() needs to know, because she can't
> > > test with all existing and future duck array implementations.
> > 
> > I think this is covered in NEP-22 already. As discussed there, I
> > don't think NumPy is in a good position to pronounce decisive APIs
> > at this time. I would welcome efforts to try, but I don't think
> > that's essential for now.
> > 
> > > An alternative to introducing np.duckarray() would be to just
> > > modify np.asarray(). Of course this has backwards compatibility
> > > impact, but if you're going to be raising a TypeError from
> > > __array__ then that impact is there anyway. Note: I don't think
> > > this is necessarily a better idea, because it may lead to less
> > > clear errors, but it's worth putting in the alternatives section
> > > at least.
> > > 
> > > Cheers,
> > > Ralf
> > > > 
> > > > Would be great to get some comments on that.
> > > > 
> > > > [1] 
> > > > https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
> > > > [2] https://github.com/numpy/numpy/pull/14170
> > > > [3] 
> > > > https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
> > > > 
> > > > Best,
> > > > Peter
> > > > _______________________________________________
> > > > NumPy-Discussion mailing list
> > > > NumPy-Discussion at python.org
> > > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > > 
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org
> > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/53089b1b/attachment.sig>

From peter at entschev.com  Tue Aug  6 11:08:34 2019
From: peter at entschev.com (Peter Andreas Entschev)
Date: Tue, 6 Aug 2019 17:08:34 +0200
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <1d4c2d74f6284bf2390d134224a86ed9d75991c0.camel@sipsolutions.net>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CANxROpziwqz4cpirJW4Gm=FnNmNfJitu8aZMzemMS+T8Pc2h9Q@mail.gmail.com>
 <1d4c2d74f6284bf2390d134224a86ed9d75991c0.camel@sipsolutions.net>
Message-ID: <CANxROpzPQMDZsTyFgur4O1-0UqiyNfxrmqs3zz8Huwr-2d_r_g@mail.gmail.com>

Sure, I wouldn't mind doing that, but it would also be better to have
clear alternative/complement to duck array (as I'm hoping to be the
case with with coerce). I will try to give a bit more thought on the
coercion ideas and start writing a NEP for that this week and the
next. Perhaps we can then sync both NEPs.

On Tue, Aug 6, 2019 at 4:23 PM Sebastian Berg
<sebastian at sipsolutions.net> wrote:
>
> On Tue, 2019-08-06 at 10:24 +0200, Peter Andreas Entschev wrote:
> > Thanks for the concerns raised, and Stephan for promptly answering
> > them.
> >
> > > An alternative to introducing np.duckarray() would be to just
> > > modify np.asarray(). Of course this has backwards compatibility
> > > impact, but if you're going to be raising a TypeError from
> > > __array__ then that impact is there anyway. Note: I don't think
> > > this is necessarily a better idea, because it may lead to less
> > > clear errors, but it's worth putting in the alternatives section at
> > > least.
> >
> > I don't know if mentioning alternatives in that way is good, it gives
> > me the impression that NumPy is encouraging them (unless it really
> > is).
>
> Well, if you think alternatives is too open to actually using them, I
> think it would be fine to list them as "Rejected Alternatives"?
>
> - Sebastian
>
>
> >  In that sense, I think the best is indeed going down the path of
> > finding a good coercion solution (as Stephan already mentioned) and
> > later we could just add a pointer to the new NEP in this one.
> >
> > Best,
> > Peter
> >
> >
> > On Tue, Aug 6, 2019 at 3:17 AM Stephan Hoyer <shoyer at gmail.com>
> > wrote:
> > > On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com
> > > > wrote:
> > > > Having __array__ give a TypeError is fine for libraries that want
> > > > to prevent unintentional coercion with, e.g.,
> > > > `np.asarray(my_ducktype)`. However that leaves the obvious
> > > > question of what the right way is to do this intentionally. Would
> > > > be good to recommend something, for example a `numpy()` or
> > > > `to_numpy()` method. Also, the NEP should make it clearer that
> > > > this is not the obviously correct thing to do, it only makes
> > > > sense in cases where coercion is very expensive, like CuPy and
> > > > Sparse. For Dask for example, coercion to a numpy array is
> > > > perfectly reasonable.
> > >
> > > I agree, we need another solution for explicit array conversion,
> > > either from duck arrays to NumPy arrays or between duck arrays.
> > >
> > > As has come-up on GitHub [1], think this should probably be another
> > > protocol, to allow for third-party conversions like sparse <-> dask
> > > that in principle could be implemented by either library.
> > >
> > > To get discussion start, here's one possible proposal for what the
> > > NumPy API(s) could look like:
> > > np.coerce(sparse_array)  # by default, coerce to np.ndarray
> > > np.coerce(sparse_array, dask.array.Array)  # coerces to dask
> > > np.coerce_like(sparse_array, dask_array)  # coerce like the second
> > > array type
> > > np.coerce_arrays(list_of_arrays)  # coerce to first type that can
> > > handle everything
> > >
> > > The protocol itself should probably either use __array_function__
> > > (e.g., for np.coerce_like, if all the dispatched on arguments are
> > > arrays) or a custom protocol in the same style that allows for
> > > implementations on either the array being converted or the type of
> > > the result [2].
> > >
> > > [1] https://github.com/numpy/numpy/issues/13831
> > > [2]
> > > https://github.com/numpy/numpy/pull/14170#issuecomment-517004293
> > >
> > > > The NEP currently does not say who this is meant for. Would you
> > > > expect libraries like SciPy to adopt it for example?
> > > >
> > > > The NEP also (understandably) punts on the question of when
> > > > something is a valid duck array. If you want this to be widely
> > > > used, that will need an answer or at least some rough guidance
> > > > though. For example, we would expect a duck array to have a
> > > > mean() method, but probably not a ptp() method. A library author
> > > > who wants to use np.duckarray() needs to know, because she can't
> > > > test with all existing and future duck array implementations.
> > >
> > > I think this is covered in NEP-22 already. As discussed there, I
> > > don't think NumPy is in a good position to pronounce decisive APIs
> > > at this time. I would welcome efforts to try, but I don't think
> > > that's essential for now.
> > >
> > > > An alternative to introducing np.duckarray() would be to just
> > > > modify np.asarray(). Of course this has backwards compatibility
> > > > impact, but if you're going to be raising a TypeError from
> > > > __array__ then that impact is there anyway. Note: I don't think
> > > > this is necessarily a better idea, because it may lead to less
> > > > clear errors, but it's worth putting in the alternatives section
> > > > at least.
> > > >
> > > > Cheers,
> > > > Ralf
> > > > >
> > > > > Would be great to get some comments on that.
> > > > >
> > > > > [1]
> > > > > https://github.com/numpy/numpy/blob/master/doc/neps/nep-0030-duck-array-protocol.rst
> > > > > [2] https://github.com/numpy/numpy/pull/14170
> > > > > [3]
> > > > > https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html
> > > > >
> > > > > Best,
> > > > > Peter
> > > > > _______________________________________________
> > > > > NumPy-Discussion mailing list
> > > > > NumPy-Discussion at python.org
> > > > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > > >
> > > > _______________________________________________
> > > > NumPy-Discussion mailing list
> > > > NumPy-Discussion at python.org
> > > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > >
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org
> > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From ralf.gommers at gmail.com  Tue Aug  6 11:13:36 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 6 Aug 2019 08:13:36 -0700
Subject: [Numpy-discussion] did anyone create numpy.slack.com?
In-Reply-To: <AE83FD68-3E9F-46E4-A693-4621EFA058DE@fastmail.com>
References: <CABL7CQhPRXngq3feqiJeS15_a19gDOJQV4k50Bj471-XVxnnpw@mail.gmail.com>
 <20190806093819.GB21160@Pankajs-MacBook-Pro.local>
 <AE83FD68-3E9F-46E4-A693-4621EFA058DE@fastmail.com>
Message-ID: <CABL7CQjqS2pG=W=uoRM-eSFc2WFY6cEGWo0mxuricBmCpFfvsw@mail.gmail.com>

On Tue, Aug 6, 2019 at 2:51 AM Juan Nunez-Iglesias <jni at fastmail.com> wrote:

> FWIW we use Zulip for scikit-image and we are very happy with it. See
> https://skimage.zulipchat.com . Advantages over Slack:
>

There's a NumPy Gitter as well, but most core devs are not using it. Note
that this is mostly meant for dev discussions and (the reason I'm creating
it) mentoring new people 1:1 or in a small group. All actual new volunteers
said in the community call that they expected Slack, and people like
Sebastian, Stefan and myself are on Slack very regularly as well.

This is related to https://github.com/numpy/numpy/pull/14027. Public
channels (mailing list, GitHub) are our preferred communication mechanism;
this and a new numpy-team at googlegroups.com are additions for private
communications to lower the bar to start contributing. It turns out not
everyone is comfortable with their first question or code being seen by
1000+ people.

tl;dr the usual worries you post below aren't very relevant in this case,
and anything but Slack is pretty much a non-starter.

Cheers,
Ralf



> - free for FOSS, including unlimited archives (
> https://zulipchat.com/for/open-source/)
> - login with GitHub
> - Zulipchat is itself FOSS (I have successfully set up my own instance
> before)
> - (upcoming) public archive (read without login)
>
> Juan.
>
> On 6 Aug 2019, at 7:38 pm, Pankaj Jangid <pankaj.jangid at gmail.com> wrote:
>
> On Mon, Aug 05, 2019 at 10:24:28PM -0700, Ralf Gommers wrote:
>
> I was trying to create a Slack workspace named numpy, but it turns out that
> numpy.slack.com already exists. Does anyone have access to it or knows who
> is administering it?
>
>
> A lot of users try to grab all possible IDs available. A friend of mine
> had reserved gmail-IDs of almost all celebrities, as if they will
> register on Gmail. :-)
>
> I guess, it is one of those. Try using some combination - 'numpy-xxxx'.
>
> Regards.
>
> --
> Pankaj
> Planet Earth.
>
> ""
> You will be recognized and honored as a community leader.
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/50f336b7/attachment-0001.html>

From sebastian at sipsolutions.net  Tue Aug  6 16:12:22 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 06 Aug 2019 15:12:22 -0500
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, July 17
Message-ID: <ce611afd38b4194985e9e2c0b24299dee9696fbc.camel@sipsolutions.net>

Hi all,

There will be a NumPy Community meeting Wednesday August 7 at 11 am
Pacific Time. Everyone is invited to join in and edit the work-in-
progress meeting topics and notes:
https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg

Best wishes

Sebastian


PS: (Sorry if you recently updated the file, I may have deleted some
changes)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/86e4d057/attachment.sig>

From charlesr.harris at gmail.com  Tue Aug  6 16:25:36 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 6 Aug 2019 14:25:36 -0600
Subject: [Numpy-discussion] (no subject)
Message-ID: <CAB6mnx+731Hbts8yFb1ZCdKGWm8OM_4ke9sbKoSLG2ncDPQ8oQ@mail.gmail.com>

Hi All,

There is a PR <https://github.com/numpy/numpy/pull/14197> up that adds a
"mode" variable to `multivariate_normal`. It allows choosing various
options for factoring the passed covariance.


   1. "fast" -- cholesky (x45 legacy method)
   2. "default" -- eigh (used in R I believe)
   3. "legacy" -- svd (current method)
   4. "factor" -- proposed

Feedback is welcome.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/513ca73c/attachment.html>

From ralf.gommers at gmail.com  Tue Aug  6 19:46:36 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 6 Aug 2019 16:46:36 -0700
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
Message-ID: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>

Hi all,

Google has announced the Season of Docs participants for this year [1]. We
had a lot of excellent candidates and had to make some hard choices. We
applied for extra slots, but unfortunately didn't win the lottery for
those; we got one slot for NumPy and one for SciPy. We chose the projects
of Anne for NumPy and Maja for SciPy:

Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
Introductory NumPy Sections" [2]

Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]

That's not all though. There was some space left in the budget of the NumPy
BIDS grant, and St?fan has reserved that so we can accept more writers and
provide them the same mentoring and funding as they would have gotten
through GSoD. We could only start the conversations about that once Google
made its decisions, so a further announcement will follow. However, we
already have one extra project confirmed, from Brandon:

Brandon David, "Improve the documentation of scipy.stats" (project details
to be published).

I will send out a poll to find a good time for everyone for a kickoff call.
Our intent is to build a documentation team with multiple writers and
mentors interacting and able to help each other out. And all of this will
also interact with the numpy.org website redesign and the people putting
energy into that:)

I'm very happy to welcome Anne, Maja and Brandon!

Cheers,
Ralf


[1] https://developers.google.com/season-of-docs/docs/participants/
[2]
https://developers.google.com/season-of-docs/docs/participants/project-numpy
[3]
https://developers.google.com/season-of-docs/docs/participants/project-scipy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/985a99bf/attachment.html>

From ilhanpolat at gmail.com  Tue Aug  6 21:33:56 2019
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Wed, 7 Aug 2019 03:33:56 +0200
Subject: [Numpy-discussion] [SciPy-Dev] Season of Docs - welcome Anne,
 Maja, Brandon
In-Reply-To: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
Message-ID: <CAEBuzr8JFsSOOGGkHY4LqJhuvSrFdTWpOTBnrF0ZcRFANcia8w@mail.gmail.com>

Great news, welcome all!

On Wed, Aug 7, 2019 at 1:47 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Google has announced the Season of Docs participants for this year [1]. We
> had a lot of excellent candidates and had to make some hard choices. We
> applied for extra slots, but unfortunately didn't win the lottery for
> those; we got one slot for NumPy and one for SciPy. We chose the projects
> of Anne for NumPy and Maja for SciPy:
>
> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
> Introductory NumPy Sections" [2]
>
> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>
> That's not all though. There was some space left in the budget of the
> NumPy BIDS grant, and St?fan has reserved that so we can accept more
> writers and provide them the same mentoring and funding as they would have
> gotten through GSoD. We could only start the conversations about that once
> Google made its decisions, so a further announcement will follow. However,
> we already have one extra project confirmed, from Brandon:
>
> Brandon David, "Improve the documentation of scipy.stats" (project details
> to be published).
>
> I will send out a poll to find a good time for everyone for a kickoff
> call. Our intent is to build a documentation team with multiple writers and
> mentors interacting and able to help each other out. And all of this will
> also interact with the numpy.org website redesign and the people putting
> energy into that:)
>
> I'm very happy to welcome Anne, Maja and Brandon!
>
> Cheers,
> Ralf
>
>
> [1] https://developers.google.com/season-of-docs/docs/participants/
> [2]
> https://developers.google.com/season-of-docs/docs/participants/project-numpy
> [3]
> https://developers.google.com/season-of-docs/docs/participants/project-scipy
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/830295cd/attachment.html>

From andyfaff at gmail.com  Tue Aug  6 21:38:29 2019
From: andyfaff at gmail.com (Andrew Nelson)
Date: Wed, 7 Aug 2019 11:38:29 +1000
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
In-Reply-To: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
Message-ID: <CAAbtOZePo98sVBukaXLPf9ySy9gcdRp8v=1HmG7dXZV=May29Q@mail.gmail.com>

Welcome Anne, Maja, Brandon!

On Wed, 7 Aug 2019 at 09:47, Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Google has announced the Season of Docs participants for this year [1]. We
> had a lot of excellent candidates and had to make some hard choices. We
> applied for extra slots, but unfortunately didn't win the lottery for
> those; we got one slot for NumPy and one for SciPy. We chose the projects
> of Anne for NumPy and Maja for SciPy:
>
> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
> Introductory NumPy Sections" [2]
>
> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>
> That's not all though. There was some space left in the budget of the
> NumPy BIDS grant, and St?fan has reserved that so we can accept more
> writers and provide them the same mentoring and funding as they would have
> gotten through GSoD. We could only start the conversations about that once
> Google made its decisions, so a further announcement will follow. However,
> we already have one extra project confirmed, from Brandon:
>
> Brandon David, "Improve the documentation of scipy.stats" (project details
> to be published).
>
> I will send out a poll to find a good time for everyone for a kickoff
> call. Our intent is to build a documentation team with multiple writers and
> mentors interacting and able to help each other out. And all of this will
> also interact with the numpy.org website redesign and the people putting
> energy into that:)
>
> I'm very happy to welcome Anne, Maja and Brandon!
>
> Cheers,
> Ralf
>
>
> [1] https://developers.google.com/season-of-docs/docs/participants/
> [2]
> https://developers.google.com/season-of-docs/docs/participants/project-numpy
> [3]
> https://developers.google.com/season-of-docs/docs/participants/project-scipy
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>


-- 
_____________________________________
Dr. Andrew Nelson


_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/06b9c58f/attachment-0001.html>

From anne at contentsimplicity.com  Tue Aug  6 22:13:25 2019
From: anne at contentsimplicity.com (Anne Bonner)
Date: Tue, 6 Aug 2019 19:13:25 -0700
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
In-Reply-To: <CAAbtOZePo98sVBukaXLPf9ySy9gcdRp8v=1HmG7dXZV=May29Q@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
 <CAAbtOZePo98sVBukaXLPf9ySy9gcdRp8v=1HmG7dXZV=May29Q@mail.gmail.com>
Message-ID: <CADSfPOS5euH2Bv==mXFK4EFm9YRpN4ZGF6K5=e8SCQdotMw0dA@mail.gmail.com>

Thank you so much!!!

I'm so excited to be a part of what you guys have created and built!

All the best,
Anne

On Tue, Aug 6, 2019 at 6:39 PM Andrew Nelson <andyfaff at gmail.com> wrote:

> Welcome Anne, Maja, Brandon!
>
> On Wed, 7 Aug 2019 at 09:47, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>> Hi all,
>>
>> Google has announced the Season of Docs participants for this year [1].
>> We had a lot of excellent candidates and had to make some hard choices. We
>> applied for extra slots, but unfortunately didn't win the lottery for
>> those; we got one slot for NumPy and one for SciPy. We chose the projects
>> of Anne for NumPy and Maja for SciPy:
>>
>> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
>> Introductory NumPy Sections" [2]
>>
>> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>>
>> That's not all though. There was some space left in the budget of the
>> NumPy BIDS grant, and St?fan has reserved that so we can accept more
>> writers and provide them the same mentoring and funding as they would have
>> gotten through GSoD. We could only start the conversations about that once
>> Google made its decisions, so a further announcement will follow. However,
>> we already have one extra project confirmed, from Brandon:
>>
>> Brandon David, "Improve the documentation of scipy.stats" (project
>> details to be published).
>>
>> I will send out a poll to find a good time for everyone for a kickoff
>> call. Our intent is to build a documentation team with multiple writers and
>> mentors interacting and able to help each other out. And all of this will
>> also interact with the numpy.org website redesign and the people putting
>> energy into that:)
>>
>> I'm very happy to welcome Anne, Maja and Brandon!
>>
>> Cheers,
>> Ralf
>>
>>
>> [1] https://developers.google.com/season-of-docs/docs/participants/
>> [2]
>> https://developers.google.com/season-of-docs/docs/participants/project-numpy
>> [3]
>> https://developers.google.com/season-of-docs/docs/participants/project-scipy
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
>
> --
> _____________________________________
> Dr. Andrew Nelson
>
>
> _____________________________________
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>


-- 
*Anne Bonner | **Content Simplicity <https://contentsimplicity.com/>
| **Making
complicated concepts seem simple.*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/00964c0f/attachment.html>

From charlesr.harris at gmail.com  Tue Aug  6 23:03:13 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 6 Aug 2019 21:03:13 -0600
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
In-Reply-To: <CADSfPOS5euH2Bv==mXFK4EFm9YRpN4ZGF6K5=e8SCQdotMw0dA@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
 <CAAbtOZePo98sVBukaXLPf9ySy9gcdRp8v=1HmG7dXZV=May29Q@mail.gmail.com>
 <CADSfPOS5euH2Bv==mXFK4EFm9YRpN4ZGF6K5=e8SCQdotMw0dA@mail.gmail.com>
Message-ID: <CAB6mnxJDW+wsGJyqAs3_i5gFqPjAJJDRb-urfjFYKTqgkQg0Ug@mail.gmail.com>

On Tue, Aug 6, 2019 at 8:14 PM Anne Bonner <anne at contentsimplicity.com>
wrote:

> Thank you so much!!!
>
> I'm so excited to be a part of what you guys have created and built!
>
> All the best,
> Anne
>
> On Tue, Aug 6, 2019 at 6:39 PM Andrew Nelson <andyfaff at gmail.com> wrote:
>
>> Welcome Anne, Maja, Brandon!
>>
>> On Wed, 7 Aug 2019 at 09:47, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> Google has announced the Season of Docs participants for this year [1].
>>> We had a lot of excellent candidates and had to make some hard choices. We
>>> applied for extra slots, but unfortunately didn't win the lottery for
>>> those; we got one slot for NumPy and one for SciPy. We chose the projects
>>> of Anne for NumPy and Maja for SciPy:
>>>
>>> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
>>> Introductory NumPy Sections" [2]
>>>
>>> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>>>
>>> That's not all though. There was some space left in the budget of the
>>> NumPy BIDS grant, and St?fan has reserved that so we can accept more
>>> writers and provide them the same mentoring and funding as they would have
>>> gotten through GSoD. We could only start the conversations about that once
>>> Google made its decisions, so a further announcement will follow. However,
>>> we already have one extra project confirmed, from Brandon:
>>>
>>> Brandon David, "Improve the documentation of scipy.stats" (project
>>> details to be published).
>>>
>>> I will send out a poll to find a good time for everyone for a kickoff
>>> call. Our intent is to build a documentation team with multiple writers and
>>> mentors interacting and able to help each other out. And all of this will
>>> also interact with the numpy.org website redesign and the people
>>> putting energy into that:)
>>>
>>> I'm very happy to welcome Anne, Maja and Brandon!
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>> [1] https://developers.google.com/season-of-docs/docs/participants/
>>> [2]
>>> https://developers.google.com/season-of-docs/docs/participants/project-numpy
>>> [3]
>>> https://developers.google.com/season-of-docs/docs/participants/project-scipy
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>>
>>
>>
Welcome aboard.

Chuck

> --
>> _____________________________________
>> Dr. Andrew Nelson
>>
>>
>> _____________________________________
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
>
> --
> *Anne Bonner | **Content Simplicity <https://contentsimplicity.com/>  | **Making
> complicated concepts seem simple.*
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190806/a7257d3e/attachment.html>

From ralf.gommers at gmail.com  Wed Aug  7 20:10:51 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 7 Aug 2019 17:10:51 -0700
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
Message-ID: <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>

On Mon, Aug 5, 2019 at 6:18 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>
>> The NEP currently does not say who this is meant for. Would you expect
>> libraries like SciPy to adopt it for example?
>>
>> The NEP also (understandably) punts on the question of when something is
>> a valid duck array. If you want this to be widely used, that will need an
>> answer or at least some rough guidance though. For example, we would expect
>> a duck array to have a mean() method, but probably not a ptp() method. A
>> library author who wants to use np.duckarray() needs to know, because she
>> can't test with all existing and future duck array implementations.
>>
>
> I think this is covered in NEP-22 already.
>

It's not really. We discussed this briefly in the community call today,
Peter said he will try to add some text.

We should not add new functions to NumPy without indicating who is supposed
to use this, and what need it fills / problem it solves. It seems pretty
clear to me that it's mostly aimed at library authors rather than end
users. And also that mature libraries like SciPy may not immediately adopt
it, because it's too fuzzy - so it's new libraries first, mature libraries
after the dust has settled a bit (I think).

As discussed there, I don't think NumPy is in a good position to pronounce
> decisive APIs at this time. I would welcome efforts to try, but I don't
> think that's essential for now.
>

There's no need to pronounce a decisive API that fully covers duck array.
Note that RNumPy is an attempt in that direction (not a full one, but way
better than nothing). In the NEP/docs, at least saying something along the
lines of "if you implement this, we recommend the following strategy: check
if a function is present in Dask, CuPy and Sparse. If so, it's reasonable
to expect any duck array to work here. If not, we suggest you indicate in
your docstring what kinds of duck arrays are accepted, or what properties
they need to have". That's a spec by implementation, which is less than
ideal but better than saying nothing.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/483f141f/attachment.html>

From ralf.gommers at gmail.com  Wed Aug  7 21:03:15 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 7 Aug 2019 18:03:15 -0700
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
In-Reply-To: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
Message-ID: <CABL7CQijUi-7+jAkxXtDdvw-k__CJBm_3L2KzHjC=ayLoGd3_w@mail.gmail.com>

On Tue, Aug 6, 2019 at 4:46 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
>
> Google has announced the Season of Docs participants for this year [1]. We
> had a lot of excellent candidates and had to make some hard choices. We
> applied for extra slots, but unfortunately didn't win the lottery for
> those; we got one slot for NumPy and one for SciPy. We chose the projects
> of Anne for NumPy and Maja for SciPy:
>
> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
> Introductory NumPy Sections" [2]
>
> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>
> That's not all though. There was some space left in the budget of the
> NumPy BIDS grant, and St?fan has reserved that so we can accept more
> writers and provide them the same mentoring and funding as they would have
> gotten through GSoD. We could only start the conversations about that once
> Google made its decisions, so a further announcement will follow. However,
> we already have one extra project confirmed, from Brandon:
>
> Brandon David, "Improve the documentation of scipy.stats" (project details
> to be published).
>

Happy to announce that we have a fourth participant:

Shekhar Rajak, "numpy.org redesign and high level documentation
restructuring for end user focus"

Welcome Shekhar!

I will send out a poll to find a good time for everyone for a kickoff call.
> Our intent is to build a documentation team with multiple writers and
> mentors interacting and able to help each other out. And all of this will
> also interact with the numpy.org website redesign and the people putting
> energy into that:)
>

Here is the poll link: https://doodle.com/poll/skgbk74gsg8zpziu. I hope we
can find a time that works for everyone - we're split over all US
timezones, Europe and India. So it's going to be early morning or late
evening somewhere.

Sending this out in public, so anyone who wants to participate is welcome
to join. I've Bcc'd all participants and mentors, to make sure they see
this.

Cheers,
Ralf



>
> I'm very happy to welcome Anne, Maja and Brandon!
>
> Cheers,
> Ralf
>
>
> [1] https://developers.google.com/season-of-docs/docs/participants/
> [2]
> https://developers.google.com/season-of-docs/docs/participants/project-numpy
> [3]
> https://developers.google.com/season-of-docs/docs/participants/project-scipy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/3614df9b/attachment.html>

From shoyer at gmail.com  Wed Aug  7 21:09:58 2019
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 7 Aug 2019 18:09:58 -0700
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>
Message-ID: <CAEQ_TvfgtKveBr8TDrnUXz_vw6-+h1n19ot+dddVd67pM8a-JA@mail.gmail.com>

On Wed, Aug 7, 2019 at 5:11 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
> On Mon, Aug 5, 2019 at 6:18 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>
>>> The NEP currently does not say who this is meant for. Would you expect
>>> libraries like SciPy to adopt it for example?
>>>
>>> The NEP also (understandably) punts on the question of when something is
>>> a valid duck array. If you want this to be widely used, that will need an
>>> answer or at least some rough guidance though. For example, we would expect
>>> a duck array to have a mean() method, but probably not a ptp() method. A
>>> library author who wants to use np.duckarray() needs to know, because she
>>> can't test with all existing and future duck array implementations.
>>>
>>
>> I think this is covered in NEP-22 already.
>>
>
> It's not really. We discussed this briefly in the community call today,
> Peter said he will try to add some text.
>
> We should not add new functions to NumPy without indicating who is
> supposed to use this, and what need it fills / problem it solves. It seems
> pretty clear to me that it's mostly aimed at library authors rather than
> end users. And also that mature libraries like SciPy may not immediately
> adopt it, because it's too fuzzy - so it's new libraries first, mature
> libraries after the dust has settled a bit (I think).
>

I totally agree -- we definitely should clarify this in the docstring and
elsewhere in the docs. An example in the new doc page on "Writing custom
array containers" (https://numpy.org/devdocs/user/basics.dispatch.html)
would also probably be appropriate.


> As discussed there, I don't think NumPy is in a good position to pronounce
>> decisive APIs at this time. I would welcome efforts to try, but I don't
>> think that's essential for now.
>>
>
> There's no need to pronounce a decisive API that fully covers duck array.
> Note that RNumPy is an attempt in that direction (not a full one, but way
> better than nothing). In the NEP/docs, at least saying something along the
> lines of "if you implement this, we recommend the following strategy: check
> if a function is present in Dask, CuPy and Sparse. If so, it's reasonable
> to expect any duck array to work here. If not, we suggest you indicate in
> your docstring what kinds of duck arrays are accepted, or what properties
> they need to have". That's a spec by implementation, which is less than
> ideal but better than saying nothing.
>

OK, I agree here as well -- some guidance is better than nothing.

Two other minor notes on this NEP, concerning naming:
1. We should have a brief note on why we settled on the name "duck array".
Namely, as discussed in NEP-22, we don't love the "duck" jargon, but we
couldn't come up with anything better since NumPy already uses "array like"
and "any array" for different purposes.
2. The protocol should use *something* more clearly namespaced as NumPy
specific than __duckarray__. All the other special protocols NumPy defines
start with "__array_". That suggests either __array_duckarray__ (sounds a
little redundant) or __numpy_duckarray__ (which I like the look of, but is
a different from the existing protocols).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/8ba50c59/attachment-0001.html>

From charlesr.harris at gmail.com  Wed Aug  7 21:18:27 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 7 Aug 2019 19:18:27 -0600
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CAEQ_TvfgtKveBr8TDrnUXz_vw6-+h1n19ot+dddVd67pM8a-JA@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>
 <CAEQ_TvfgtKveBr8TDrnUXz_vw6-+h1n19ot+dddVd67pM8a-JA@mail.gmail.com>
Message-ID: <CAB6mnxLruM1wm=UdVOa14UOJOGPFD7PZGQK_haNrDDgCk2B+hA@mail.gmail.com>

On Wed, Aug 7, 2019 at 7:10 PM Stephan Hoyer <shoyer at gmail.com> wrote:

> On Wed, Aug 7, 2019 at 5:11 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>> On Mon, Aug 5, 2019 at 6:18 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>>
>>> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>>
>>>
>>>> The NEP currently does not say who this is meant for. Would you expect
>>>> libraries like SciPy to adopt it for example?
>>>>
>>>> The NEP also (understandably) punts on the question of when something
>>>> is a valid duck array. If you want this to be widely used, that will need
>>>> an answer or at least some rough guidance though. For example, we would
>>>> expect a duck array to have a mean() method, but probably not a ptp()
>>>> method. A library author who wants to use np.duckarray() needs to know,
>>>> because she can't test with all existing and future duck array
>>>> implementations.
>>>>
>>>
>>> I think this is covered in NEP-22 already.
>>>
>>
>> It's not really. We discussed this briefly in the community call today,
>> Peter said he will try to add some text.
>>
>> We should not add new functions to NumPy without indicating who is
>> supposed to use this, and what need it fills / problem it solves. It seems
>> pretty clear to me that it's mostly aimed at library authors rather than
>> end users. And also that mature libraries like SciPy may not immediately
>> adopt it, because it's too fuzzy - so it's new libraries first, mature
>> libraries after the dust has settled a bit (I think).
>>
>
> I totally agree -- we definitely should clarify this in the docstring and
> elsewhere in the docs. An example in the new doc page on "Writing custom
> array containers" (https://numpy.org/devdocs/user/basics.dispatch.html)
> would also probably be appropriate.
>
>
>> As discussed there, I don't think NumPy is in a good position to
>>> pronounce decisive APIs at this time. I would welcome efforts to try, but I
>>> don't think that's essential for now.
>>>
>>
>> There's no need to pronounce a decisive API that fully covers duck array.
>> Note that RNumPy is an attempt in that direction (not a full one, but way
>> better than nothing). In the NEP/docs, at least saying something along the
>> lines of "if you implement this, we recommend the following strategy: check
>> if a function is present in Dask, CuPy and Sparse. If so, it's reasonable
>> to expect any duck array to work here. If not, we suggest you indicate in
>> your docstring what kinds of duck arrays are accepted, or what properties
>> they need to have". That's a spec by implementation, which is less than
>> ideal but better than saying nothing.
>>
>
> OK, I agree here as well -- some guidance is better than nothing.
>
> Two other minor notes on this NEP, concerning naming:
> 1. We should have a brief note on why we settled on the name "duck array".
> Namely, as discussed in NEP-22, we don't love the "duck" jargon, but we
> couldn't come up with anything better since NumPy already uses "array like"
> and "any array" for different purposes.
> 2. The protocol should use *something* more clearly namespaced as NumPy
> specific than __duckarray__. All the other special protocols NumPy defines
> start with "__array_". That suggests either __array_duckarray__ (sounds a
> little redundant) or __numpy_duckarray__ (which I like the look of, but is
> a different from the existing protocols).
>
>
`__numpy_like__` ?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/d1d557bd/attachment.html>

From shoyer at gmail.com  Wed Aug  7 21:40:46 2019
From: shoyer at gmail.com (Stephan Hoyer)
Date: Wed, 7 Aug 2019 18:40:46 -0700
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CAB6mnxLruM1wm=UdVOa14UOJOGPFD7PZGQK_haNrDDgCk2B+hA@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>
 <CAEQ_TvfgtKveBr8TDrnUXz_vw6-+h1n19ot+dddVd67pM8a-JA@mail.gmail.com>
 <CAB6mnxLruM1wm=UdVOa14UOJOGPFD7PZGQK_haNrDDgCk2B+hA@mail.gmail.com>
Message-ID: <CAEQ_TvdMn+iU8Qt+0PE79U61qDa6aO3c0U4n-c_1Xjth+fAJ6Q@mail.gmail.com>

On Wed, Aug 7, 2019 at 6:18 PM Charles R Harris <charlesr.harris at gmail.com>
wrote:

>
>
> On Wed, Aug 7, 2019 at 7:10 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> On Wed, Aug 7, 2019 at 5:11 PM Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>> On Mon, Aug 5, 2019 at 6:18 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>>>
>>>> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>> The NEP currently does not say who this is meant for. Would you expect
>>>>> libraries like SciPy to adopt it for example?
>>>>>
>>>>> The NEP also (understandably) punts on the question of when something
>>>>> is a valid duck array. If you want this to be widely used, that will need
>>>>> an answer or at least some rough guidance though. For example, we would
>>>>> expect a duck array to have a mean() method, but probably not a ptp()
>>>>> method. A library author who wants to use np.duckarray() needs to know,
>>>>> because she can't test with all existing and future duck array
>>>>> implementations.
>>>>>
>>>>
>>>> I think this is covered in NEP-22 already.
>>>>
>>>
>>> It's not really. We discussed this briefly in the community call today,
>>> Peter said he will try to add some text.
>>>
>>> We should not add new functions to NumPy without indicating who is
>>> supposed to use this, and what need it fills / problem it solves. It seems
>>> pretty clear to me that it's mostly aimed at library authors rather than
>>> end users. And also that mature libraries like SciPy may not immediately
>>> adopt it, because it's too fuzzy - so it's new libraries first, mature
>>> libraries after the dust has settled a bit (I think).
>>>
>>
>> I totally agree -- we definitely should clarify this in the docstring and
>> elsewhere in the docs. An example in the new doc page on "Writing custom
>> array containers" (https://numpy.org/devdocs/user/basics.dispatch.html)
>> would also probably be appropriate.
>>
>>
>>> As discussed there, I don't think NumPy is in a good position to
>>>> pronounce decisive APIs at this time. I would welcome efforts to try, but I
>>>> don't think that's essential for now.
>>>>
>>>
>>> There's no need to pronounce a decisive API that fully covers duck
>>> array. Note that RNumPy is an attempt in that direction (not a full one,
>>> but way better than nothing). In the NEP/docs, at least saying something
>>> along the lines of "if you implement this, we recommend the following
>>> strategy: check if a function is present in Dask, CuPy and Sparse. If so,
>>> it's reasonable to expect any duck array to work here. If not, we suggest
>>> you indicate in your docstring what kinds of duck arrays are accepted, or
>>> what properties they need to have". That's a spec by implementation, which
>>> is less than ideal but better than saying nothing.
>>>
>>
>> OK, I agree here as well -- some guidance is better than nothing.
>>
>> Two other minor notes on this NEP, concerning naming:
>> 1. We should have a brief note on why we settled on the name "duck
>> array". Namely, as discussed in NEP-22, we don't love the "duck" jargon,
>> but we couldn't come up with anything better since NumPy already uses
>> "array like" and "any array" for different purposes.
>> 2. The protocol should use *something* more clearly namespaced as NumPy
>> specific than __duckarray__. All the other special protocols NumPy defines
>> start with "__array_". That suggests either __array_duckarray__ (sounds a
>> little redundant) or __numpy_duckarray__ (which I like the look of, but is
>> a different from the existing protocols).
>>
>>
> `__numpy_like__` ?
>


This could work, but I think we would also want to rename the NumPy
function itself to either np.like or np.numpy_like. The later is a little
redundant but definitely more self-descriptive than "duck array".


> Chuck
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190807/782778ee/attachment-0001.html>

From bennet at umich.edu  Thu Aug  8 08:37:19 2019
From: bennet at umich.edu (Bennet Fauber)
Date: Thu, 8 Aug 2019 08:37:19 -0400
Subject: [Numpy-discussion] [SciPy-Dev] Season of Docs - welcome Anne,
 Maja, Brandon
In-Reply-To: <mailman.3735.1565266858.29304.numpy-discussion@python.org>
References: <mailman.3735.1565266858.29304.numpy-discussion@python.org>
Message-ID: <CAB2ovoupB3Hc9rR1G21upBGhUzoTqU75o0=Jda0nT1JiwmJZYQ@mail.gmail.com>

I can offer some time for alpha or beta reading and proofreading.

I have experience as a proofreader, copy editor (American Statistical
Association), and mathematical typesetter (Wiley, Academic Press,
Addison-Wesley, et al.).  I've taught statistical software workshops,
(very) introductory python, have a Software Carpentry instructor
certificate, and work daily with people who are finding themselves
needing technical and scientific computing but who don't have strong
backgrounds.  Hopefully that would be good context for early review of
a couple of the projects.

I will add my name to the poll, if that's OK?

-- bennet

On Wed, Aug 7, 2019 at 9:03 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Tue, Aug 6, 2019 at 4:46 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>> Hi all,
>>
>> Google has announced the Season of Docs participants for this year [1]. We had a lot of excellent candidates and had to make some hard choices. We applied for extra slots, but unfortunately didn't win the lottery for those; we got one slot for NumPy and one for SciPy. We chose the projects of Anne for NumPy and Maja for SciPy:
>>
>> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the Introductory NumPy Sections" [2]
>>
>> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>>
>> That's not all though. There was some space left in the budget of the NumPy BIDS grant, and St?fan has reserved that so we can accept more writers and provide them the same mentoring and funding as they would have gotten through GSoD. We could only start the conversations about that once Google made its decisions, so a further announcement will follow. However, we already have one extra project confirmed, from Brandon:
>>
>> Brandon David, "Improve the documentation of scipy.stats" (project details to be published).
>
>
> Happy to announce that we have a fourth participant:
>
> Shekhar Rajak, "numpy.org redesign and high level documentation restructuring for end user focus"
>
> Welcome Shekhar!
>
>> I will send out a poll to find a good time for everyone for a kickoff call. Our intent is to build a documentation team with multiple writers and mentors interacting and able to help each other out. And all of this will also interact with the numpy.org website redesign and the people putting energy into that:)
>
>
> Here is the poll link: https://doodle.com/poll/skgbk74gsg8zpziu. I hope we can find a time that works for everyone - we're split over all US timezones, Europe and India. So it's going to be early morning or late evening somewhere.
>
> Sending this out in public, so anyone who wants to participate is welcome to join. I've Bcc'd all participants and mentors, to make sure they see this.
>
> Cheers,
> Ralf
>
>
>>
>>
>> I'm very happy to welcome Anne, Maja and Brandon!
>>
>> Cheers,
>> Ralf
>>
>>
>> [1] https://developers.google.com/season-of-docs/docs/participants/
>> [2] https://developers.google.com/season-of-docs/docs/participants/project-numpy
>> [3] https://developers.google.com/season-of-docs/docs/participants/project-scipy
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev

From matti.picus at gmail.com  Thu Aug  8 09:23:54 2019
From: matti.picus at gmail.com (Matti Picus)
Date: Thu, 8 Aug 2019 16:23:54 +0300
Subject: [Numpy-discussion] Reformat the wall-of-text release notes
Message-ID: <a9872145-3e08-2e16-b33a-e0748c48dc7e@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/7c938eca/attachment.html>

From ralf.gommers at gmail.com  Thu Aug  8 13:07:26 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 8 Aug 2019 10:07:26 -0700
Subject: [Numpy-discussion] [SciPy-Dev] Season of Docs - welcome Anne,
 Maja, Brandon
In-Reply-To: <CAB2ovoupB3Hc9rR1G21upBGhUzoTqU75o0=Jda0nT1JiwmJZYQ@mail.gmail.com>
References: <mailman.3735.1565266858.29304.numpy-discussion@python.org>
 <CAB2ovoupB3Hc9rR1G21upBGhUzoTqU75o0=Jda0nT1JiwmJZYQ@mail.gmail.com>
Message-ID: <CABL7CQjy7rvvyW37eMkn-8vqYgJVO5PWs2nXoSc7RdREAp_vxQ@mail.gmail.com>

On Thu, Aug 8, 2019 at 5:37 AM Bennet Fauber <bennet at umich.edu> wrote:

> I can offer some time for alpha or beta reading and proofreading.
>
> I have experience as a proofreader, copy editor (American Statistical
> Association), and mathematical typesetter (Wiley, Academic Press,
> Addison-Wesley, et al.).  I've taught statistical software workshops,
> (very) introductory python, have a Software Carpentry instructor
> certificate, and work daily with people who are finding themselves
> needing technical and scientific computing but who don't have strong
> backgrounds.  Hopefully that would be good context for early review of
> a couple of the projects.
>

Awesome, thanks Bennet!


> I will add my name to the poll, if that's OK?
>

Yes definitely!

Cheers,
Ralf


> -- bennet
>
> On Wed, Aug 7, 2019 at 9:03 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Tue, Aug 6, 2019 at 4:46 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >>
> >> Hi all,
> >>
> >> Google has announced the Season of Docs participants for this year [1].
> We had a lot of excellent candidates and had to make some hard choices. We
> applied for extra slots, but unfortunately didn't win the lottery for
> those; we got one slot for NumPy and one for SciPy. We chose the projects
> of Anne for NumPy and Maja for SciPy:
> >>
> >> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
> Introductory NumPy Sections" [2]
> >>
> >> Maja Gwozdz, "User-oriented documentation and thorough restructuring"
> [3]
> >>
> >> That's not all though. There was some space left in the budget of the
> NumPy BIDS grant, and St?fan has reserved that so we can accept more
> writers and provide them the same mentoring and funding as they would have
> gotten through GSoD. We could only start the conversations about that once
> Google made its decisions, so a further announcement will follow. However,
> we already have one extra project confirmed, from Brandon:
> >>
> >> Brandon David, "Improve the documentation of scipy.stats" (project
> details to be published).
> >
> >
> > Happy to announce that we have a fourth participant:
> >
> > Shekhar Rajak, "numpy.org redesign and high level documentation
> restructuring for end user focus"
> >
> > Welcome Shekhar!
> >
> >> I will send out a poll to find a good time for everyone for a kickoff
> call. Our intent is to build a documentation team with multiple writers and
> mentors interacting and able to help each other out. And all of this will
> also interact with the numpy.org website redesign and the people putting
> energy into that:)
> >
> >
> > Here is the poll link: https://doodle.com/poll/skgbk74gsg8zpziu. I hope
> we can find a time that works for everyone - we're split over all US
> timezones, Europe and India. So it's going to be early morning or late
> evening somewhere.
> >
> > Sending this out in public, so anyone who wants to participate is
> welcome to join. I've Bcc'd all participants and mentors, to make sure they
> see this.
> >
> > Cheers,
> > Ralf
> >
> >
> >>
> >>
> >> I'm very happy to welcome Anne, Maja and Brandon!
> >>
> >> Cheers,
> >> Ralf
> >>
> >>
> >> [1] https://developers.google.com/season-of-docs/docs/participants/
> >> [2]
> https://developers.google.com/season-of-docs/docs/participants/project-numpy
> >> [3]
> https://developers.google.com/season-of-docs/docs/participants/project-scipy
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/81e88f48/attachment.html>

From ralf.gommers at gmail.com  Thu Aug  8 21:20:58 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 8 Aug 2019 18:20:58 -0700
Subject: [Numpy-discussion] Reformat the wall-of-text release notes
In-Reply-To: <a9872145-3e08-2e16-b33a-e0748c48dc7e@gmail.com>
References: <a9872145-3e08-2e16-b33a-e0748c48dc7e@gmail.com>
Message-ID: <CABL7CQh2ozVV+fPJh9bg1L9r_-NWH5RuDywNf0bf5wR_KhU-fg@mail.gmail.com>

On Thu, Aug 8, 2019 at 6:24 AM Matti Picus <matti.picus at gmail.com> wrote:

> Our Release Note page https://numpy.org/devdocs/release.html is one long
> litany of all the releases ever. While this is convenient for searching
> with CTRL-F, it is not conducive to browsing. I suggested splitting it into
> pages with a TOC page
> https://8001-908607-gh.circle-artifacts.com/0/home/circleci/repo/doc/build/html/release.html
> .
>
>
> In the pull request https://github.com/numpy/numpy/pull/13886 there was a
> question about which level of detail to put into the TOC. Too much and we
> are back where we started. Too little makes CTRL-F searching useless.
>

I quite like how your current version looks.

Cheers,
Ralf


> Thoughts?
>
>
> Matti
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/a9c08566/attachment.html>

From ralf.gommers at gmail.com  Thu Aug  8 21:45:04 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 8 Aug 2019 18:45:04 -0700
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon
In-Reply-To: <CABL7CQijUi-7+jAkxXtDdvw-k__CJBm_3L2KzHjC=ayLoGd3_w@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
 <CABL7CQijUi-7+jAkxXtDdvw-k__CJBm_3L2KzHjC=ayLoGd3_w@mail.gmail.com>
Message-ID: <CABL7CQgDWZ4mho6x_DETC=ZEKCC1Zwt3eX+gqhNZqg5ngFdJYQ@mail.gmail.com>

On Wed, Aug 7, 2019 at 6:03 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Tue, Aug 6, 2019 at 4:46 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
> I will send out a poll to find a good time for everyone for a kickoff
>> call. Our intent is to build a documentation team with multiple writers and
>> mentors interacting and able to help each other out. And all of this will
>> also interact with the numpy.org website redesign and the people putting
>> energy into that:)
>>
>
> Here is the poll link: https://doodle.com/poll/skgbk74gsg8zpziu. I hope
> we can find a time that works for everyone - we're split over all US
> timezones, Europe and India. So it's going to be early morning or late
> evening somewhere.
>
> Sending this out in public, so anyone who wants to participate is welcome
> to join. I've Bcc'd all participants and mentors, to make sure they see
> this.
>

That worked out pretty well; all participants, mentors and a few more
people can make the meeting on 13 Aug at 3pm UTC. Sent out an invite, and
meeting notes doc with Hangouts link can be found at
https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/92ee5d99/attachment-0001.html>

From tyler.je.reddy at gmail.com  Fri Aug  9 00:06:38 2019
From: tyler.je.reddy at gmail.com (Tyler Reddy)
Date: Thu, 8 Aug 2019 22:06:38 -0600
Subject: [Numpy-discussion] ANN: SciPy 1.3.1
Message-ID: <CAHPuU_ZUzUZsNs+9Y256RB1BgM6hhf7Pohxd9OM4ckbkv2dfUw@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi all,

On behalf of the SciPy development team I'm pleased to announce
the release of SciPy 1.3.1, which is a bug fix release.

Sources and binary wheels can be found at:
https://pypi.org/project/scipy/
and at: https://github.com/scipy/scipy/releases/tag/v1.3.1

One of a few ways to install this release with pip:

pip install scipy==1.3.1

==========================
SciPy 1.3.1 Release Notes
==========================

SciPy 1.3.1 is a bug-fix release with no new features
compared to 1.3.0.

Authors
=======

* Matt Haberland
* Geordie McBain
* Yu Feng
* Evgeni Burovski
* Sturla Molden
* Tapasweni Pathak
* Eric Larson
* Peter Bell
* Carlos Ramos Carre?o +
* Ralf Gommers
* David Hagen
* Antony Lee
* Ayappan P
* Tyler Reddy
* Pauli Virtanen

A total of 15 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully
complete.

Issues closed for 1.3.1
-------------------------------

* `#5040 <https://github.com/scipy/scipy/issues/5040>`__: BUG: Empty data
handling of (c)KDTrees
* `#9901 <https://github.com/scipy/scipy/issues/9901>`__: lsoda fails to
detect stiff problem when called from solve_ivp
* `#10206 <https://github.com/scipy/scipy/issues/10206>`__: sparse matrices
indexing with scipy 1.3
* `#10232 <https://github.com/scipy/scipy/issues/10232>`__: Exception in
loadarff with quoted nominal attributes in scipy...
* `#10292 <https://github.com/scipy/scipy/issues/10292>`__: DOC/REL: Some
sections of the release notes are not nested correctly.
* `#10303 <https://github.com/scipy/scipy/issues/10303>`__: BUG: optimize:
`linprog` failing
TestLinprogSimplexBland::test_unbounded_below_no_presolve_corrected
* `#10376 <https://github.com/scipy/scipy/issues/10376>`__: TST: Travis CI
fails (with pytest 5.0 ?)
* `#10384 <https://github.com/scipy/scipy/issues/10384>`__: CircleCI doc
build failing on new warnings
* `#10398 <https://github.com/scipy/scipy/issues/10398>`__: Scipy 1.3.0
build broken in AIX
* `#10501 <https://github.com/scipy/scipy/issues/10501>`__: BUG:
scipy.spatial.HalfspaceIntersection works incorrectly
* `#10514 <https://github.com/scipy/scipy/issues/10514>`__: BUG: cKDTree
GIL handling is incorrect
* `#10535 <https://github.com/scipy/scipy/issues/10535>`__: TST: master
branch CI failures
* `#10572 <https://github.com/scipy/scipy/issues/10572>`__: BUG: ckdtree
query_ball_point errors on discontiguous input
* `#10597 <https://github.com/scipy/scipy/issues/10597>`__: BUG: No warning
on PchipInterpolator changing from bernstein base to local power base

Pull requests for 1.3.1
------------------------------

* `#10071 <https://github.com/scipy/scipy/pull/10071>`__: DOC: reconstruct
SuperLU permutation matrices avoiding SparseEfficiencyWarning
* `#10196 <https://github.com/scipy/scipy/pull/10196>`__: Fewer checks on
xdata for curve_fit.
* `#10207 <https://github.com/scipy/scipy/pull/10207>`__: BUG: Compressed
matrix indexing should return a scalar
* `#10233 <https://github.com/scipy/scipy/pull/10233>`__: Fix for ARFF
reader regression (#10232)
* `#10306 <https://github.com/scipy/scipy/pull/10306>`__: BUG: optimize:
Fix for 10303
* `#10309 <https://github.com/scipy/scipy/pull/10309>`__: BUG: Pass
jac=None directly to lsoda
* `#10377 <https://github.com/scipy/scipy/pull/10377>`__: TST, MAINT:
adjustments for pytest 5.0
* `#10379 <https://github.com/scipy/scipy/pull/10379>`__: BUG: sparse: set
writeability to be forward-compatible with numpy>=1.17
* `#10426 <https://github.com/scipy/scipy/pull/10426>`__: MAINT: Fix doc
build bugs
* `#10431 <https://github.com/scipy/scipy/pull/10431>`__: Update numpy
version for AIX
* `#10457 <https://github.com/scipy/scipy/pull/10457>`__: BUG: Allow
ckdtree to accept empty data input
* `#10503 <https://github.com/scipy/scipy/pull/10503>`__: BUG:
spatial/qhull: get HalfspaceIntersection.dual_points from the correct array
* `#10516 <https://github.com/scipy/scipy/pull/10516>`__: BUG: Use nogil
contexts in cKDTree
* `#10520 <https://github.com/scipy/scipy/pull/10520>`__: DOC: Proper .rst
formatting for deprecated features and Backwards incompatible changes
* `#10540 <https://github.com/scipy/scipy/pull/10540>`__: MAINT: Fix Travis
and Circle
* `#10573 <https://github.com/scipy/scipy/pull/10573>`__: BUG: Fix
query_ball_point with discontiguous input
* `#10600 <https://github.com/scipy/scipy/pull/10600>`__: BUG: interpolate:
fix broken conversions between PPoly/BPoly objects

Checksums
=========

MD5
~~~

818dc6325a4511d656582ff2946eed80
 scipy-1.3.1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
83a11b3127b19e71353ee2a04c4be20c  scipy-1.3.1-cp35-cp35m-manylinux1_i686.whl
49c52e00706b47b7311171fe37b9efac
 scipy-1.3.1-cp35-cp35m-manylinux1_x86_64.whl
015b3d443e2a9e4e664a50af64a7f5b6  scipy-1.3.1-cp35-cp35m-win32.whl
43cf62be72bf7b8e42a1a0fad6570e22  scipy-1.3.1-cp35-cp35m-win_amd64.whl
08d697cdeeb2a4121bbeca8d8d756da9
 scipy-1.3.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
25f6364aa052213d4f504bc96031e431  scipy-1.3.1-cp36-cp36m-manylinux1_i686.whl
2fb8c8c5c17dd7d811165b59d070ef4a
 scipy-1.3.1-cp36-cp36m-manylinux1_x86_64.whl
6559f4a0438d849cac85ea57f1baa3ba  scipy-1.3.1-cp36-cp36m-win32.whl
8164a4832b3b5e948135b92f91d6e8fa  scipy-1.3.1-cp36-cp36m-win_amd64.whl
1054925a3d2130f803b27a30e8779282
 scipy-1.3.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
cf90a33975bfcd08f7275bbc885785cf  scipy-1.3.1-cp37-cp37m-manylinux1_i686.whl
a9f6dbda23d81ff544d5e0debcc78320
 scipy-1.3.1-cp37-cp37m-manylinux1_x86_64.whl
701657c017eb5164582035232bde5769  scipy-1.3.1-cp37-cp37m-win32.whl
f7d8824ff9193c34b10017a1d67ce9fe  scipy-1.3.1-cp37-cp37m-win_amd64.whl
69db58ceb4b4c3ff3f3ea816e4e426b9  scipy-1.3.1.tar.gz
66e95ade5399a9a336c1b2b78edb2d3a  scipy-1.3.1.tar.xz
62ebcbc144342800c5e6b1c3ba86ba0d  scipy-1.3.1.zip

SHA256
~~~~~~

3ae3692616975d3c10aca6d574d6b4ff95568768d4525f76222fb60f142075b9
 scipy-1.3.1-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
7ccfa44a08226825126c4ef0027aa46a38c928a10f0a8a8483c80dd9f9a0ad44
 scipy-1.3.1-cp35-cp35m-manylinux1_i686.whl
cbc0611699e420774e945f6a4e2830f7ca2b3ee3483fca1aa659100049487dd5
 scipy-1.3.1-cp35-cp35m-manylinux1_x86_64.whl
435d19f80b4dcf67dc090cc04fde2c5c8a70b3372e64f6a9c58c5b806abfa5a8
 scipy-1.3.1-cp35-cp35m-win32.whl
243b04730d7223d2b844bda9500310eecc9eda0cba9ceaf0cde1839f8287dfa8
 scipy-1.3.1-cp35-cp35m-win_amd64.whl
46a5e55850cfe02332998b3aef481d33f1efee1960fe6cfee0202c7dd6fc21ab
 scipy-1.3.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
dd3b52e00f93fd1c86f2d78243dfb0d02743c94dd1d34ffea10055438e63b99d
 scipy-1.3.1-cp36-cp36m-manylinux1_i686.whl
75b513c462e58eeca82b22fc00f0d1875a37b12913eee9d979233349fce5c8b2
 scipy-1.3.1-cp36-cp36m-manylinux1_x86_64.whl
396eb4cdad421f846a1498299474f0a3752921229388f91f60dc3eda55a00488
 scipy-1.3.1-cp36-cp36m-win32.whl
a81da2fe32f4eab8b60d56ad43e44d93d392da228a77e229e59b51508a00299c
 scipy-1.3.1-cp36-cp36m-win_amd64.whl
0baa64bf42592032f6f6445a07144e355ca876b177f47ad8d0612901c9375bef
 scipy-1.3.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
d02d813ec9958ed63b390ded463163685af6025cb2e9a226ec2c477df90c6957
 scipy-1.3.1-cp37-cp37m-manylinux1_i686.whl
89dd6a6d329e3f693d1204d5562dd63af0fd7a17854ced17f9cbc37d5b853c8d
 scipy-1.3.1-cp37-cp37m-manylinux1_x86_64.whl
ac37eb652248e2d7cbbfd89619dce5ecfd27d657e714ed049d82f19b162e8d45
 scipy-1.3.1-cp37-cp37m-win32.whl
a9d606d11eb2eec7ef893eb825017fbb6eef1e1d0b98a5b7fc11446ebeb2b9b1
 scipy-1.3.1-cp37-cp37m-win_amd64.whl
2643cfb46d97b7797d1dbdb6f3c23fe3402904e3c90e6facfe6a9b98d808c1b5
 scipy-1.3.1.tar.gz
326ffdad79f113659ed0bca80f5d0ed5e28b2e967b438bb1f647d0738073a92e
 scipy-1.3.1.tar.xz
47dd0ff4a9f17d97e9b68b363c54111f11b73a388cac55069d6e88a602d20552
 scipy-1.3.1.zip
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJdTOWrAAoJELD/41ZX0J71Y1UP/ivoDqq6BDDdIB45Oo+XIH6h
oZ3c5np+ui/jo/SIF82vM7W/7ZfTEf/BZEKM/ypqO8vk/EltF6jJb8pwyte8Mdic
5hlAQEbLLez2hipGvIIe3K2kS7OFWWorrZWcd+xuZxn9xV3BK4wDXNndpUW61xBF
ElKI+HqfwZ6BLSOpDE9UuMWfIQ9wVACr+TtZSmJsQ1A55o9pcMR5HC0HrTBT40Sj
SPQM1ECOJw2yNzijaYhEUvPIZsFIHjLqtAVkk+4HG675TtLw0NyqZIw9jhUax6vj
rdkj1tSChtSEXE7D2AfZ446bHT+KsxHW6gMXhI96SFi0w698+h2UErZ7uqbnBkQO
OAyzewcUp9yqgSf9pciR6WNiRKvPW8yW+l4XJGqERZDePQ1U4d1Ou9yx1efAkb6t
j4SxpziM4n3ib3TKzRwQ574EJKKOK5TNYHAwH0BIqN4Ace3aN3D/XNojmelUh8gH
d1TklvMGj54pwZjHnqeJMQG+vXbJdI2TXN5qUjFUcsghGvhKPWhl+Un9qoi0+/ZX
nBOEXylKF1MkdV1vd4xLS5gFuyCf7EPnJl9vKMHnrkoUuCoxLM2YtAzjgiUiTjK2
lJAxppWZuNMgTcBsDgg6rtHiLcgl44YehPquV1pvs5DrxQzdq5kVwGu7yU2XRSbf
QIsVcdAujT8MNGOvoJae
=G7vC
-----END PGP SIGNATURE-----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/599bc9e0/attachment.html>

From lxndr.rvs at gmail.com  Fri Aug  9 00:33:56 2019
From: lxndr.rvs at gmail.com (Alexander Reeves)
Date: Thu, 8 Aug 2019 21:33:56 -0700
Subject: [Numpy-discussion] merging empty bins used by
 histogram(bins='auto') to avoid MemoryErrors
Message-ID: <CABeAeRzD45Ve-2Kuqsc2Ww-nnQX=bD5t+_i7o7741Vu_V0g-gg@mail.gmail.com>

Hi all,

This thread is to do with the github issues raised in #11879, #10297, #8203
of and possibly others that didn't appear in my search.

The main issue is that histogram(bins='auto') will sometimes raise a memory
error if the number of automatically-generated bin edges is too large. In
all documented cases, the conditions producing outsized bin numbers are
when the auto-binning defaults to the 'fd' method.

I have taken a crack at minimizing the number of bins used by setting bins
to 'auto' in numpy's histogram method. Based on suggestions from
eric-weiser, the approach merges empty bins.

The method works for the sample datasets in all the issues related to the
FD estimator (#11879, #10297, #8203).

Note that this method produces unequal bin widths.

You can see some code I've already written in a comment on issue #11879 in
the link below.

https://github.com/numpy/numpy/issues/11879#issuecomment-516686087

Thanks for reading this far. Would be happy to turn this into a PR if there
is interest.

-areeves87
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190808/dc20e961/attachment-0001.html>

From peter at entschev.com  Mon Aug 12 06:59:21 2019
From: peter at entschev.com (Peter Andreas Entschev)
Date: Mon, 12 Aug 2019 12:59:21 +0200
Subject: [Numpy-discussion] NEP 30 - Duck Typing for NumPy Arrays -
 Implementation
In-Reply-To: <CAEQ_TvdMn+iU8Qt+0PE79U61qDa6aO3c0U4n-c_1Xjth+fAJ6Q@mail.gmail.com>
References: <CANxROpwq+MuR=noghriRQ9BVWAGaPeo9ax8SkeQzEGSAc4Dr7g@mail.gmail.com>
 <CABL7CQjXuno3V95EbbyXrOwYbAhamLndXOECcdh9t7arDy=P0Q@mail.gmail.com>
 <CAEQ_TvcQDBs1SsWmweUL_hvQotWyaL6RBvwNy9C38r0XtT_Urg@mail.gmail.com>
 <CABL7CQhj9MqM4G4bYJZ0spmT_Pr_ard-KCJGtTyVOav_Gp0y3w@mail.gmail.com>
 <CAEQ_TvfgtKveBr8TDrnUXz_vw6-+h1n19ot+dddVd67pM8a-JA@mail.gmail.com>
 <CAB6mnxLruM1wm=UdVOa14UOJOGPFD7PZGQK_haNrDDgCk2B+hA@mail.gmail.com>
 <CAEQ_TvdMn+iU8Qt+0PE79U61qDa6aO3c0U4n-c_1Xjth+fAJ6Q@mail.gmail.com>
Message-ID: <CANxROpyv9PJwnPmB5d9xp94PBg5erPFQ-uomdinmBAfY-JRWow@mail.gmail.com>

Apologies for the late reply. I've opened a new PR
https://github.com/numpy/numpy/pull/14257 with the changes requested
on clarifying the text. After reading the detailed description, I've
decided to add a subsection "Scope" to clarify the scope where NEP-30
would be useful. I think the inclusion of this new subsection
complements the "Detail description" forming a complete text w.r.t.
motivation of the NEP, but feel free to point out disagreements with
my suggestion. I've also added a new section "Usage" pointing out how
one would use duck array in replacement to np.asarray where relevant.

Regarding the naming discussion, I must say I like the idea of keeping
the __array_ prefix, but it seems like that is going to be difficult
given that none of the existing ideas so far play very nicely with
that. So if the general consensus is to go with __numpy_like__, I
would also update the NEP to reflect that changes. FWIW, I
particularly neither like nor dislike __numpy_like__, but I don't have
any better suggestions than that or keeping the current naming.

Best,
Peter

On Thu, Aug 8, 2019 at 3:40 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>
>
> On Wed, Aug 7, 2019 at 6:18 PM Charles R Harris <charlesr.harris at gmail.com> wrote:
>>
>>
>>
>> On Wed, Aug 7, 2019 at 7:10 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>>>
>>> On Wed, Aug 7, 2019 at 5:11 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>>>
>>>>
>>>> On Mon, Aug 5, 2019 at 6:18 PM Stephan Hoyer <shoyer at gmail.com> wrote:
>>>>>
>>>>> On Mon, Aug 5, 2019 at 2:48 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> The NEP currently does not say who this is meant for. Would you expect libraries like SciPy to adopt it for example?
>>>>>>
>>>>>> The NEP also (understandably) punts on the question of when something is a valid duck array. If you want this to be widely used, that will need an answer or at least some rough guidance though. For example, we would expect a duck array to have a mean() method, but probably not a ptp() method. A library author who wants to use np.duckarray() needs to know, because she can't test with all existing and future duck array implementations.
>>>>>
>>>>>
>>>>> I think this is covered in NEP-22 already.
>>>>
>>>>
>>>> It's not really. We discussed this briefly in the community call today, Peter said he will try to add some text.
>>>>
>>>> We should not add new functions to NumPy without indicating who is supposed to use this, and what need it fills / problem it solves. It seems pretty clear to me that it's mostly aimed at library authors rather than end users. And also that mature libraries like SciPy may not immediately adopt it, because it's too fuzzy - so it's new libraries first, mature libraries after the dust has settled a bit (I think).
>>>
>>>
>>> I totally agree -- we definitely should clarify this in the docstring and elsewhere in the docs. An example in the new doc page on "Writing custom array containers" (https://numpy.org/devdocs/user/basics.dispatch.html) would also probably be appropriate.
>>>
>>>>>
>>>>> As discussed there, I don't think NumPy is in a good position to pronounce decisive APIs at this time. I would welcome efforts to try, but I don't think that's essential for now.
>>>>
>>>>
>>>> There's no need to pronounce a decisive API that fully covers duck array. Note that RNumPy is an attempt in that direction (not a full one, but way better than nothing). In the NEP/docs, at least saying something along the lines of "if you implement this, we recommend the following strategy: check if a function is present in Dask, CuPy and Sparse. If so, it's reasonable to expect any duck array to work here. If not, we suggest you indicate in your docstring what kinds of duck arrays are accepted, or what properties they need to have". That's a spec by implementation, which is less than ideal but better than saying nothing.
>>>
>>>
>>> OK, I agree here as well -- some guidance is better than nothing.
>>>
>>> Two other minor notes on this NEP, concerning naming:
>>> 1. We should have a brief note on why we settled on the name "duck array". Namely, as discussed in NEP-22, we don't love the "duck" jargon, but we couldn't come up with anything better since NumPy already uses "array like" and "any array" for different purposes.
>>> 2. The protocol should use *something* more clearly namespaced as NumPy specific than __duckarray__. All the other special protocols NumPy defines start with "__array_". That suggests either __array_duckarray__ (sounds a little redundant) or __numpy_duckarray__ (which I like the look of, but is a different from the existing protocols).
>>>
>>
>> `__numpy_like__` ?
>
>
>
> This could work, but I think we would also want to rename the NumPy function itself to either np.like or np.numpy_like. The later is a little redundant but definitely more self-descriptive than "duck array".
>
>>
>> Chuck
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From sebastian at sipsolutions.net  Tue Aug 13 10:27:53 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 13 Aug 2019 09:27:53 -0500
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, August 14
Message-ID: <714c2f3c9d91b685ccb1cb68b0c5e0ed3cf9a486.camel@sipsolutions.net>

Hi all,

There will be a NumPy Community meeting Wednesday August 14 at 11 am
Pacific Time. Everyone is invited to join in and edit the work-in-
progress meeting topics and notes:
https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg

Best wishes

Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190813/c6811494/attachment.sig>

From abdulsamod2 at gmail.com  Wed Aug 14 08:45:28 2019
From: abdulsamod2 at gmail.com (Azeez Abdulsamod)
Date: Wed, 14 Aug 2019 13:45:28 +0100
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, August 14
In-Reply-To: <714c2f3c9d91b685ccb1cb68b0c5e0ed3cf9a486.camel@sipsolutions.net>
References: <714c2f3c9d91b685ccb1cb68b0c5e0ed3cf9a486.camel@sipsolutions.net>
Message-ID: <CAFmXvAZUWiU4eG0o9tU1DQ__kExodDcu2YYhXAi3MZcjAmm=Pg@mail.gmail.com>

When will there be  Numpy community in Nigeria

On Tue, Aug 13, 2019, 3:31 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> Hi all,
>
> There will be a NumPy Community meeting Wednesday August 14 at 11 am
> Pacific Time. Everyone is invited to join in and edit the work-in-
> progress meeting topics and notes:
> https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg
>
> Best wishes
>
> Sebastian
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190814/6ff027b4/attachment.html>

From sebastian at sipsolutions.net  Wed Aug 14 09:07:52 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 14 Aug 2019 08:07:52 -0500
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, August 14
In-Reply-To: <CAFmXvAZUWiU4eG0o9tU1DQ__kExodDcu2YYhXAi3MZcjAmm=Pg@mail.gmail.com>
References: <714c2f3c9d91b685ccb1cb68b0c5e0ed3cf9a486.camel@sipsolutions.net>
 <CAFmXvAZUWiU4eG0o9tU1DQ__kExodDcu2YYhXAi3MZcjAmm=Pg@mail.gmail.com>
Message-ID: <143e6acd2ed07fdd25d2c7a33a366731b29f21ed.camel@sipsolutions.net>

On Wed, 2019-08-14 at 13:45 +0100, Azeez Abdulsamod wrote:
> When will there be  Numpy community in Nigeria
> 

These meetings are online through video/telephone conference. The link
information is in the hackmd link.

- Sebastian


> On Tue, Aug 13, 2019, 3:31 PM Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
> > Hi all,
> > 
> > There will be a NumPy Community meeting Wednesday August 14 at 11
> > am
> > Pacific Time. Everyone is invited to join in and edit the work-in-
> > progress meeting topics and notes:
> > https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg
> > 
> > Best wishes
> > 
> > Sebastian
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190814/43f8973e/attachment.sig>

From robbmcleod at gmail.com  Wed Aug 14 10:19:03 2019
From: robbmcleod at gmail.com (Robert McLeod)
Date: Wed, 14 Aug 2019 07:19:03 -0700
Subject: [Numpy-discussion] ANN: NumExpr 2.7.0 Release
Message-ID: <CAEFUWWUF8ujx3XpGEevRcA+1Ssqzrc19fJVMvAR9SzNrYZaCAw@mail.gmail.com>

=========================
 Announcing Numexpr 2.7.0
=========================

Hi everyone,

This is a minor version bump for NumExpr. We would like to highlight the
changes
made in 2.6.9 (which in retrospect should have been a minor version bump),
where
the maximum number of threads spawned can be limited by setting the
environment
variable "NUMEXPR_MAX_THREADS". If this variable is not set, in 2.7.0 the
historical limit of 8 threads will be used. The lack of a check caused some
problems on very large hosts in cluster environments in 2.6.9.

In addition, we are officially dropping Python 2.6 support in this release
as
we cannot perform continuous integration for it.

Project documentation is available at:

http://numexpr.readthedocs.io/

Changes from 2.6.9 to 2.7.0
----------------------------

- The default number of 'safe' threads has been restored to the historical
limit
  of 8, if the environment variable "NUMEXPR_MAX_THREADS" has not been set.
- Thanks to @eltoder who fixed a small memory leak.
- Support for Python 2.6 has been dropped, as it is no longer available via
  TravisCI.
- A typo in the test suite that had a less than rather than greater than
symbol
  in the NumPy version check has been corrected thanks to dhomeier.
- The file `site.cfg` was being accidentally included in the sdists on
PyPi.
  It has now been excluded.

What's Numexpr?
---------------

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It has multi-threaded capabilities, as well as support for Intel's
MKL (Math Kernel Library), which allows an extremely fast evaluation
of transcendental functions (sin, cos, tan, exp, log...) while
squeezing the last drop of performance out of your multi-core
processors.  Look here for a some benchmarks of numexpr using MKL:

https://github.com/pydata/numexpr/wiki/NumexprMKL

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational engine for projects that
don't want to adopt other solutions requiring more heavy dependencies.

Where I can find Numexpr?
-------------------------

The project is hosted at GitHub in:

https://github.com/pydata/numexpr

You can get the packages from PyPI as well (but not for RC releases):

http://pypi.python.org/pypi/numexpr

Documentation is hosted at:

http://numexpr.readthedocs.io/en/latest/

Share your experience
---------------------

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.


Enjoy data!

-- 
Robert McLeod
robbmcleod at gmail.com
robert.mcleod at hitachi-hhtc.ca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190814/9390d488/attachment.html>

From arj.python at gmail.com  Fri Aug 16 23:10:04 2019
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sat, 17 Aug 2019 07:10:04 +0400
Subject: [Numpy-discussion] Numpy twitter handle?
Message-ID: <CADrxXXmuWJCWwLEwiYkBFEm5ZyGGpAv6smy8NsnJL5+FB_dtbQ@mail.gmail.com>

Greetings, what is the numpy twitter handle? Thanks!

Abdur-Rahmaan Janhangeer
Mauritius
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190817/b9426f3f/attachment.html>

From ralf.gommers at gmail.com  Sat Aug 17 13:40:15 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 17 Aug 2019 10:40:15 -0700
Subject: [Numpy-discussion] Numpy twitter handle?
In-Reply-To: <CADrxXXmuWJCWwLEwiYkBFEm5ZyGGpAv6smy8NsnJL5+FB_dtbQ@mail.gmail.com>
References: <CADrxXXmuWJCWwLEwiYkBFEm5ZyGGpAv6smy8NsnJL5+FB_dtbQ@mail.gmail.com>
Message-ID: <CABL7CQhW3vOjjxyXvUto_o+kv1KnJsG00tsguMQF204=6MNXEQ@mail.gmail.com>

On Fri, Aug 16, 2019 at 8:10 PM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Greetings, what is the numpy twitter handle? Thanks!
>

There is no such thing. We're typically pretty busy with code/website/docs.
A blog, Twitter, or other social media presence has not really been on our
radar.

Cheers,
Ralf



> Abdur-Rahmaan Janhangeer
> Mauritius
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190817/f7df3cf3/attachment.html>

From arj.python at gmail.com  Sun Aug 18 00:36:12 2019
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 18 Aug 2019 08:36:12 +0400
Subject: [Numpy-discussion] Numpy twitter handle?
In-Reply-To: <CABL7CQhW3vOjjxyXvUto_o+kv1KnJsG00tsguMQF204=6MNXEQ@mail.gmail.com>
References: <CADrxXXmuWJCWwLEwiYkBFEm5ZyGGpAv6smy8NsnJL5+FB_dtbQ@mail.gmail.com>
 <CABL7CQhW3vOjjxyXvUto_o+kv1KnJsG00tsguMQF204=6MNXEQ@mail.gmail.com>
Message-ID: <CADrxXXmhDRdxj4-3A2zYKdq3eEkueyHbiQL-b2QkDra4HJMKHw@mail.gmail.com>

Poor me a paper has been published citing numpy in the top 10 imports (the
others being from the std lib), it was a great event as it was the
conclusion after analysing ALL of pypi. A ping would have sufficed to
notify.

>From the tweet: https://twitter.com/di_codes/status/1161475147045507077?s=19

Paper:
https://t.co/WKJdS9FpeV

Yours,

Abdur-Rahmaan Janhangeer
Mauritius
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190818/ad222d4d/attachment.html>

From matti.picus at gmail.com  Sun Aug 18 01:04:33 2019
From: matti.picus at gmail.com (Matti Picus)
Date: Sun, 18 Aug 2019 08:04:33 +0300
Subject: [Numpy-discussion] Numpy twitter handle?
In-Reply-To: <CADrxXXmhDRdxj4-3A2zYKdq3eEkueyHbiQL-b2QkDra4HJMKHw@mail.gmail.com>
References: <CADrxXXmuWJCWwLEwiYkBFEm5ZyGGpAv6smy8NsnJL5+FB_dtbQ@mail.gmail.com>
 <CABL7CQhW3vOjjxyXvUto_o+kv1KnJsG00tsguMQF204=6MNXEQ@mail.gmail.com>
 <CADrxXXmhDRdxj4-3A2zYKdq3eEkueyHbiQL-b2QkDra4HJMKHw@mail.gmail.com>
Message-ID: <25806b45-4525-c0ef-3145-62382a139097@gmail.com>

On 18/8/19 7:36 am, Abdur-Rahmaan Janhangeer wrote:
> Poor me a paper has been published citing numpy in the top 10 imports 
> (the others being from the std lib), it was a great event as it was 
> the conclusion after analysing ALL of pypi. A ping would have sufficed 
> to notify.
>
> From the tweet: 
> https://twitter.com/di_codes/status/1161475147045507077?s=19
>
> Paper:
> https://t.co/WKJdS9FpeV
>
> Yours,
>
> Abdur-Rahmaan Janhangeer
> Mauritius
>

Thanks for the heads up.

A direct link to the paper https://arxiv.org/abs/1907.11073

and the github repo 
https://github.com/bommarito-consulting/pypi-research-data (still not 
complete with data, code or paper source)


Lots of interesting tidbits in that paper, for instance "Together, the 
four ?deep learning? packages tf-nightly, mxnet-cu100mkl, mxnet-cu100, 
and tf-nightly-gpu use approximately 500 gigabytes of PyPI storage - 
nearly 25% of all PyPI storage". and "Table 19: Number of unique 
importing packages by top-level package import" which shows the number 
of import statements in new releases over time - seeing the change in 
emphasis between NumPy vs. django.


Matti


From ralf.gommers at gmail.com  Sun Aug 18 23:28:24 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 18 Aug 2019 20:28:24 -0700
Subject: [Numpy-discussion] Season of Docs - welcome Anne, Maja, Brandon,
 Shekhar, Christina
In-Reply-To: <CABL7CQijUi-7+jAkxXtDdvw-k__CJBm_3L2KzHjC=ayLoGd3_w@mail.gmail.com>
References: <CABL7CQiK6sP-4C-=-iZfzjf7jh8ZfY4tQvmx758NWugOeLZH0Q@mail.gmail.com>
 <CABL7CQijUi-7+jAkxXtDdvw-k__CJBm_3L2KzHjC=ayLoGd3_w@mail.gmail.com>
Message-ID: <CABL7CQgbZiM193SJ0hELOt4MvhoKGfjKmNjKEi4O__Om5QDbPw@mail.gmail.com>

Hi all,

Happy to announce that we have a fifth participant:

Christina Lee, "SciPy documentation: Design, Usability and Content".

Welcome Christina!

I expect that this is the final announcement. Really enjoying the momentum
that's already building up around documentation and website, and looking
forward to the next couple of months!

Cheers,
Ralf


On Wed, Aug 7, 2019 at 6:03 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Tue, Aug 6, 2019 at 4:46 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi all,
>>
>> Google has announced the Season of Docs participants for this year [1].
>> We had a lot of excellent candidates and had to make some hard choices. We
>> applied for extra slots, but unfortunately didn't win the lottery for
>> those; we got one slot for NumPy and one for SciPy. We chose the projects
>> of Anne for NumPy and Maja for SciPy:
>>
>> Anne Bonner, "Making "The Basics" a Little More Basic: Improving the
>> Introductory NumPy Sections" [2]
>>
>> Maja Gwozdz, "User-oriented documentation and thorough restructuring" [3]
>>
>> That's not all though. There was some space left in the budget of the
>> NumPy BIDS grant, and St?fan has reserved that so we can accept more
>> writers and provide them the same mentoring and funding as they would have
>> gotten through GSoD. We could only start the conversations about that once
>> Google made its decisions, so a further announcement will follow. However,
>> we already have one extra project confirmed, from Brandon:
>>
>> Brandon David, "Improve the documentation of scipy.stats" (project
>> details to be published).
>>
>
> Happy to announce that we have a fourth participant:
>
> Shekhar Rajak, "numpy.org redesign and high level documentation
> restructuring for end user focus"
>
> Welcome Shekhar!
>
> I will send out a poll to find a good time for everyone for a kickoff
>> call. Our intent is to build a documentation team with multiple writers and
>> mentors interacting and able to help each other out. And all of this will
>> also interact with the numpy.org website redesign and the people putting
>> energy into that:)
>>
>
> Here is the poll link: https://doodle.com/poll/skgbk74gsg8zpziu. I hope
> we can find a time that works for everyone - we're split over all US
> timezones, Europe and India. So it's going to be early morning or late
> evening somewhere.
>
> Sending this out in public, so anyone who wants to participate is welcome
> to join. I've Bcc'd all participants and mentors, to make sure they see
> this.
>
> Cheers,
> Ralf
>
>
>
>>
>> I'm very happy to welcome Anne, Maja and Brandon!
>>
>> Cheers,
>> Ralf
>>
>>
>> [1] https://developers.google.com/season-of-docs/docs/participants/
>> [2]
>> https://developers.google.com/season-of-docs/docs/participants/project-numpy
>> [3]
>> https://developers.google.com/season-of-docs/docs/participants/project-scipy
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190818/7338d5e0/attachment.html>

From jorisvandenbossche at gmail.com  Mon Aug 19 11:35:59 2019
From: jorisvandenbossche at gmail.com (Joris Van den Bossche)
Date: Mon, 19 Aug 2019 17:35:59 +0200
Subject: [Numpy-discussion] Releasing the GIL in ufuncs dealing with object
 arrays
Message-ID: <CALQtMBboc_Mp1rpyKx=acepBkK5n5b1sB-AfuQAMaUfUdD18Ew@mail.gmail.com>

Hi all,

PyGEOS (https://github.com/caspervdw/pygeos) is an experimental package
implementing a set of numpy ufuncs to provide vectorized geometry
functionality (wrapping the C++ GEOS library).

The way it does this is by implementing a Python extension type
(pygeos.Geometry) that wraps an actual GEOSGeometry object by storing a
pointer to it in the PyObject struct of the extension type. This way, we
can store those objects in an object dtype array in numpy, but still access
the pointer in the ufunc inner loop without needing the python interpreter.

The single threaded performance of the ufuncs with the approach above is
very good. There doesn't seem to be an overhead of using the object array
approach.
However, as far as I can find in the docs, the GIL is only released in
ufuncs for non-object dtypes.

So the question here is: is there a way to let numpy release the GIL in
such a case nonetheless? Although the array holds python objects, the ufunc
inner loop only accesses a static attribute of the object struct, not
needing any explicit Python interaction.

Best,
Joris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190819/520bcafe/attachment.html>

From sebastian at sipsolutions.net  Mon Aug 19 11:56:02 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Mon, 19 Aug 2019 10:56:02 -0500
Subject: [Numpy-discussion] Releasing the GIL in ufuncs dealing with
 object arrays
In-Reply-To: <CALQtMBboc_Mp1rpyKx=acepBkK5n5b1sB-AfuQAMaUfUdD18Ew@mail.gmail.com>
References: <CALQtMBboc_Mp1rpyKx=acepBkK5n5b1sB-AfuQAMaUfUdD18Ew@mail.gmail.com>
Message-ID: <ec3901b22b12daf950ccffbeee2a25c8e2e4442a.camel@sipsolutions.net>

Hi Joris,

On Mon, 2019-08-19 at 17:35 +0200, Joris Van den Bossche wrote:
> Hi all,
> 
> PyGEOS (https://github.com/caspervdw/pygeos) is an experimental
> package implementing a set of numpy ufuncs to provide vectorized
> geometry functionality (wrapping the C++ GEOS library).
> 
> The way it does this is by implementing a Python extension type
> (pygeos.Geometry) that wraps an actual GEOSGeometry object by storing
> a pointer to it in the PyObject struct of the extension type. This
> way, we can store those objects in an object dtype array in numpy,
> but still access the pointer in the ufunc inner loop without needing
> the python interpreter.
> 
> The single threaded performance of the ufuncs with the approach above
> is very good. There doesn't seem to be an overhead of using the
> object array approach. 
> However, as far as I can find in the docs, the GIL is only released
> in ufuncs for non-object dtypes. 
> 
> So the question here is: is there a way to let numpy release the GIL
> in such a case nonetheless? Although the array holds python objects,
> the ufunc inner loop only accesses a static attribute of the object
> struct, not needing any explicit Python interaction.
> 

Hmmm, interesting use case. No, I do not think there currently is a
reasonable way to do this (I think there may be ways to hack it). Even
when all access to the objects is safe by itself, you still have the
problem that the object stored inside the array could be replaced (and
invalidated) at any time if you run multithreaded.

We would like to type such objects in the future, even then, I am not
sure how to make things safe against race conditions if elements are
replaced (and deleted).

This is an interesting use case, since arrays of pointers (or specific
pyobjects) will always have this type of issue, and I am not sure how
you would avoid it (a cheap lock on the object itself works probably,
but even if it is cheap, it is probably fairly expensive?).

Best,

Sebastian


> Best,
> Joris
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190819/d391c384/attachment.sig>

From sebastian at sipsolutions.net  Mon Aug 19 15:04:08 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Mon, 19 Aug 2019 14:04:08 -0500
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, Aug. 21
Message-ID: <f631f158574b948df2db84902179638ed0ef838a.camel@sipsolutions.net>

Hi all,

There will be a NumPy Community meeting Wednesday August 21 at 11 am
Pacific Time. Everyone is invited to join in and edit the work-in-
progress meeting topics and notes:
https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both

Best wishes

Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190819/7b1d1704/attachment.sig>

From charlesr.harris at gmail.com  Mon Aug 19 16:22:07 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 19 Aug 2019 14:22:07 -0600
Subject: [Numpy-discussion] NumPy 1.17.1
Message-ID: <CAB6mnxK-PY2=R-8qdYeRXBKutq66ywfsVfPyoWm=XH55kF1XMQ@mail.gmail.com>

Hi All,

I'm planning on releasing NumPy 1.17.1 this weekend. If there are fixes
that you think need to be included that are not currently marked for
backport, please let me know.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190819/5db42dae/attachment.html>

From jorisvandenbossche at gmail.com  Tue Aug 20 16:30:01 2019
From: jorisvandenbossche at gmail.com (Joris Van den Bossche)
Date: Tue, 20 Aug 2019 22:30:01 +0200
Subject: [Numpy-discussion] Releasing the GIL in ufuncs dealing with
 object arrays
In-Reply-To: <ec3901b22b12daf950ccffbeee2a25c8e2e4442a.camel@sipsolutions.net>
References: <CALQtMBboc_Mp1rpyKx=acepBkK5n5b1sB-AfuQAMaUfUdD18Ew@mail.gmail.com>
 <ec3901b22b12daf950ccffbeee2a25c8e2e4442a.camel@sipsolutions.net>
Message-ID: <CALQtMBaz=X7y4FewDwQ+WQ2zNF8gZghXh0FkPUqeka=2pYB8aA@mail.gmail.com>

Hi Sebastian,

Thanks for the answer!

On Mon, 19 Aug 2019 at 17:57, Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> ...
>
> Hmmm, interesting use case. No, I do not think there currently is a
> reasonable way to do this (I think there may be ways to hack it). Even
> when all access to the objects is safe by itself, you still have the
> problem that the object stored inside the array could be replaced (and
> invalidated) at any time if you run multithreaded.
>

Would it help to have a custom dtype that ensures that all objects in the
array are of this specific extension type? (I don't know if a custom dtype
(done in C, like the quaternion examples) are possible for storing python
objects)


> We would like to type such objects in the future, even then, I am not
> sure how to make things safe against race conditions if elements are
> replaced (and deleted).
>
> This is an interesting use case, since arrays of pointers (or specific
> pyobjects) will always have this type of issue, and I am not sure how
> you would avoid it (a cheap lock on the object itself works probably,
> but even if it is cheap, it is probably fairly expensive?).
>

Currently, we are thinking of doing two loops in the ufunc. First one for
getting all the pointers into a C array, and then manually releasing the
gil for the second loop doing the actual operation on the array of
pointers. See https://github.com/caspervdw/pygeos/issues/7 for example
code. From a quick experiment that seems to give only a small overhead (in
a single threaded case).

That of course still has the same problems as you mentioned (although in
our case, we are, in principle, the holders of the array and know what it
contains, and the individual extension type objects are not mutable), but
then at least it is our own responsibility of making sure that the array
contains the correct objects and is not mutated.

Joris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190820/9afbe1f6/attachment.html>

From sebastian at sipsolutions.net  Wed Aug 21 12:42:14 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Wed, 21 Aug 2019 11:42:14 -0500
Subject: [Numpy-discussion] Releasing the GIL in ufuncs dealing with
 object arrays
In-Reply-To: <CALQtMBaz=X7y4FewDwQ+WQ2zNF8gZghXh0FkPUqeka=2pYB8aA@mail.gmail.com>
References: <CALQtMBboc_Mp1rpyKx=acepBkK5n5b1sB-AfuQAMaUfUdD18Ew@mail.gmail.com>
 <ec3901b22b12daf950ccffbeee2a25c8e2e4442a.camel@sipsolutions.net>
 <CALQtMBaz=X7y4FewDwQ+WQ2zNF8gZghXh0FkPUqeka=2pYB8aA@mail.gmail.com>
Message-ID: <085608ac1f740d68f5be30eba391d11afe1c7e42.camel@sipsolutions.net>

On Tue, 2019-08-20 at 22:30 +0200, Joris Van den Bossche wrote:
> Hi Sebastian,
> 
> Thanks for the answer!
> 
> On Mon, 19 Aug 2019 at 17:57, Sebastian Berg <
> sebastian at sipsolutions.net> wrote:
> > ...
> > 
> > Hmmm, interesting use case. No, I do not think there currently is a
> > reasonable way to do this (I think there may be ways to hack it).
> > Even
> > when all access to the objects is safe by itself, you still have
> > the
> > problem that the object stored inside the array could be replaced
> > (and
> > invalidated) at any time if you run multithreaded.
> 
> Would it help to have a custom dtype that ensures that all objects in
> the array are of this specific extension type? (I don't know if a
> custom dtype (done in C, like the quaternion examples) are possible
> for storing python objects)
>  

You can do a custom dtype like the quaternion. We are working on
creating new custom dtypes, but that will be a while until it lands.
That is one thing I am not quite sure about, whether it is possible to
do an object backed dtype currently (the issue is whether the reference
counting is done -- especially without adding other issues), I could
have a look if you like.

Making that easy is very high up on the "what I want in the future"
list.


> > We would like to type such objects in the future, even then, I am
> > not
> > sure how to make things safe against race conditions if elements
> > are
> > replaced (and deleted).
> > 
> > This is an interesting use case, since arrays of pointers (or
> > specific
> > pyobjects) will always have this type of issue, and I am not sure
> > how
> > you would avoid it (a cheap lock on the object itself works
> > probably,
> > but even if it is cheap, it is probably fairly expensive?).
> 
> Currently, we are thinking of doing two loops in the ufunc. First one
> for getting all the pointers into a C array, and then manually
> releasing the gil for the second loop doing the actual operation on
> the array of pointers. See 
> https://github.com/caspervdw/pygeos/issues/7 for example code. From a
> quick experiment that seems to give only a small overhead (in a
> single threaded case).

I suppose that should work. If you are within a inner loop, you have
only limited control on the chunking/buffersize though, so in the worst
case you might be releasing the GIL very often.
I suppose in the event that the array is not writeable, you could
actually release the GIL.

This is something that we are thinking about enabling full control
over, although it is not on the high list for priorities right now
(Basically my plan/thought is to start off without allowing such
things, but keeping it open for later addition).

In practice I suppose that such objects (and ufuncs) can be fairly
heavy, so that even indiscriminately copying the full input arrays
really is not a big issue as such.

> 
> That of course still has the same problems as you mentioned (although
> in our case, we are, in principle, the holders of the array and know
> what it contains, and the individual extension type objects are not
> mutable), but then at least it is our own responsibility of making
> sure that the array contains the correct objects and is not mutated.
> 

I understood it as: You copy+incref. After that all seems OK with me,
unless your object itself is mutable (in a non-threadsafe way).

Best,

Sebastian


> Joris
>  
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190821/25848af4/attachment.sig>

From ralf.gommers at gmail.com  Wed Aug 21 19:46:53 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 21 Aug 2019 16:46:53 -0700
Subject: [Numpy-discussion] NumFOCUS Summit and 4 Nov sprint
Message-ID: <CABL7CQgFZku8fegBmsVPvENCj6_j5KH8zgo8bhFhY5u9Pme1xw@mail.gmail.com>

Hi all,

On 2-3 November the NumFOCUS Summit will be held in NYC, and as a sponsored
project NumPy can send two people to it (with NumFOCUS covering travel and
hotel costs). To choose those two people we asked the members of the
Steering Council first. It seems that we may have an open slot left, so
we're asking anyone else who is a part of the NumPy project if they're
interested in going.

Details from the invitation:

We will kick things off Friday, November 1st with a Reception Dinner at Havana
Central
<https://numfocus-dot-yamm-track.appspot.com/Redirect?ukey=1my4O8gi_yIWFTKq0Hw07hrwX07MM9zdg73znqAmuJDY-1540241207&key=YAMMID-30545080&link=https%3A%2F%2Fhavanacentral.com%2F>
.

The Summit program (Nov 2-3) will be aimed at addressing both the challenge
of securing funding and navigating the challenges of project governance and
leadership. You can view the schedule in more detail here
<https://numfocus-dot-yamm-track.appspot.com/Redirect?ukey=1my4O8gi_yIWFTKq0Hw07hrwX07MM9zdg73znqAmuJDY-1540241207&key=YAMMID-30545080&link=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1u0NWbFcwNrw-bPz_Pu1wWOIG-LJf5aL6q3oyHlclptM%2Fedit%23gid%3D1464008080>
.


If you're interested, please say so either here or off-list.


Also note that we'll likely organize a sprint on 4 Nov, so if there are
people who would be interested in that (I imagine there's quite a few
people on this list from near NYC?), that'd be good to know as well.


Cheers,

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190821/80c42c99/attachment.html>

From jpc-ml at zenburn.net  Fri Aug 23 05:27:39 2019
From: jpc-ml at zenburn.net (=?UTF-8?Q?Jakub_Piotr_C=c5=82apa?=)
Date: Fri, 23 Aug 2019 11:27:39 +0200
Subject: [Numpy-discussion] NumPy cross-compilation on macOS host
Message-ID: <244ab376-31b4-ea52-09b5-32c1b5c4f1b5@zenburn.net>

Hi,

I spent some time trying to cross-compile NumPy for Linux ARM64 on a 
macOS host as part of an OpenWRT compilation [1]. It was not so bad but 
certainly not trivial and I was wondering if we could use my experience 
to help streamline the process.

I encountered two challenges:

1. Python distutils did not allow me to provide a proper cross-ranlib 
program path. I opened an issue for this on the Python bug tracker [2].

2. numpy distutils scripts check sys.platform in many places. 
sys.platform returns the platform on which compilation occurs (darwin in 
my case) which in case of cross-compilation is not the one on which the 
code will be run (linux). This of course results in invalid compiler and 
linker flags.

I made a quick'n'dirty fix [3] and got numpy to compile and run 
correctly (but I have yet to run the full test suite). Of course this is 
not a proper way to solve this so I was hoping to get some feedback on 
the preferred/best way. I did not find any info about an official way to 
do this with distutils.

One idea (from @commodo on GitHub) would be to just change my hardcoded 
string ("Linux") to something like this:

target_platform = os.getenviron(?TARGET_PLATFORM?, sys.platform)

It could also be further refactored by moving it to a single place but 
while certainly more elegant and "proper" it may ultimately make it less 
readable and harder to follow. I am also unsure where would be the best 
place to put it and how to pass it around to other modules.

[1]: https://github.com/openwrt/packages/pull/9797
[2]: https://bugs.python.org/issue37916
[3]: 
https://github.com/openwrt/packages/pull/9797/commits/e701e122fb9c78d5cecffb88c1d750178a239083#diff-1ea8d839590bcdc406854d3998a96704

From ralf.gommers at gmail.com  Fri Aug 23 18:16:28 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 23 Aug 2019 15:16:28 -0700
Subject: [Numpy-discussion] NumPy cross-compilation on macOS host
In-Reply-To: <244ab376-31b4-ea52-09b5-32c1b5c4f1b5@zenburn.net>
References: <244ab376-31b4-ea52-09b5-32c1b5c4f1b5@zenburn.net>
Message-ID: <CABL7CQi=X6KfHwCcAe4h=Ae0vHSKuAMrp2XakwTJ1god+uty3g@mail.gmail.com>

On Fri, Aug 23, 2019 at 2:28 AM Jakub Piotr C?apa <jpc-ml at zenburn.net>
wrote:

> Hi,
>
> I spent some time trying to cross-compile NumPy for Linux ARM64 on a
> macOS host as part of an OpenWRT compilation [1]. It was not so bad but
> certainly not trivial and I was wondering if we could use my experience
> to help streamline the process.
>

Thanks, that would be great!

We have quite some issues with cross-compilation, and usually they're hard
to resolve (plus it's not something we officially support - but there's a
clear need for improvement). Note that conda is also a cross-compiler (to
the same system), sometimes worth looking into how it deals with
(numpy.)distutils issues.


> I encountered two challenges:
>
> 1. Python distutils did not allow me to provide a proper cross-ranlib
> program path. I opened an issue for this on the Python bug tracker [2].
>

That seems like a straightforward enough patch. distutils isn't
well-maintained but hopefully that gets merged soon. If it doesn't get
responses in a couple of weeks, please feel free to ping me and I can
review/test (I'm not a core Python dev, but it may help push it forwards).


>
> 2. numpy distutils scripts check sys.platform in many places.
> sys.platform returns the platform on which compilation occurs (darwin in
> my case) which in case of cross-compilation is not the one on which the
> code will be run (linux). This of course results in invalid compiler and
> linker flags.
>
> I made a quick'n'dirty fix [3] and got numpy to compile and run
> correctly (but I have yet to run the full test suite). Of course this is
> not a proper way to solve this so I was hoping to get some feedback on
> the preferred/best way. I did not find any info about an official way to
> do this with distutils.
>
> One idea (from @commodo on GitHub) would be to just change my hardcoded
> string ("Linux") to something like this:
>
> target_platform = os.getenviron(?TARGET_PLATFORM?, sys.platform)
>

That seems reasonable enough. Prefixing that name with NPY_ would be good,
to not get accidental clashes.

I'm not sure if there's a more standard/preferred way, I haven't come
across it in distutils. There's some issues with reasonably detailed
discussion for SciPy - this one may be useful:
https://github.com/scipy/scipy/issues/8571


> It could also be further refactored by moving it to a single place but
> while certainly more elegant and "proper" it may ultimately make it less
> readable and harder to follow. I am also unsure where would be the best
> place to put it and how to pass it around to other modules.
>

Keeping all platform comparisons where they are now seems right. Just need
one central place to parse NPY_TARGET_PLATFORM - perhaps in
numpy/distutils/misc_util.py

Cheers,
Ralf


> [1]: https://github.com/openwrt/packages/pull/9797
> [2]: https://bugs.python.org/issue37916
> [3]:
>
> https://github.com/openwrt/packages/pull/9797/commits/e701e122fb9c78d5cecffb88c1d750178a239083#diff-1ea8d839590bcdc406854d3998a96704
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190823/0b104a83/attachment.html>

From ralf.gommers at gmail.com  Sun Aug 25 15:15:17 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 25 Aug 2019 12:15:17 -0700
Subject: [Numpy-discussion] first Season of Docs blog post
Message-ID: <CABL7CQhN_uD5KwviQiRkimUUjqHUeL5PPzCbd8kwZb_J_9aFbw@mail.gmail.com>

Hi all,

Anne has written a very nice blog post about the start of the Season of
Docs projects:
https://towardsdatascience.com/what-do-you-want-to-see-in-the-numpy-docs-de73efb80375

It even includes a pineapple with sunglasses and a party hat, so a good
read:)

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190825/ac32036a/attachment.html>

From nsg27 at case.edu  Sun Aug 25 18:30:15 2019
From: nsg27 at case.edu (Nicholas Georgescu)
Date: Sun, 25 Aug 2019 18:30:15 -0400
Subject: [Numpy-discussion] ENH: Add Moving Average Function
Message-ID: <58478F5E-3390-4C6D-8AA4-0B8724FC079A@getmailspring.com>

Hi all,

I opened a Pull Request (https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fpull%2F13923&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn) to include this package in numpy (https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/1?redirect=https%3A%2F%2Fpypi.org%2Fproject%2Fmvgavg%2F&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn), along with the associated sliding window function in this PR (https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/2?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fissues%2F7753&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn).

The function picks the fastest method to do a moving average if there is no weighting, but with weights it resorts to the second-fastest method which has an easier implementation. It also contains a binning option which cuts the number of points down by a factor of n rather than by subtracting n. The details are in the package documentation and PR.

Thanks,
Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190825/10ecf702/attachment.html>

From shoyer at gmail.com  Mon Aug 26 00:23:52 2019
From: shoyer at gmail.com (Stephan Hoyer)
Date: Sun, 25 Aug 2019 22:23:52 -0600
Subject: [Numpy-discussion] ENH: Add Moving Average Function
In-Reply-To: <58478F5E-3390-4C6D-8AA4-0B8724FC079A@getmailspring.com>
References: <58478F5E-3390-4C6D-8AA4-0B8724FC079A@getmailspring.com>
Message-ID: <CAEQ_TvddcVQ3wAynj0ng_RHVMFk2J2Byf1nznSRsMjbrWU1t5g@mail.gmail.com>

I would be very interested to see the ?sliding window view? function merged
into np.lib.stride_tricks.

I don?t think it makes sense to add a suite of dedicated functions for
sliding window calculations that wrap that function. If we are going to go
down the path of adding sliding window calculations into a NumPy, they
should use efficient algorithms, like those found in the ?bottleneck?
package.

Best,
Stephan

On Sun, Aug 25, 2019 at 3:33 PM Nicholas Georgescu <nsg27 at case.edu> wrote:

> Hi all,
>
> I opened a Pull Request
> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fpull%2F13923&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn> to
> include this package in numpy
> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/1?redirect=https%3A%2F%2Fpypi.org%2Fproject%2Fmvgavg%2F&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>,
> along with the associated sliding window function in this PR
> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/2?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fissues%2F7753&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>
> .
>
> The function picks the fastest method to do a moving average if there is
> no weighting, but with weights it resorts to the second-fastest method
> which has an easier implementation.  It also contains a binning option
> which cuts the number of points down by a factor of n rather than by
> subtracting n.  The details are in the package documentation and PR.
>
> Thanks,
> Nicholas
> [image: Sent from Mailspring]
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190825/da1fe9e1/attachment.html>

From toddrjen at gmail.com  Mon Aug 26 11:37:45 2019
From: toddrjen at gmail.com (Todd)
Date: Mon, 26 Aug 2019 11:37:45 -0400
Subject: [Numpy-discussion] ENH: Add Moving Average Function
In-Reply-To: <CAEQ_TvddcVQ3wAynj0ng_RHVMFk2J2Byf1nznSRsMjbrWU1t5g@mail.gmail.com>
References: <58478F5E-3390-4C6D-8AA4-0B8724FC079A@getmailspring.com>
 <CAEQ_TvddcVQ3wAynj0ng_RHVMFk2J2Byf1nznSRsMjbrWU1t5g@mail.gmail.com>
Message-ID: <CAFpSVpK3n4kE8JuuAoDzmYR0qRRdayROZnf4Fr5UO8EpQbskSg@mail.gmail.com>

I think having some function for common cases like moving average and
spectrogram would be good.  Having a jumping-off point and simple reference
for testing against could encourage someone to make a faster implementation
down the road.

-Todd

On Mon, Aug 26, 2019 at 12:24 AM Stephan Hoyer <shoyer at gmail.com> wrote:

> I would be very interested to see the ?sliding window view? function
> merged into np.lib.stride_tricks.
>
> I don?t think it makes sense to add a suite of dedicated functions for
> sliding window calculations that wrap that function. If we are going to go
> down the path of adding sliding window calculations into a NumPy, they
> should use efficient algorithms, like those found in the ?bottleneck?
> package.
>
> Best,
> Stephan
>
> On Sun, Aug 25, 2019 at 3:33 PM Nicholas Georgescu <nsg27 at case.edu> wrote:
>
>> Hi all,
>>
>> I opened a Pull Request
>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fpull%2F13923&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn> to
>> include this package in numpy
>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/1?redirect=https%3A%2F%2Fpypi.org%2Fproject%2Fmvgavg%2F&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>,
>> along with the associated sliding window function in this PR
>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/2?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fissues%2F7753&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>
>> .
>>
>> The function picks the fastest method to do a moving average if there is
>> no weighting, but with weights it resorts to the second-fastest method
>> which has an easier implementation.  It also contains a binning option
>> which cuts the number of points down by a factor of n rather than by
>> subtracting n.  The details are in the package documentation and PR.
>>
>> Thanks,
>> Nicholas
>> [image: Sent from Mailspring]
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190826/2cd61913/attachment-0001.html>

From charlesr.harris at gmail.com  Mon Aug 26 21:25:12 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 26 Aug 2019 19:25:12 -0600
Subject: [Numpy-discussion] NumPy 1.17.1 released
Message-ID: <CAB6mnxKTxuxHrw+5=cEyafgxi6bzV0sEDVJR8Q6WAs0x9-tRXg@mail.gmail.com>

Hi All,

On behalf of the NumPy team I am pleased to announce that NumPy 1.17.1 has
been released. This release contains a number of fixes for bugs reported
against NumPy 1.17.0 along with a few documentation and build
improvements.  The Python versions supported are 3.5-3.7, note that Python
2.7 has been dropped.  Python 3.8b3 should work with the released source
packages, but there are no future guarantees.

Downstream developers should use Cython >= 0.29.13 for Python 3.8 support
and OpenBLAS >= 3.7 to avoid wrong results on the Skylake architecture. The
NumPy wheels on PyPI are built from the OpenBLAS development branch in
order to avoid those problems. Wheels for this release can be downloaded
from PyPI <https://pypi.org/project/numpy/1.17.1/>, source archives and
release notes are available from Github
<https://github.com/numpy/numpy/releases/tag/v1.17.1>.

*Contributors*

A total of 17 people contributed to this release.  People with a "+" by
their
names contributed a patch for the first time.


   - Alexander Jung +
   - Allan Haldane
   - Charles Harris
   - Eric Wieser
   - Giuseppe Cuccu +
   - Hiroyuki V. Yamazaki
   - J?r?mie du Boisberranger
   - Kmol Yuan +
   - Matti Picus
   - Max Bolingbroke +
   - Maxwell Aladago +
   - Oleksandr Pavlyk
   - Peter Andreas Entschev
   - Sergei Lebedev
   - Seth Troisi +
   - Vladimir Pershin +
   - Warren Weckesser


*Pull requests merged*

A total of 24 pull requests were merged for this release.


   - gh-14156: TST: Allow fuss in testing strided/non-strided exp/log loops
   - gh-14157: BUG: avx2_scalef_ps must be static
   - gh-14158: BUG: Remove stray print that causes a SystemError on python
   3.7.
   - gh-14159: BUG: Fix DeprecationWarning in python 3.8.
   - gh-14160: BLD: Add missing gcd/lcm definitions to npy_math.h
   - gh-14161: DOC, BUILD: cleanups and fix (again) 'build dist'
   - gh-14166: TST: Add 3.8-dev to travisCI testing.
   - gh-14194: BUG: Remove the broken clip wrapper (Backport)
   - gh-14198: DOC: Fix hermitian argument docs in svd.
   - gh-14199: MAINT: Workaround for Intel compiler bug leading to failing
   test
   - gh-14200: TST: Clean up of test_pocketfft.py
   - gh-14201: BUG: Make advanced indexing result on read-only subclass
   writeable...
   - gh-14236: BUG: Fixed default BitGenerator name
   - gh-14237: ENH: add c-imported modules for freeze analysis in np.random
   - gh-14296: TST: Pin pytest version to 5.0.1
   - gh-14301: BUG: Fix leak in the f2py-generated module init and
   `PyMem_Del`...
   - gh-14302: BUG: Fix formatting error in exception message
   - gh-14307: MAINT: random: Match type of SeedSequence.pool_size to
   DEFAULT_POOL_SIZE.
   - gh-14308: BUG: Fix numpy.random bug in platform detection
   - gh-14309: ENH: Enable huge pages in all Linux builds
   - gh-14330: BUG: Fix segfault in `random.permutation(x)` when x is a
   string.
   - gh-14338: BUG: don't fail when lexsorting some empty arrays (#14228)
   - gh-14339: BUG: Fix misuse of .names and .fields in various places
   (backport...
   - gh-14345: BUG: fix behavior of structured_to_unstructured on
   non-trivial...
   - gh-14350: REL: Prepare 1.17.1 release

Cheers,

Charles Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190826/52e83299/attachment.html>

From hameerabbasi at yahoo.com  Tue Aug 27 04:45:16 2019
From: hameerabbasi at yahoo.com (Hameer Abbasi)
Date: Tue, 27 Aug 2019 10:45:16 +0200
Subject: [Numpy-discussion] PyData/Sparse 0.8.0
Message-ID: <A1A7DEF8-90A4-4B5D-8A9C-B76CC3C88404@yahoo.com>

(Apologies for the cross-posting)

 

Hello, everyone! PyData/Sparse 0.8.0 has been released. It is available on PyPI (and a source distribution and a universal binary) and conda-forge.
Changelog with credits: https://sparse.pydata.org/en/0.8.0/changelog.html
Documentation: https://sparse.pydata.org/en/0.8.0/
Source: https://github.com/pydata/sparse/releases/tag/0.8.0
 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/0cd43cfd/attachment.html>

From jjmo at dtu.dk  Tue Aug 27 07:18:42 2019
From: jjmo at dtu.dk (=?UTF-8?Q?Jens_J=c3=b8rgen_Mortensen?=)
Date: Tue, 27 Aug 2019 13:18:42 +0200
Subject: [Numpy-discussion] Calling BLAS functions from Python
Message-ID: <cceed78e-292a-00df-69e2-f03b889c1b8d@dtu.dk>

Hi!

I'm trying to use dgemm, zgemm and friends from scipy.linalg.blas to 
multiply matrices efficiently.  As an example, I'd like to do:

     c += a.dot(b)

using whatever BLAS scipy is linked to and I want to avoid copies of 
large matrices.  This works the way I want it:

 >>> import numpy as np
 >>> from scipy.linalg.blas import dgemm
 >>> a = np.ones((2, 3), order='F')
 >>> b = np.ones((3, 4), order='F')
 >>> c = np.zeros((2, 4), order='F')
 >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
array([[3., 3., 3., 3.],
        [3., 3., 3., 3.]])
 >>> print(c)
[[3. 3. 3. 3.]
  [3. 3. 3. 3.]]

but if c is not contiguous, then c is not overwritten:

 >>> c = np.zeros((7, 4), order='F')[:2, :]
 >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
array([[3., 3., 3., 3.],
        [3., 3., 3., 3.]])
 >>> print(c)
[[0. 0. 0. 0.]
  [0. 0. 0. 0.]]

Which is also what the docs say, but I think the raw BLAS function dgemm 
could do the update of c in-place by setting LDC=7.  See here:

     http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f.html

Is there a way to call the raw BLAS function from Python?

I found this capsule thing, but I don't know if there is a way to call 
that (maybe using ctypes):

 >>> from scipy.linalg import cython_blas
 >>> cython_blas.__pyx_capi__['dgemm']
<capsule object "void (char *, char *, int *, int *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *)" at 0x7f06fe1d2ba0>

Best,
Jens J?rgen

From deak.andris at gmail.com  Tue Aug 27 08:49:45 2019
From: deak.andris at gmail.com (Andras Deak)
Date: Tue, 27 Aug 2019 14:49:45 +0200
Subject: [Numpy-discussion] Calling BLAS functions from Python
In-Reply-To: <cceed78e-292a-00df-69e2-f03b889c1b8d@dtu.dk>
References: <cceed78e-292a-00df-69e2-f03b889c1b8d@dtu.dk>
Message-ID: <CAMEWA4NoWN6JDYLcuc2UYjDbOsb2i1JiyFeB81o_7+gQaBSDyg@mail.gmail.com>

On Tue, Aug 27, 2019 at 1:18 PM Jens J?rgen Mortensen <jjmo at dtu.dk> wrote:
>
> Hi!
>
> I'm trying to use dgemm, zgemm and friends from scipy.linalg.blas to
> multiply matrices efficiently.  As an example, I'd like to do:
>
>      c += a.dot(b)
>
> using whatever BLAS scipy is linked to and I want to avoid copies of
> large matrices.  This works the way I want it:
(snip)

Hi,

This is not a direct answer to your question, but are you only trying
to use low-level BLAS or the sake of memory, or are there other
considerations? I'm not certain but I think `a.dot` will call BLAS for
matrices (hence its speed and multithreaded capabilities), so CPU time
might already be optimal. As for memory, most numpy functions (and
definitely numpy arithmetic) can be used with functions in the main
numpy namespace, using the `out` keyword to specify an existing array
in which to print. So for your simple example,

>>> import numpy as np
... a = np.ones((2, 3), order='F')
... b = np.ones((3, 4), order='F')
... c = np.zeros((7, 4), order='F')[:2, :]
... np.add(c, a.dot(b), out=c)
array([[3., 3., 3., 3.],
       [3., 3., 3., 3.]])

>>> c
array([[3., 3., 3., 3.],
       [3., 3., 3., 3.]])

As you can see non-contiguous arrays Just Work?. You will still create
a temporary array for `a.dot(b)` but I'm not sure you can spare that.
Would low-level BLAS allow you to reduce memory at that step as well?
And is there other motivation for you to go down to the metal?
Regards,

Andr?s



>
>  >>> import numpy as np
>  >>> from scipy.linalg.blas import dgemm
>  >>> a = np.ones((2, 3), order='F')
>  >>> b = np.ones((3, 4), order='F')
>  >>> c = np.zeros((2, 4), order='F')
>  >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
>  >>> print(c)
> [[3. 3. 3. 3.]
>   [3. 3. 3. 3.]]
>
> but if c is not contiguous, then c is not overwritten:
>
>  >>> c = np.zeros((7, 4), order='F')[:2, :]
>  >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
>  >>> print(c)
> [[0. 0. 0. 0.]
>   [0. 0. 0. 0.]]
>
> Which is also what the docs say, but I think the raw BLAS function dgemm
> could do the update of c in-place by setting LDC=7.  See here:
>
>      http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f.html
>
> Is there a way to call the raw BLAS function from Python?
>
> I found this capsule thing, but I don't know if there is a way to call
> that (maybe using ctypes):
>
>  >>> from scipy.linalg import cython_blas
>  >>> cython_blas.__pyx_capi__['dgemm']
> <capsule object "void (char *, char *, int *, int *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *)" at 0x7f06fe1d2ba0>
>
> Best,
> Jens J?rgen
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

From jjmo at dtu.dk  Tue Aug 27 10:19:11 2019
From: jjmo at dtu.dk (=?UTF-8?Q?Jens_J=c3=b8rgen_Mortensen?=)
Date: Tue, 27 Aug 2019 16:19:11 +0200
Subject: [Numpy-discussion] Calling BLAS functions from Python
In-Reply-To: <CAMEWA4NoWN6JDYLcuc2UYjDbOsb2i1JiyFeB81o_7+gQaBSDyg@mail.gmail.com>
References: <cceed78e-292a-00df-69e2-f03b889c1b8d@dtu.dk>
 <CAMEWA4NoWN6JDYLcuc2UYjDbOsb2i1JiyFeB81o_7+gQaBSDyg@mail.gmail.com>
Message-ID: <6071ec1b-54eb-6114-d87b-1f3d3f9f49b2@dtu.dk>

On 8/27/19 2:49 PM, Andras Deak wrote:
> On Tue, Aug 27, 2019 at 1:18 PM Jens J?rgen Mortensen <jjmo at dtu.dk> wrote:
>>
>> Hi!
>>
>> I'm trying to use dgemm, zgemm and friends from scipy.linalg.blas to
>> multiply matrices efficiently.  As an example, I'd like to do:
>>
>>       c += a.dot(b)
>>
>> using whatever BLAS scipy is linked to and I want to avoid copies of
>> large matrices.  This works the way I want it:
> (snip)
> 
> Hi,
> 
> This is not a direct answer to your question, but are you only trying
> to use low-level BLAS or the sake of memory, or are there other
> considerations? I'm not certain but I think `a.dot` will call BLAS for
> matrices (hence its speed and multithreaded capabilities), so CPU time
> might already be optimal. As for memory, most numpy functions (and
> definitely numpy arithmetic) can be used with functions in the main
> numpy namespace, using the `out` keyword to specify an existing array
> in which to print. So for your simple example,
> 
>>>> import numpy as np
> ... a = np.ones((2, 3), order='F')
> ... b = np.ones((3, 4), order='F')
> ... c = np.zeros((7, 4), order='F')[:2, :]
> ... np.add(c, a.dot(b), out=c)
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
> 
>>>> c
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
> 
> As you can see non-contiguous arrays Just Work?. You will still create
> a temporary array for `a.dot(b)` but I'm not sure you can spare that.
> Would low-level BLAS allow you to reduce memory at that step as well?
> And is there other motivation for you to go down to the metal?
> Regards,

Thanks for your suggestion.  What I really need is something like:

     c += constant * a.dot(b.conj())

where the matrices have dtype=complex.  The BLAS zgemm() function does 
that without copies.  I also need zherk() for this:

     c += constant a.dot(a.conj().T)

You *can* do that with numpy, but I need both time and memory 
efficiency.  So, yes, I want to go down to the metal.  At the moment, 
the code I'm working on has an interface to BLAS, but I'm trying to see 
if we can use scipy's interface in order to simplify installation of our 
code.

Jens J?rgen

> Andr?s
> 
> 
> 
>>
>>   >>> import numpy as np
>>   >>> from scipy.linalg.blas import dgemm
>>   >>> a = np.ones((2, 3), order='F')
>>   >>> b = np.ones((3, 4), order='F')
>>   >>> c = np.zeros((2, 4), order='F')
>>   >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
>> array([[3., 3., 3., 3.],
>>          [3., 3., 3., 3.]])
>>   >>> print(c)
>> [[3. 3. 3. 3.]
>>    [3. 3. 3. 3.]]
>>
>> but if c is not contiguous, then c is not overwritten:
>>
>>   >>> c = np.zeros((7, 4), order='F')[:2, :]
>>   >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
>> array([[3., 3., 3., 3.],
>>          [3., 3., 3., 3.]])
>>   >>> print(c)
>> [[0. 0. 0. 0.]
>>    [0. 0. 0. 0.]]
>>
>> Which is also what the docs say, but I think the raw BLAS function dgemm
>> could do the update of c in-place by setting LDC=7.  See here:
>>
>>       http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f.html
>>
>> Is there a way to call the raw BLAS function from Python?
>>
>> I found this capsule thing, but I don't know if there is a way to call
>> that (maybe using ctypes):
>>
>>   >>> from scipy.linalg import cython_blas
>>   >>> cython_blas.__pyx_capi__['dgemm']
>> <capsule object "void (char *, char *, int *, int *, int *,
>> __pyx_t_5scipy_6linalg_11cython_blas_d *,
>> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
>> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
>> __pyx_t_5scipy_6linalg_11cython_blas_d *,
>> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *)" at 0x7f06fe1d2ba0>
>>
>> Best,
>> Jens J?rgen
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 


From jjmo at dtu.dk  Tue Aug 27 10:39:09 2019
From: jjmo at dtu.dk (=?UTF-8?Q?Jens_J=c3=b8rgen_Mortensen?=)
Date: Tue, 27 Aug 2019 16:39:09 +0200
Subject: [Numpy-discussion] Fwd: Re:  Calling BLAS functions from Python
In-Reply-To: <6071ec1b-54eb-6114-d87b-1f3d3f9f49b2@dtu.dk>
References: <6071ec1b-54eb-6114-d87b-1f3d3f9f49b2@dtu.dk>
Message-ID: <715f28d7-bdbb-f64a-684a-ebce3fb2d85d@dtu.dk>

Sorry!  Stupid me, asking scipy questions on numpy-discussion.  Now 
continuing on scipy-user.  Any help is much appreciated.  See short 
numpy-discussion thread here: 
https://mail.python.org/pipermail/numpy-discussion/2019-August/079945.html


Hi!

I'm trying to use dgemm, zgemm and friends from scipy.linalg.blas to 
multiply matrices efficiently.  As an example, I'd like to do:

     c += a.dot(b)

using whatever BLAS scipy is linked to and I want to avoid copies of 
large matrices.  This works the way I want it:

 >>> import numpy as np
 >>> from scipy.linalg.blas import dgemm
 >>> a = np.ones((2, 3), order='F')
 >>> b = np.ones((3, 4), order='F')
 >>> c = np.zeros((2, 4), order='F')
 >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
array([[3., 3., 3., 3.],
        [3., 3., 3., 3.]])
 >>> print(c)
[[3. 3. 3. 3.]
  [3. 3. 3. 3.]]

but if c is not contiguous, then c is not overwritten:

 >>> c = np.zeros((7, 4), order='F')[:2, :]
 >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
array([[3., 3., 3., 3.],
        [3., 3., 3., 3.]])
 >>> print(c)
[[0. 0. 0. 0.]
  [0. 0. 0. 0.]]

Which is also what the docs say, but I think the raw BLAS function dgemm 
could do the update of c in-place by setting LDC=7.  See here:

     http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f.html

Is there a way to call the raw BLAS function from Python?

I found this capsule thing, but I don't know if there is a way to call 
that (maybe using ctypes):

 >>> from scipy.linalg import cython_blas
 >>> cython_blas.__pyx_capi__['dgemm']
<capsule object "void (char *, char *, int *, int *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, 
__pyx_t_5scipy_6linalg_11cython_blas_d *, int *)" at 0x7f06fe1d2ba0>

Best,
Jens J?rgen

From sebastian at sipsolutions.net  Tue Aug 27 10:54:42 2019
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Tue, 27 Aug 2019 09:54:42 -0500
Subject: [Numpy-discussion] NumPy Community Meeting Wednesday, Aug. 28
Message-ID: <9469dff37af6423cde5a4c42b4b5666e9ddaec38.camel@sipsolutions.net>

Hi all,

There will be a NumPy Community meeting Wednesday August 28 at 11 am
Pacific Time. Everyone is invited to join in and edit the work-in-
progress meeting topics and notes:
https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both

Best wishes

Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: NumPy_Community_Call.ics
Type: text/calendar
Size: 3264 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/f7f60866/attachment.ics>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/f7f60866/attachment.sig>

From ilhanpolat at gmail.com  Tue Aug 27 14:04:27 2019
From: ilhanpolat at gmail.com (Ilhan Polat)
Date: Tue, 27 Aug 2019 20:04:27 +0200
Subject: [Numpy-discussion] Fwd: Re: Calling BLAS functions from Python
In-Reply-To: <715f28d7-bdbb-f64a-684a-ebce3fb2d85d@dtu.dk>
References: <6071ec1b-54eb-6114-d87b-1f3d3f9f49b2@dtu.dk>
 <715f28d7-bdbb-f64a-684a-ebce3fb2d85d@dtu.dk>
Message-ID: <CAEBuzr8O1mOBag2cKjb3gmahdgSZH2RyAXuYJoWiOnu93PA7Cg@mail.gmail.com>

The inplace overwriting is done if f2py can forward the original array down
to the low level.

When it is not contiguous then it has to somehow marshall the view into a
compatible array and that is when the inbetween array is formed. And also
that array can also be overwritten but that would not be the original view
you started with. Hence it is kind of a convenience cost you pay.

Cython might be a better option for you such that you can pass things
around via memory views and cython wrappers of BLAS.

On Tue, Aug 27, 2019, 16:40 Jens J?rgen Mortensen <jjmo at dtu.dk> wrote:

> Sorry!  Stupid me, asking scipy questions on numpy-discussion.  Now
> continuing on scipy-user.  Any help is much appreciated.  See short
> numpy-discussion thread here:
> https://mail.python.org/pipermail/numpy-discussion/2019-August/079945.html
>
>
> Hi!
>
> I'm trying to use dgemm, zgemm and friends from scipy.linalg.blas to
> multiply matrices efficiently.  As an example, I'd like to do:
>
>      c += a.dot(b)
>
> using whatever BLAS scipy is linked to and I want to avoid copies of
> large matrices.  This works the way I want it:
>
>  >>> import numpy as np
>  >>> from scipy.linalg.blas import dgemm
>  >>> a = np.ones((2, 3), order='F')
>  >>> b = np.ones((3, 4), order='F')
>  >>> c = np.zeros((2, 4), order='F')
>  >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
>  >>> print(c)
> [[3. 3. 3. 3.]
>   [3. 3. 3. 3.]]
>
> but if c is not contiguous, then c is not overwritten:
>
>  >>> c = np.zeros((7, 4), order='F')[:2, :]
>  >>> dgemm(1.0, a, b, 1.0, c, 0, 0, 1)
> array([[3., 3., 3., 3.],
>         [3., 3., 3., 3.]])
>  >>> print(c)
> [[0. 0. 0. 0.]
>   [0. 0. 0. 0.]]
>
> Which is also what the docs say, but I think the raw BLAS function dgemm
> could do the update of c in-place by setting LDC=7.  See here:
>
>      http://www.netlib.org/lapack/explore-html/d7/d2b/dgemm_8f.html
>
> Is there a way to call the raw BLAS function from Python?
>
> I found this capsule thing, but I don't know if there is a way to call
> that (maybe using ctypes):
>
>  >>> from scipy.linalg import cython_blas
>  >>> cython_blas.__pyx_capi__['dgemm']
> <capsule object "void (char *, char *, int *, int *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *,
> __pyx_t_5scipy_6linalg_11cython_blas_d *, int *)" at 0x7f06fe1d2ba0>
>
> Best,
> Jens J?rgen
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/bcc5069a/attachment.html>

From ralf.gommers at gmail.com  Tue Aug 27 18:57:03 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 27 Aug 2019 15:57:03 -0700
Subject: [Numpy-discussion] NumPy logo refresh
Message-ID: <CABL7CQjNyEi2hK=RXPTg-toqcmYVisqOrJBaRyLs8JcyAuuxtw@mail.gmail.com>

Hi all,

At https://github.com/numpy/numpy.org/issues/37 we have an update to the
NumPy logo, changing the colors for more contrast / a more modern look.
Please have a look!

What would also be helpful is to know who made this logo, and with what
tool. We'd like to produce a new SVG, but doing that from the old SVG
turned out to be problematic. So for now it's PNG only.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/5631b26d/attachment.html>

From charlesr.harris at gmail.com  Tue Aug 27 22:43:42 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 27 Aug 2019 20:43:42 -0600
Subject: [Numpy-discussion] NumPy 1.16.5 released
Message-ID: <CAB6mnx+hF8-_pP3ctoZ-2HG0a3FW_wSEGPJrzXXW12+1bTSYnQ@mail.gmail.com>

Hi All,

On behalf of the NumPy team I am pleased to announce that NumPy 1.16.5  has
been released. This release fixes bugs reported against the 1.16.4 release
and backports several enhancements from master that seem appropriate for
the LTS release series that is the last to support Python 2.7.

Downstream developers should use Cython >= 0.29.2 and OpenBLAS >= 3.7 to
avoid wrong results on the Skylake architecture. The NumPy wheels on PyPI
are built from the OpenBLAS development branch in order to avoid those
problems. Wheels for this release can be downloaded from PyPI
<https://pypi.org/project/numpy/1.16.5/>, source archives and release notes
are available from Github
<https://github.com/numpy/numpy/releases/tag/v1.16.5>.

*Contributors*

A total of 18 people contributed to this release.  People with a "+" by
their
names contributed a patch for the first time.

   - Alexander Shadchin
   - Allan Haldane
   - Bruce Merry +
   - Charles Harris
   - Colin Snyder +
   - Dan Allan +
   - Emile +
   - Eric Wieser
   - Grey Baker +
   - Maksim Shabunin +
   - Marten van Kerkwijk
   - Matti Picus
   - Peter Andreas Entschev +
   - Ralf Gommers
   - Richard Harris +
   - Sebastian Berg
   - Sergei Lebedev +
   - Stephan Hoyer



*Pull requests merged*
A total of 23 pull requests were merged for this release.

   - gh-13742: ENH: Add project URLs to setup.py
   - gh-13823: TEST, ENH: fix tests and ctypes code for PyPy
   - gh-13845: BUG: use npy_intp instead of int for indexing array
   - gh-13867: TST: Ignore DeprecationWarning during nose imports
   - gh-13905: BUG: Fix use-after-free in boolean indexing
   - gh-13933: MAINT/BUG/DOC: Fix errors in _add_newdocs
   - gh-13984: BUG: fix byte order reversal for datetime64[ns]
   - gh-13994: MAINT,BUG: Use nbytes to also catch empty descr during
   allocation
   - gh-14042: BUG: np.array cleared errors occured in
   PyMemoryView_FromObject
   - gh-14043: BUG: Fixes for Undefined Behavior Sanitizer (UBSan) errors.
   - gh-14044: BUG: ensure that casting to/from structured is properly
   checked.
   - gh-14045: MAINT: fix histogram*d dispatchers
   - gh-14046: BUG: further fixup to histogram2d dispatcher.
   - gh-14052: BUG: Replace contextlib.suppress for Python 2.7
   - gh-14056: BUG: fix compilation of 3rd party modules with
   Py_LIMITED_API...
   - gh-14057: BUG: Fix memory leak in dtype from dict contructor
   - gh-14058: DOC: Document array_function at a higher level.
   - gh-14084: BUG, DOC: add new recfunctions to `__all__`
   - gh-14162: BUG: Remove stray print that causes a SystemError on python
   3.7
   - gh-14297: TST: Pin pytest version to 5.0.1.
   - gh-14322: ENH: Enable huge pages in all Linux builds
   - gh-14346: BUG: fix behavior of structured_to_unstructured on
   non-trivial...
   - gh-14382: REL: Prepare for the NumPy 1.16.5 release.

Cheers,

Charles Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/58b2a4c5/attachment.html>

From ralf.gommers at gmail.com  Tue Aug 27 23:12:08 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 27 Aug 2019 20:12:08 -0700
Subject: [Numpy-discussion] ENH: Add Moving Average Function
In-Reply-To: <CAFpSVpK3n4kE8JuuAoDzmYR0qRRdayROZnf4Fr5UO8EpQbskSg@mail.gmail.com>
References: <58478F5E-3390-4C6D-8AA4-0B8724FC079A@getmailspring.com>
 <CAEQ_TvddcVQ3wAynj0ng_RHVMFk2J2Byf1nznSRsMjbrWU1t5g@mail.gmail.com>
 <CAFpSVpK3n4kE8JuuAoDzmYR0qRRdayROZnf4Fr5UO8EpQbskSg@mail.gmail.com>
Message-ID: <CABL7CQjMg5=p+JHOFonOh-BJ8WkxVyXTdEeZmk5-T7VWw3PZZw@mail.gmail.com>

On Mon, Aug 26, 2019 at 8:38 AM Todd <toddrjen at gmail.com> wrote:

> I think having some function for common cases like moving average and
> spectrogram would be good.  Having a jumping-off point and simple reference
> for testing against could encourage someone to make a faster implementation
> down the road.
>

This could also be done as a docstring example in the sliding window view
function. It's pretty straightforward, the moving average function in PR
13923 (fnjn_mvgavg) is ~5 lines of code.

That way we don't get stuck with a function that's not as efficient as it
should be, and we can point from there to `bottleneck` and/or something
else that's a high-quality implementation.

Cheers,
Ralf



>
> -Todd
>
> On Mon, Aug 26, 2019 at 12:24 AM Stephan Hoyer <shoyer at gmail.com> wrote:
>
>> I would be very interested to see the ?sliding window view? function
>> merged into np.lib.stride_tricks.
>>
>> I don?t think it makes sense to add a suite of dedicated functions for
>> sliding window calculations that wrap that function. If we are going to go
>> down the path of adding sliding window calculations into a NumPy, they
>> should use efficient algorithms, like those found in the ?bottleneck?
>> package.
>>
>> Best,
>> Stephan
>>
>> On Sun, Aug 25, 2019 at 3:33 PM Nicholas Georgescu <nsg27 at case.edu>
>> wrote:
>>
>>> Hi all,
>>>
>>> I opened a Pull Request
>>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fpull%2F13923&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn> to
>>> include this package in numpy
>>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/1?redirect=https%3A%2F%2Fpypi.org%2Fproject%2Fmvgavg%2F&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>,
>>> along with the associated sliding window function in this PR
>>> <https://link.getmailspring.com/link/58478F5E-3390-4C6D-8AA4-0B8724FC079A at getmailspring.com/2?redirect=https%3A%2F%2Fgithub.com%2Fnumpy%2Fnumpy%2Fissues%2F7753&recipient=bnVtcHktZGlzY3Vzc2lvbkBweXRob24ub3Jn>
>>> .
>>>
>>> The function picks the fastest method to do a moving average if there is
>>> no weighting, but with weights it resorts to the second-fastest method
>>> which has an easier implementation.  It also contains a binning option
>>> which cuts the number of points down by a factor of n rather than by
>>> subtracting n.  The details are in the package documentation and PR.
>>>
>>> Thanks,
>>> Nicholas
>>> [image: Sent from Mailspring]
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190827/86e1d40e/attachment-0001.html>

From hameerabbasi at yahoo.com  Wed Aug 28 04:18:32 2019
From: hameerabbasi at yahoo.com (Hameer Abbasi)
Date: Wed, 28 Aug 2019 10:18:32 +0200
Subject: [Numpy-discussion] NEP 31: Context-local and global overrides of
 the NumPy API
Message-ID: <86E1E11A-0350-4335-BEBA-0E61A7A5F5FB@yahoo.com>

Hello,

 

Me, Ralf Gommers and Peter Bell (both cc?d) have come up with a proposal on how to solve the array creation and duck array problems. The solution is outlined in NEP-31, currently in the form of a PR, [1]

Following the high level discussion in NEP-22. [2]

 

It would be nice to get some feedback.

 

Best regards,

Hameer Abbasi

 

[1] https://github.com/numpy/numpy/pull/14389

[2] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190828/37c7a594/attachment.html>

From albuscode at gmail.com  Thu Aug 29 00:41:09 2019
From: albuscode at gmail.com (Inessa Pawson)
Date: Thu, 29 Aug 2019 00:41:09 -0400
Subject: [Numpy-discussion] NumPy Community Survey
Message-ID: <CAPBcLCFjtQcV5b_-h6JSw-pfaBmN5mVJw-zGHiRu8agxNx_J8w@mail.gmail.com>

You know that NumPy is essential to the Python community. The NumPy team
wants you to know that YOU, our user and developer community, are essential
to us. That?s why we are putting together a team to create the inaugural
NumPy Community Survey.
We hope feedback will provide insights that will help us to guide better
decision-making about the development of NumPy as software and community.
For more information about the proposed survey please refer to
github.com/numpy/numpy-surveys .

*Call for Contributions*
We are looking for volunteers experienced in survey design and translating
English into Spanish, Portuguese, Russian, Hindi, Chinese and other
languages.

If you?d like to learn more about these volunteer opportunities, or
additional ways to support NumPy, feel free to reach out to our community
coordinators at numpy-team at googlegroups.com or join us on Slack
numpy-team.slack.com (email to numpy-team at googlegroups.com for an invite
first).

-- 
Every good wish,
Inessa Pawson
NumPy Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190829/548affff/attachment.html>

From ralf.gommers at gmail.com  Thu Aug 29 01:34:36 2019
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 28 Aug 2019 22:34:36 -0700
Subject: [Numpy-discussion] NumPy Community Survey
In-Reply-To: <CAPBcLCFjtQcV5b_-h6JSw-pfaBmN5mVJw-zGHiRu8agxNx_J8w@mail.gmail.com>
References: <CAPBcLCFjtQcV5b_-h6JSw-pfaBmN5mVJw-zGHiRu8agxNx_J8w@mail.gmail.com>
Message-ID: <CABL7CQgx0nnHDM_y18vZ4GPdtjaVuiNC17V7W5sEvoKF0yfEXQ@mail.gmail.com>

On Wed, Aug 28, 2019 at 9:42 PM Inessa Pawson <albuscode at gmail.com> wrote:

> You know that NumPy is essential to the Python community. The NumPy team
> wants you to know that YOU, our user and developer community, are essential
> to us. That?s why we are putting together a team to create the inaugural
> NumPy Community Survey.
> We hope feedback will provide insights that will help us to guide better
> decision-making about the development of NumPy as software and community.
> For more information about the proposed survey please refer to
> github.com/numpy/numpy-surveys .
>

Thanks Inessa! I think this survey is going to be super valuable.


> *Call for Contributions*
> We are looking for volunteers experienced in survey design and translating
> English into Spanish, Portuguese, Russian, Hindi, Chinese and other
> languages.
>
> If you?d like to learn more about these volunteer opportunities, or
> additional ways to support NumPy, feel free to reach out to our community
> coordinators at numpy-team at googlegroups.com or join us on Slack
> numpy-team.slack.com (email to numpy-team at googlegroups.com for an invite
> first).
>

Just a note for people who have not seen this list and Slack group before:
we recently added these [1] to
https://numpy.org/devdocs/dev/index.html#contributing-to-numpy as
alternatives to this list and GitHub (which are still preferred)
specifically for people who are interested in contributing but are hesitant
to inquire on a list with many hundreds of people on it.

Cheers,
Ralf

[1] https://github.com/numpy/numpy/pull/14027
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190828/04ea3021/attachment.html>

From matti.picus at gmail.com  Thu Aug 29 02:03:05 2019
From: matti.picus at gmail.com (Matti Picus)
Date: Thu, 29 Aug 2019 09:03:05 +0300
Subject: [Numpy-discussion] Allowing Dependabot access to the numpy repo
Message-ID: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190829/0fc51a7c/attachment.html>

From jni at fastmail.com  Thu Aug 29 02:12:02 2019
From: jni at fastmail.com (Juan Nunez-Iglesias)
Date: Thu, 29 Aug 2019 08:12:02 +0200
Subject: [Numpy-discussion] Allowing Dependabot access to the numpy repo
In-Reply-To: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>
References: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>
Message-ID: <C6AB0263-1607-4190-89D2-032574D14C44@fastmail.com>

I?m confused about why it needs write access to code... if I were doing this for scikit-image I would possibly clone the code to a new repo.

> On 29 Aug 2019, at 8:03 am, Matti Picus <matti.picus at gmail.com> wrote:
> 
> In PR 14378 https://github.com/numpy/numpy/pull/14378 I moved all our python test dependencies to a test_requirements.txt file (for building numpy the only requirement is cython). This is worthy since it unifies the different "pip install" commands across the different CI systems we use. Additionally, there are services that monitor the file and will issue a PR if any of those packages have a new release, so we can test out new versions of dependencies in a controlled fashion. Someone suggested Dependabot (thanks Ryan), which turns out to be run by a company bought by github itself.
> 
> 
> 
> When signing up for the service, it asks for permissions: https://pasteboard.co/IuTeWNz.png. The service is in use by other projects like cpython. Does it seem OK to sign up for this service?
> 
> 
> 
> Matti
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190829/3ca16aa5/attachment-0001.html>

From rmay31 at gmail.com  Thu Aug 29 04:44:49 2019
From: rmay31 at gmail.com (Ryan May)
Date: Thu, 29 Aug 2019 02:44:49 -0600
Subject: [Numpy-discussion] Allowing Dependabot access to the numpy repo
In-Reply-To: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>
References: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>
Message-ID: <CAKH0P+UQi+SQSL5XOGb3C9x0+yLDg2fF8y5_w6=fMQ5CXFqgxQ@mail.gmail.com>

Hi,

The answer to why Dependabot needs write permission seems to be to be able
to work with private repos:

https://github.com/dependabot/feedback/issues/22

There doesn't seem to be any way around it... :(

Ryan

On Thu, Aug 29, 2019 at 12:04 AM Matti Picus <matti.picus at gmail.com> wrote:

> In PR 14378 https://github.com/numpy/numpy/pull/14378 I moved all our
> python test dependencies to a test_requirements.txt file (for building
> numpy the only requirement is cython). This is worthy since it unifies the
> different "pip install" commands across the different CI systems we use.
> Additionally, there are services that monitor the file and will issue a PR
> if any of those packages have a new release, so we can test out new
> versions of dependencies in a controlled fashion. Someone suggested
> Dependabot (thanks Ryan), which turns out to be run by a company bought by
> github itself.
>
>
> When signing up for the service, it asks for permissions:
> https://pasteboard.co/IuTeWNz.png. The service is in use by other
> projects like cpython. Does it seem OK to sign up for this service?
>
>
> Matti
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>


-- 
Ryan May
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190829/456c8319/attachment.html>

From njs at pobox.com  Thu Aug 29 05:07:25 2019
From: njs at pobox.com (Nathaniel Smith)
Date: Thu, 29 Aug 2019 02:07:25 -0700
Subject: [Numpy-discussion] Allowing Dependabot access to the numpy repo
In-Reply-To: <CAKH0P+UQi+SQSL5XOGb3C9x0+yLDg2fF8y5_w6=fMQ5CXFqgxQ@mail.gmail.com>
References: <51892585-ba6e-94dd-fb73-9d1091231939@gmail.com>
 <CAKH0P+UQi+SQSL5XOGb3C9x0+yLDg2fF8y5_w6=fMQ5CXFqgxQ@mail.gmail.com>
Message-ID: <CAPJVwBkEz8GgeOL3+_AfXYUXv6ZkUwb9s+uiYxhfbX_86Gvcuw@mail.gmail.com>

AFAICT all these services work by creating branches inside your repo and
then making a PR from that ? they don't make their own forks. (Which makes
some sense when you consider they would need tens of thousands of forked
epos for all the projects they work with.)

I don't think there's any need to worry about giving GitHub Inc. (dba
Dependabot) write permissions to a GitHub repo, though.

You do maybe want to set up CI so that it doesn't run on these branches,
since it will also run on the PRs, and running CI twice on the same branch
is slow and wasteful.

-n

On Thu, Aug 29, 2019, 01:45 Ryan May <rmay31 at gmail.com> wrote:

> Hi,
>
> The answer to why Dependabot needs write permission seems to be to be able
> to work with private repos:
>
> https://github.com/dependabot/feedback/issues/22
>
> There doesn't seem to be any way around it... :(
>
> Ryan
>
> On Thu, Aug 29, 2019 at 12:04 AM Matti Picus <matti.picus at gmail.com>
> wrote:
>
>> In PR 14378 https://github.com/numpy/numpy/pull/14378 I moved all our
>> python test dependencies to a test_requirements.txt file (for building
>> numpy the only requirement is cython). This is worthy since it unifies the
>> different "pip install" commands across the different CI systems we use.
>> Additionally, there are services that monitor the file and will issue a PR
>> if any of those packages have a new release, so we can test out new
>> versions of dependencies in a controlled fashion. Someone suggested
>> Dependabot (thanks Ryan), which turns out to be run by a company bought by
>> github itself.
>>
>>
>> When signing up for the service, it asks for permissions:
>> https://pasteboard.co/IuTeWNz.png. The service is in use by other
>> projects like cpython. Does it seem OK to sign up for this service?
>>
>>
>> Matti
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
>
> --
> Ryan May
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190829/1fa5a8d4/attachment.html>

From hameerabbasi at yahoo.com  Thu Aug 29 22:26:36 2019
From: hameerabbasi at yahoo.com (Hameer Abbasi)
Date: Fri, 30 Aug 2019 04:26:36 +0200
Subject: [Numpy-discussion] NEP 31: Context-local and global overrides
 of the NumPy API
In-Reply-To: <1C914542-37B8-45D1-BB9B-42C37E6B1096@yahoo.com>
References: <1C914542-37B8-45D1-BB9B-42C37E6B1096@yahoo.com>
Message-ID: <3B0AB9FC-86CC-4E37-AE06-DA18899A222F@yahoo.com>

The full-text of the NEP is quoted below, apologies for not including it earlier:

 

============================================================

NEP 31 ? Context-local and global overrides of the NumPy API

============================================================

 

:Author: Hameer Abbasi <habbasi at quansight.com>

:Author: Ralf Gommers <rgommers at quansight.com>

:Author: Peter Bell <peterbell10 at live.co.uk>

:Status: Draft

:Type: Standards Track

:Created: 2019-08-22

 

 

Abstract

--------

 

This NEP proposes to make all of NumPy's public API overridable via an

extensible backend mechanism, using a library called ``uarray`` `[1]`_

 

``uarray`` provides global and context-local overrides, as well as a dispatch

mechanism similar to NEP-18 `[2]`_. First experiences with ``__array_function__``

show that it is necessary to be able to override NumPy functions that

*do not take an array-like argument*, and hence aren't overridable via

``__array_function__``. The most pressing need is array creation and coercion

functions - see e.g. NEP-30 `[9]`_.

 

This NEP proposes to allow, in an opt-in fashion, overriding any part of the NumPy API.

It is intended as a comprehensive resolution to NEP-22 `[3]`_, and obviates the need to

add an ever-growing list of new protocols for each new type of function or object that needs

to become overridable.

 

Motivation and Scope

--------------------

 

The motivation behind ``uarray`` is manyfold: First, there have been several attempts to allow

dispatch of parts of the NumPy API, including (most prominently), the ``__array_ufunc__`` protocol

in NEP-13 `[4]`_, and the ``__array_function__`` protocol in NEP-18 `[2]`_, but this has shown the

need for further protocols to be developed, including a protocol for coercion (see `[5]`_). The reasons

these overrides are needed have been extensively discussed in the references, and this NEP will not

attempt to go into the details of why these are needed. Another pain point requiring yet another

protocol is the duck-array protocol (see `[9]`_).

 

This NEP takes a more holistic approach: It assumes that there are parts of the API that need to be

overridable, and that these will grow over time. It provides a general framework and a mechanism to

avoid a design of a new protocol each time this is required.

 

This NEP proposes the following: That ``unumpy`` `[8]`_? becomes the recommended override mechanism

for the parts of the NumPy API not yet covered by ``__array_function__`` or ``__array_ufunc__``,

and that ``uarray`` is vendored into a new namespace within NumPy to give users and downstream dependencies

access to these overrides.? This vendoring mechanism is similar to what SciPy decided to do for

making ``scipy.fft`` overridable (see `[10]`_).

 

 

Detailed description

--------------------

 

**Note:** *This section will not attempt to explain the specifics or the mechanism of ``uarray``,

that is explained in the ``uarray`` documentation.* `[1]`_ *However, the NumPy community

will have input into the design of ``uarray``, and any backward-incompatible changes

will be discussed on the mailing list.*

 

The way we propose the overrides will be used by end users is::

 

??? import numpy.overridable as np

??? with np.set_backend(backend):

??????? x = np.asarray(my_array, dtype=dtype)

 

And a library that implements a NumPy-like API will use it in the following manner (as an example)::

 

??? import numpy.overridable as np

??? _ua_implementations = {}

 

??? __ua_domain__ = "numpy"

 

??? def __ua_function__(func, args, kwargs):

??????? fn = _ua_implementations.get(func, None)

??????? return fn(*args, **kwargs) if fn is not None else NotImplemented

 

??? def implements(ua_func):

??????? def inner(func):

??????????? _ua_implementations[ua_func] = func

??????????? return func

 

??????? return inner

 

??? @implements(np.asarray)

??? def asarray(a, dtype=None, order=None):

??????? # Code here

??????? # Either this method or __ua_convert__ must

??????? # return NotImplemented for unsupported types,

??????? # Or they shouldn't be marked as dispatchable.

 

??? # Provides a default implementation for ones and zeros.

??? @implements(np.full)

??? def full(shape, fill_value, dtype=None, order='C'):

??????? # Code here

 

The only change this NEP proposes at its acceptance, is to make ``unumpy`` the officially recommended

way to override NumPy. ``unumpy`` will remain a separate repository/package (which we propose to vendor

to avoid a hard dependency, and use the separate ``unumpy`` package only if it is installed)

rather than depend on for the time being), and will be developed

primarily with the input of duck-array authors and secondarily, custom dtype authors, via the usual

GitHub workflow. There are a few reasons for this:

 

* Faster iteration in the case of bugs or issues.

* Faster design changes, in the case of needed functionality.

* ``unumpy`` will work with older versions of NumPy as well.

* The user and library author opt-in to the override process,

? rather than breakages happening when it is least expected.

? In simple terms, bugs in ``unumpy`` mean that ``numpy`` remains

? unaffected.

 

Advantanges of ``unumpy`` over other solutions

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

``unumpy`` offers a number of advantanges over the approach of defining a new protocol for every

problem encountered: Whenever there is something requiring an override, ``unumpy`` will be able to

offer a unified API with very minor changes. For example:

 

* ``ufunc`` objects can be overridden via their ``__call__``, ``reduce`` and other methods.

* ``dtype`` objects can be overridden via the dispatch/backend mechanism, going as far as to allow

? ``np.float32`` et. al. to be overridden by overriding ``__get__``.

* Other functions can be overridden in a similar fashion.

* ``np.asduckarray`` goes away, and becomes ``np.asarray`` with a backend set.

* The same holds for array creation functions such as ``np.zeros``, ``np.empty`` and so on.

 

This also holds for the future: Making something overridable would require only minor changes to ``unumpy``.

 

Another promise ``unumpy`` holds is one of default implementations. Default implementations can be provided for

any multimethod, in terms of others. This allows one to override a large part of the NumPy API by defining

only a small part of it. This is to ease the creation of new duck-arrays, by providing default implementations of many

functions that can be easily expressed in terms of others, as well as a repository of utility functions

that help in the implementation of duck-arrays that most duck-arrays would require.

 

The last benefit is a clear way to coerce to a given backend, and a protocol for coercing not only arrays,

but also ``dtype`` objects and ``ufunc`` objects with similar ones from other libraries. This is due to the existence of

actual, third party dtype packages, and their desire to blend into the NumPy ecosystem (see `[6]`_). This is a separate

issue compared to the C-level dtype redesign proposed in `[7]`_, it's about allowing third-party dtype implementations to

work with NumPy, much like third-party array implementations.

 

Mixing NumPy and ``unumpy`` in the same file

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

Normally, one would only want to import only one of ``unumpy`` or ``numpy``, you would import it as ``np`` for

familiarity. However, there may be situations where one wishes to mix NumPy and the overrides, and there are

a few ways to do this, depending on the user's style::

 

??? import numpy.overridable as unumpy

??? import numpy as np

 

or::

 

??? import numpy as np

 

??? # Use unumpy via np.overridable

 

Related Work

------------

 

Previous override mechanisms

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

* NEP-18, the ``__array_function__`` protocol. `[2]`_

* NEP-13, the ``__array_ufunc__`` protocol. `[3]`_

 

Existing NumPy-like array implementations

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

* Dask: https://dask.org/

* CuPy: https://cupy.chainer.org/

* PyData/Sparse: https://sparse.pydata.org/

* Xnd: https://xnd.readthedocs.io/

* Astropy's Quantity: https://docs.astropy.org/en/stable/units/

 

Existing and potential consumers of alternative arrays

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

* Dask: https://dask.org/

* scikit-learn: https://scikit-learn.org/

* Xarray: https://xarray.pydata.org/

* TensorLy: http://tensorly.org/

 

Existing alternate dtype implementations

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

* ``ndtypes``: https://ndtypes.readthedocs.io/en/latest/

* Datashape: https://datashape.readthedocs.io

* Plum: https://plum-py.readthedocs.io/

 

Implementation

--------------

 

The implementation of this NEP will require the following steps:

 

* Implementation of ``uarray`` multimethods corresponding to the

? NumPy API, including classes for overriding ``dtype``, ``ufunc``

? and ``array`` objects, in the ``unumpy`` repository.

* Moving backends from ``unumpy`` into the respective array libraries.

 

Backward compatibility

----------------------

 

There are no backward incompatible changes proposed in this NEP.

 

 

Alternatives

------------

 

The current alternative to this problem is NEP-30 plus adding more protocols

(not yet specified) in addition to it.? Even then, some parts of the NumPy

API will remain non-overridable, so it's a partial alternative.

 

The main alternative to vendoring ``unumpy`` is to simply move it into NumPy

completely and not distribute it as a separate package. This would also achieve

the proposed goals, however we prefer to keep it a separate package for now,

for reasons already stated above.

 

 

Discussion

----------

 

* ``uarray`` blogpost: https://labs.quansight.org/blog/2019/07/uarray-update-api-changes-overhead-and-comparison-to-__array_function__/

* The discussion section of NEP-18: https://numpy.org/neps/nep-0018-array-function-protocol.html#discussion

* NEP-22: https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html

* Dask issue #4462: https://github.com/dask/dask/issues/4462

* PR #13046: https://github.com/numpy/numpy/pull/13046

* Dask issue #4883: https://github.com/dask/dask/issues/4883

* Issue #13831: https://github.com/numpy/numpy/issues/13831

* Discussion PR 1: https://github.com/hameerabbasi/numpy/pull/3

* Discussion PR 2: https://github.com/hameerabbasi/numpy/pull/4

 

 

References and Footnotes

------------------------

 

.. _[1]:

 

[1] uarray, A general dispatch mechanism for Python: https://uarray.readthedocs.io

 

.. _[2]:

 

[2] NEP 18 ? A dispatch mechanism for NumPy?s high level array functions: https://numpy.org/neps/nep-0018-array-function-protocol.html

 

.. _[3]:

 

[3] NEP 22 ? Duck typing for NumPy arrays ? high level overview: https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html

 

.. _[4]:

 

[4] NEP 13 ? A Mechanism for Overriding Ufuncs: https://numpy.org/neps/nep-0013-ufunc-overrides.html

 

.. _[5]:

 

[5] Reply to Adding to the non-dispatched implementation of NumPy methods: http://numpy-discussion.10968.n7.nabble.com/Adding-to-the-non-dispatched-implementation-of-NumPy-methods-tp46816p46874.html

 

.. _[6]:

 

[6] Custom Dtype/Units discussion: http://numpy-discussion.10968.n7.nabble.com/Custom-Dtype-Units-discussion-td43262.html

 

.. _[7]:

 

[7] The epic dtype cleanup plan: https://github.com/numpy/numpy/issues/2899

 

.. _[8]:

 

[8] unumpy: NumPy, but implementation-independent: https://unumpy.readthedocs.io

 

.. _[9]:

 

[9] NEP 30 ? Duck Typing for NumPy Arrays - Implementation: https://www.numpy.org/neps/nep-0030-duck-array-protocol.html

 

.. _[10]:

 

[10] http://scipy.github.io/devdocs/fft.html#backend-control

 

 

Copyright

---------

 

This document has been placed in the public domain.

 

 

From: Hameer Abbasi <hameerabbasi at yahoo.com>
Date: Wednesday, 28. August 2019 at 10:18
To: Discussion of Numerical Python <numpy-discussion at python.org>
Cc: "rgommers at quansight.com" <rgommers at quansight.com>, Peter Bell <peterbell10 at live.co.uk>
Subject: NEP 31: Context-local and global overrides of the NumPy API

 

Hello,

 

Me, Ralf Gommers and Peter Bell (both cc?d) have come up with a proposal on how to solve the array creation and duck array problems. The solution is outlined in NEP-31, currently in the form of a PR, [1]

Following the high level discussion in NEP-22. [2]

 

It would be nice to get some feedback.

 

Best regards,

Hameer Abbasi

 

[1] https://github.com/numpy/numpy/pull/14389

[2] https://numpy.org/neps/nep-0022-ndarray-duck-typing-overview.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190830/f171fa51/attachment-0001.html>