Possible Deprecation of np.ediff1d
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in. Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1... Thoughts? Best wishes, Tyler
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly). -- Robert Kern
On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <robert.kern@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).
I was thinking we might want to add something to `np.diff`, maybe using `np.pad`. Chuck
There is already a patch to add such a feature to np.diff at https://github.com/numpy/numpy/pull/8206 On Mon, 27 Aug 2018 at 10:47 Charles R Harris <charlesr.harris@gmail.com> wrote:
On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <robert.kern@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).
I was thinking we might want to add something to `np.diff`, maybe using `np.pad`.
Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Mon, Aug 27, 2018 at 1:46 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <robert.kern@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/ numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/ questions/39014324/difference-between-numpy-ediff1d-and-diff
Thoughts?
Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).
same for me never seen, never used
I was thinking we might want to add something to `np.diff`, maybe using `np.pad`.
ediff1d avoids one extra temp array compared to simple padding. However, ediff1d pads the final, difference array instead of the original data array. AFAIR, I almost always need padding of the original array. eg as in the standard example of round tripping between cumsum and diff Josef
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Hi, it was originally used in other functions of arraysetops (culprit yours truly) which used to use the padding functionality. Nowadays the only use I can see git-grepping the numpy sources is in the tests, so +1 for deprecating. Cheers, r. On 08/27/2018 07:28 PM, Tyler Reddy wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Best wishes, Tyler
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Best wishes, Tyler
I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged. My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy: https://mail.python.org/pipermail/numpy-discussion/2018-July/078474.html I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects: - It appears in astropy, dask, pandas, pint, scipy and TensorFlow. - It used in six different internal projects Cheers, Stephan
On Mon, Aug 27, 2018 at 8:05 PM Stephan Hoyer <shoyer@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Best wishes, Tyler
I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged.
My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy: https://mail.python.org/pipermail/numpy-discussion/2018-July/078474.html
I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects: - It appears in astropy, dask, pandas, pint, scipy and TensorFlow. - It used in six different internal projects
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning. Chuck
On Tue, Aug 28, 2018 at 8:04 AM Charles R Harris <charlesr.harris@gmail.com> wrote:
On Mon, Aug 27, 2018 at 8:05 PM Stephan Hoyer <shoyer@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <tyler.je.reddy@gmail.com> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.
Related discussion on SO: https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1...
Thoughts?
Best wishes, Tyler
I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged.
My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy: https://mail.python.org/pipermail/numpy-discussion/2018-July/078474.html
I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects: - It appears in astropy, dask, pandas, pint, scipy and TensorFlow. - It used in six different internal projects
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.
Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead. In such a docstring warning we could include an easily searchable phrase that we start using for all such functions, but I don't think there's much value in that. Cheers, Ralf
On Tue, Aug 28, 2018 at 9:03 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably
have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.
Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead.
To be clear, I don't think np.ediff1d is particularly useful or necessary, despite these uses. Most of these uses don't even use the optional arguments, so the author was probably simply ignorant of np.diff. This is more or less inevitable for most corners of NumPy's API, given how many users we have. "PendingDeprecationWarning" is Python's built-in warning for signaling that something is obsolete but not deprecated yet. It might be appropriate to use in these cases. The default warning filters silence it for users, so it doesn't show up unless you're very aggressive about enabling all warnings.
In the meantime I'll make a PR to get rid of it from SciPy. We can also signal other libraries to do so. Anything frees up the already-very-crowded namespace of NumPy dot is worth it in my opinion. On Tue, Aug 28, 2018 at 7:40 PM Stephan Hoyer <shoyer@gmail.com> wrote:
On Tue, Aug 28, 2018 at 9:03 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should
probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.
Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead.
To be clear, I don't think np.ediff1d is particularly useful or necessary, despite these uses. Most of these uses don't even use the optional arguments, so the author was probably simply ignorant of np.diff. This is more or less inevitable for most corners of NumPy's API, given how many users we have.
"PendingDeprecationWarning" is Python's built-in warning for signaling that something is obsolete but not deprecated yet. It might be appropriate to use in these cases. The default warning filters silence it for users, so it doesn't show up unless you're very aggressive about enabling all warnings.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <shoyer@gmail.com> wrote: - It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it... So that's at least a few hits that do not count as arguments to keep it! I'm in favour of a PendingDeprecationWarning. -- Marten
On Tue, Aug 28, 2018 at 12:21 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <shoyer@gmail.com> wrote:
- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it...
So that's at least a few hits that do not count as arguments to keep it! I'm in favour of a PendingDeprecationWarning.
We should at least first merge the PR that adds the same padding behavior to np.diff before doing this, then such a warning could say to just use that and get unchanged behavior. Ralf
On Tue, Aug 28, 2018 at 12:27 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Tue, Aug 28, 2018 at 12:21 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <shoyer@gmail.com> wrote:
- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it...
So that's at least a few hits that do not count as arguments to keep it! I'm in favour of a PendingDeprecationWarning.
We should at least first merge the PR that adds the same padding behavior to np.diff before doing this, then such a warning could say to just use that and get unchanged behavior.
The proposed behavior for np.diff() is different, but it should solve the same use-cases.
participants (10)
-
Charles R Harris -
Eric Wieser -
Ilhan Polat -
josef.pktd＠gmail.com -
Marten van Kerkwijk -
Ralf Gommers -
Robert Cimrman -
Robert Kern -
Stephan Hoyer -
Tyler Reddy