Deprecate matrices in 1.15 and remove in 1.17?
Hi All, I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices. For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132). What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear). All the best, Marten
+1 (not that my lurking vote should necessarily carry much weight). Rip it out asap. The existence of the matrix class has been literally the single biggest source of confusion and subtle bugs in my and my students' codes for years. Best, Chris
On Nov 30, 2017, at 9:23 AM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices. For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132).
What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear).
All the best,
Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
This is exactly what we did with the bokeh.charts deprecation. Moving to a separate projects was both a huge relief for the developers as well as a great way to focus and clarify expectations for users. Bryan
On Nov 30, 2017, at 10:20, Chris Laumann <chris.laumann@gmail.com> wrote:
+1 (not that my lurking vote should necessarily carry much weight). Rip it out asap.
The existence of the matrix class has been literally the single biggest source of confusion and subtle bugs in my and my students' codes for years.
Best, Chris
On Nov 30, 2017, at 9:23 AM, Marten van Kerkwijk <m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices. For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132).
What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear).
All the best,
Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Moving to a subpackage may indeed make more sense, though it might not help as much with getting rid of the hacks inside other parts of numpy to keep matrix working. In that respect it seems a bit different at least from weave. Then again, independently of whether we remove or release a separate package, it is probably best to start by moving all tests involving matrix to matrixlib/tests, so we can at least get a sense of what hacks are actually present. -- Marten
This would be really good to remove the apparent confusion. Moreover, I think cleanly explaining why using "np.matrix" is not a good idea *before* announcing the news would encourage people to accept this decision along the way. That would greatly reduce the sporadic "the devs are deprecating stuff as they see fit without asking us" sentiment. On Thu, Nov 30, 2017 at 6:00 PM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Moving to a subpackage may indeed make more sense, though it might not help as much with getting rid of the hacks inside other parts of numpy to keep matrix working. In that respect it seems a bit different at least from weave.
Then again, independently of whether we remove or release a separate package, it is probably best to start by moving all tests involving matrix to matrixlib/tests, so we can at least get a sense of what hacks are actually present.
-- Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Nov 30, 2017 09:24, "Marten van Kerkwijk" <m.h.vankerkwijk@gmail.com> wrote: Hi All, I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices. For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132). What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear). All the best, Marten I still think moving it out into its own package would be better, making it clear that anyone who cares about the class should step up because numpy developers will not do any additional work on it. Similar to how weave was handled with scipy. So simultaneous with the deprecation you release a package with the matrix class. Then people have until the deprecation period is over to port (which should just be a matter of changing the imports).
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices.
I don't think the matrix class was a bad idea at the time. numpy was the underdog, I came from GAUSS and Matlab and numpy arrays were just weird, especially loosing a dimension all the time and the heavy required use of np.newaxis. I guess nowadays kids don't learn `matrix` languages first anymore. recarrays are another half-hearted feature in numpy that is mostly obsolete with pandas and pandas_like DataFrames in other packages. (I don't mind the changes, but the deprecation cycle is often short, especially for users like me that update numpy only about every 3 main versions.) Josef
For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132).
What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear).
All the best,
Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
How would the community handle the scipy.sparse matrix subclasses? These are still in common use. Somewhat related: https://github.com/scipy/scipy/issues/8162 On Thu, Nov 30, 2017 at 1:13 PM, <josef.pktd@gmail.com> wrote:
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices.
I don't think the matrix class was a bad idea at the time.
numpy was the underdog, I came from GAUSS and Matlab and numpy arrays were just weird, especially loosing a dimension all the time and the heavy required use of np.newaxis. I guess nowadays kids don't learn `matrix` languages first anymore.
recarrays are another half-hearted feature in numpy that is mostly obsolete with pandas and pandas_like DataFrames in other packages.
(I don't mind the changes, but the deprecation cycle is often short, especially for users like me that update numpy only about every 3 main versions.)
Josef
For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132).
What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear).
All the best,
Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin <mrocklin@gmail.com> wrote:
How would the community handle the scipy.sparse matrix subclasses? These are still in common use.
Somewhat related: https://github.com/scipy/scipy/issues/8162
On Thu, Nov 30, 2017 at 1:13 PM, <josef.pktd@gmail.com> wrote:
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices.
I'd suggest any release in the next couple of years is fine,but the one where we drop Python 2 support is probably the worst choice. That's one of
They're not going anywhere for quite a while (until the sparse ndarrays materialize at least). Hence np.matrix needs to be moved, not deleted. We discussed this earlier this year: https://mail.python.org/pipermail/numpy-discussion/2017-January/076332.html the few things the core Python devs got 100% right with the Python 3 move: advocate that in the 2->3 transition packages would not make any API changes in order to make porting the least painful. Ralf
I don't think the matrix class was a bad idea at the time.
numpy was the underdog, I came from GAUSS and Matlab and numpy arrays were just weird, especially loosing a dimension all the time and the heavy required use of np.newaxis. I guess nowadays kids don't learn `matrix` languages first anymore.
recarrays are another half-hearted feature in numpy that is mostly obsolete with pandas and pandas_like DataFrames in other packages.
(I don't mind the changes, but the deprecation cycle is often short, especially for users like me that update numpy only about every 3 main versions.)
Josef
For 1.17, minimum python supposedly is >=3.5, so we will be guaranteed to have the matrix multiply operator @ available, and hence there is arguably even less of a case for keeping the matrix class; removing it would allow taking out quite a bit of accumulated special-casing (the immediate reasons for writing this were gh-10123 and 10132).
What do people think? If we do go in this direction, we might want to add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for python3; for python2 matrix would never disappear).
All the best,
Marten _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin <mrocklin@gmail.com> wrote:
How would the community handle the scipy.sparse matrix subclasses? These are still in common use.
They're not going anywhere for quite a while (until the sparse ndarrays materialize at least). Hence np.matrix needs to be moved, not deleted. We discussed this earlier this year: https://mail.python.org/ pipermail/numpy-discussion/2017-January/076332.html
Somewhat related: https://github.com/scipy/scipy/issues/8162
On Thu, Nov 30, 2017 at 1:13 PM, <josef.pktd@gmail.com> wrote:
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices.
I'd suggest any release in the next couple of years is fine,but the one where we drop Python 2 support is probably the worst choice. That's one of the few things the core Python devs got 100% right with the Python 3 move: advocate that in the 2->3 transition packages would not make any API changes in order to make porting the least painful.
Ralf
Agree, we don't want to pile in too many changes at once. I think the big sticking point is the sparse matrices in SciPy, even issuing a DeprecationWarning could be problematic as long as there are sparse matrices. May I suggest that we put together an NEP for the NumPy side of things? Ralf, does SciPy have a mechanism for proposing such changes? <snip> Chuck
On Thu, Nov 30, 2017, at 11:39, Charles R Harris wrote:
Agree, we don't want to pile in too many changes at once. I think the big sticking point is the sparse matrices in SciPy, even issuing a DeprecationWarning could be problematic as long as there are sparse matrices.
Could you explain what you mean by SciPy sparse matrices being a big sticking point? Stéfan
On Thu, Nov 30, 2017 at 11:39 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
I'd suggest any release in the next couple of years is fine,but the one where we drop Python 2 support is probably the worst choice. That's one of the few things the core Python devs got 100% right with the Python 3 move: advocate that in the 2->3 transition packages would not make any API changes in order to make porting the least painful.
Agree, we don't want to pile in too many changes at once. I think the big sticking point is the sparse matrices in SciPy, even issuing a DeprecationWarning could be problematic as long as there are sparse matrices. May I suggest that we put together an NEP for the NumPy side of things? Ralf, does SciPy have a mechanism for proposing such changes?
Agreed here as well... while I want to get rid of np.matrix as much as anyone, doing that anytime soon would be *really* disruptive. - There are tons of little scripts out there written by people who didn't know better; we do want them to learn not to use np.matrix but breaking all their scripts is a painful way to do that - There are major projects like scikit-learn that simply have no alternative to using np.matrix, because of scipy.sparse. So I think the way forward is something like: - Now or whenever someone gets together a PR: issue a PendingDeprecationWarning in np.matrix.__init__ (unless it kills performance for scikit-learn and friends), and put a big warning box at the top of the docs. The idea here is to not actually break anyone's code, but start to get out the message that we definitely don't think anyone should use this if they have any alternative. - After there's an alternative to scipy.sparse: ramp up the warnings, possibly all the way to FutureWarning so that existing scripts don't break but they do get noisy warnings - Eventually, if we think it will reduce maintenance costs: split it into a subpackage I expect that one way or another we'll be maintaining matrix for quite some time, and I agree with whoever said that most of the burden seems to be in keeping the rest of numpy working sensibly with it, so I don't think moving it into a subpackage is itself going to make a big different either way. To me the logic is more like, if/when we decide to actually break everyone's code by making `np.matrix` raise AttributeError, then we should probably provide some package they can import to get their code limping along again, and if we're going to do that anyway then probably we should split it out first and shake out any bugs before we make `np.matrix` start raising errors. But it's going to be quite some time until we reach the "break everyone's code" stage, given just how much code is out there using matrix, so there's no point in making detailed plans right now. -n -- Nathaniel J. Smith -- https://vorpus.org
On Thu, Nov 30, 2017, at 16:15, Nathaniel Smith wrote:
PendingDeprecationWarning in np.matrix.__init__ (unless it kills performance for scikit-learn and friends), and put a big warning box at the top of the docs. The idea here is to not actually break anyone's code, but start to get out the message that we definitely don't think anyone should use this if they have any alternative.
- After there's an alternative to scipy.sparse: ramp up the warnings, possibly all the way to FutureWarning so that existing scripts don't break but they do get noisy warnings
- Eventually, if we think it will reduce maintenance costs: split it into a subpackage
Can't we make `np.matrix` into a new package right now, and have NumPy depend on it internally? At that point, start warning users that they should also be using the external package, and eventually just remove the shim in NumPy. Stéfan
Hi Nathaniel, Thanks for the concrete suggestion: see https://github.com/numpy/numpy/pull/10142 I think this is useful independent of exactly how the eventual move to a new package would work; next step might be to collect all matrix tests in the `libmatrix` sub-module. All the best, Marten
- There are major projects like scikit-learn that simply have no alternative to using np.matrix, because of scipy.sparse.
Well, we have no love for np.matrix, we just use scipy.sparse and np.ndarray.
- After there's an alternative to scipy.sparse:
+1 for alternative to scipy.sparse before transition. Gaël
On Fri, Dec 1, 2017 at 8:39 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin <mrocklin@gmail.com> wrote:
How would the community handle the scipy.sparse matrix subclasses? These are still in common use.
They're not going anywhere for quite a while (until the sparse ndarrays materialize at least). Hence np.matrix needs to be moved, not deleted. We discussed this earlier this year: https://mail.python.org/piperm ail/numpy-discussion/2017-January/076332.html
Somewhat related: https://github.com/scipy/scipy/issues/8162
On Thu, Nov 30, 2017 at 1:13 PM, <josef.pktd@gmail.com> wrote:
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Hi All,
I wondered if the move to python3-only starting with numpy 1.17 would be a good reason to act on what we all seem to agree: that the matrix class was a bad idea, with its overriding of multiplication and lack of support for stacks of matrices.
I'd suggest any release in the next couple of years is fine,but the one where we drop Python 2 support is probably the worst choice. That's one of the few things the core Python devs got 100% right with the Python 3 move: advocate that in the 2->3 transition packages would not make any API changes in order to make porting the least painful.
Ralf
Agree, we don't want to pile in too many changes at once. I think the big sticking point is the sparse matrices in SciPy, even issuing a DeprecationWarning could be problematic as long as there are sparse matrices. May I suggest that we put together an NEP for the NumPy side of things? Ralf, does SciPy have a mechanism for proposing such changes?
Not an official one, but any NEP-like proposal can be considered. I imagine the NEP itself will have to say something about it anyway, so could as well put it all in there. Ralf
Hi Ralf, Sorry not to have recalled the previous thread. Your point about not doing things in the python 2->3 move makes sense; handy for me is no reason to give users an incentive not to move to python3 because their matrix-dependent code breaks. It does sound like, given the use of sparse, a separate package - or perhaps (temporary) inclusion in scipy - would be the way to go. In turn, collecting as much of the matrix tests and work-arounds together in the `matrixlib` would be the right first step. And, even better, to collect thoughts in a NEP. Now if only I had not written this while procrastinating on other things... All the best, Marten
On Thu, Nov 30, 2017 at 11:58 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Your point about not doing things in the python 2->3 move makes sense;
But this is NOT the 2->3 move -- numpy as been py3 compatible for years. At some point, it is a really good idea to deprecate some things. Personally, I think Matrix should have been deprecated a good while ago -- it never really worked well, and folks have been advised not to use it for years. But anyway, once we can count on having @ then there really is no reason to have Matrix, so it happens that dropping py2 support is the first time we can count on that. But this is really deprecating something when we stop support for py < 3.5, not the py2 to py3 transition. Remember that deprecating is different than dropping. If we want to keep Matrix around for one release after py2 is dropped, so that people can use it once they are "forced" to move to py3, OK, but let's get clear deprecation plan in place. Also -- we aren't requiring people to move to py3 -- we are only requiring people to move to py3 if they want the latest numpy features. One last note: Guido's suggestion that libraries not take py3 as an opportunity to change APIs was a good one, but it was also predicated on the fact that py2+p3 support was going to be needed for a good while. So this is really a different case. It's really a regular old deprecation -- you aren't going to have this feature in future numpy releases -- py2/3 has nothing to do with it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Chris, I'm easily convinced - yes, your argument makes sense too. Fortunately, at some level it doesn't affect what we do now. For 1.15 it should at least have a PendingDeprecationWarning. Since putting that in place means that all tests involving matrices now fail by default, it also becomes a good excuse to put all those tests together in `libmatrix`, so that we can set an exception to the warning=exception rule in one place. Hence, we automatically move closer to making it possible to take matrix out. In the meantime, for everybody, help with this initial step most welcome! PRs to master or to mine. (I'm at end of term and it's only procrastination that made me start this...) Thanks, all the best, Marten
On Sat, Dec 2, 2017 at 7:03 AM, Chris Barker <chris.barker@noaa.gov> wrote:
On Thu, Nov 30, 2017 at 11:58 AM, Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
Your point about not doing things in the python 2->3 move makes sense;
But this is NOT the 2->3 move -- numpy as been py3 compatible for years. At some point, it is a really good idea to deprecate some things.
Deprecating can be done whenever (at least from this point of view, not from the scipy.sparse situation), the original proposal and my reply were about removal in 1.17. Effective what will happen is that users want to upgrade to the release that drops py2 (for whatever reason), and at that point are forced to upgrade to py3. So it would be good if that release did not have other API breaks.
Personally, I think Matrix should have been deprecated a good while ago -- it never really worked well, and folks have been advised not to use it for years. But anyway, once we can count on having @ then there really is no reason to have Matrix, so it happens that dropping py2 support is the first time we can count on that. But this is really deprecating something when we stop support for py < 3.5, not the py2 to py3 transition.
Remember that deprecating is different than dropping. If we want to keep Matrix around for one release after py2 is dropped, so that people can use it once they are "forced" to move to py3, OK, but let's get clear deprecation plan in place.
Also -- we aren't requiring people to move to py3 -- we are only requiring people to move to py3 if they want the latest numpy features.
One last note: Guido's suggestion that libraries not take py3 as an opportunity to change APIs was a good one, but it was also predicated on the fact that py2+p3 support was going to be needed for a good while. So this is really a different case. It's really a regular old deprecation -- you aren't going to have this feature in future numpy releases -- py2/3 has nothing to do with it.
I think you're massively underestimating how many people are still on py2, are going to have to upgrade when we drop it, and how painful that is. Ralf
On Thu, Nov 30, 2017, at 10:13, josef.pktd@gmail.com wrote:
recarrays are another half-hearted feature in numpy that is mostly obsolete with pandas and pandas_like DataFrames in other packages.
I'm fully on board with factoring out np.matrix into a subpackage. But I would not touch structured arrays; they are quite useful, and sometimes perform surprisingly well compared to the other solutions around. Stéfan
On Thu, Nov 30, 2017 at 2:51 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Thu, Nov 30, 2017, at 10:13, josef.pktd@gmail.com wrote:
recarrays are another half-hearted feature in numpy that is mostly obsolete with pandas and pandas_like DataFrames in other packages.
I'm fully on board with factoring out np.matrix into a subpackage. But I would not touch structured arrays; they are quite useful, and sometimes perform surprisingly well compared to the other solutions around.
I think Josef specifically meant `recarrays`, which give access to elements of a structured array via attribute access. I'd tend to agree with him that those turned out not to be such a great idea. But (I think) nobody is arguing we should get rid of arrays with structured dtypes - I use them regularly myself too. -- Marten
On Thu, Nov 30, 2017, at 12:02, Marten van Kerkwijk wrote:
I think Josef specifically meant `recarrays`, which give access to elements of a structured array via attribute access. I'd tend to agree with him that those turned out not to be such a great idea. But (I think) nobody is arguing we should get rid of arrays with structured dtypes - I use them regularly myself too.
Ah, okay, that makes sense! Which reminds me: while these are quite useful, they're not always particularly pleasant to use. A good first improvement would be to allow columnar printing, and a few utility functions to give you some of the basic functionality of pandas (calculating descriptive statistics like mean, dropping NaN rows, some equivalent of groupby). All these are only a few lines of Python, but can be annoying to figure out. If this sounds appealing, I'd be willing to put together a small NEP. Stéfan
On 2017/11/30 12:00 PM, Stefan van der Walt wrote:
I think Josef specifically meant `recarrays`, which give access to elements of a structured array via attribute access. I'd tend to agree with him that those turned out not to be such a great idea. But (I
I have found recarrays to be useful, providing an alternative view that can be convenient. What is the problem with them? Eric
Unlike for matrix, it is not so much a problem as an unclear use case - the main thing they bring to structured dtype arrays is access by attribute, which is slower than just doing getting the field by its key. Anyway, I don't think anybody is suggesting to remove them - they're not a problem in the way matrix is, with its shape-mangling, etc. -- Marten
An NEP on utility functions for structured array definitely sounds appealing to me. On Thu, Nov 30, 2017 at 2:00 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
On Thu, Nov 30, 2017, at 12:02, Marten van Kerkwijk wrote:
I think Josef specifically meant `recarrays`, which give access to elements of a structured array via attribute access. I'd tend to agree with him that those turned out not to be such a great idea. But (I think) nobody is arguing we should get rid of arrays with structured dtypes - I use them regularly myself too.
Ah, okay, that makes sense!
Which reminds me: while these are quite useful, they're not always particularly pleasant to use. A good first improvement would be to allow columnar printing, and a few utility functions to give you some of the basic functionality of pandas (calculating descriptive statistics like mean, dropping NaN rows, some equivalent of groupby). All these are only a few lines of Python, but can be annoying to figure out. If this sounds appealing, I'd be willing to put together a small NEP.
Stéfan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
participants (15)
-
Bryan Van de ven -
Charles R Harris -
Chris Barker -
Chris Laumann -
Eric Firing -
Feng Yu -
Gael Varoquaux -
Ilhan Polat -
josef.pktd＠gmail.com -
Marten van Kerkwijk -
Matthew Rocklin -
Nathaniel Smith -
Ralf Gommers -
Stefan van der Walt -
Todd