On Wed, 2019-06-26 at 17:22 -0400, Marten van Kerkwijk wrote:
Hi Ralf,
I realize you feel strongly that this whole thread is rehashing history, but I think it is worth pointing out that many seem to consider that the criterion for allowing backward incompatible changes, i.e., that "existing code is buggy or is consistently confusing many users", is actually fulfilled here.
Indeed, this appears true to such an extent that even those among the steering council do not agree: while the topic of this thread was about introducing *new* properties (because in the relevant issue I had suggested to Steward it was not possible to change .T), it was Eric who brought up the question whether we shouldn't just change `.T` after all. And in the relevant issue, Sebastian noted that "I am not quite convinced that we cannot change .T (at least in the sense of deprecation) myself", with Chuck chiming in that "I don't recall being in opposition, and I also think the current transpose is not what we want."
That makes three of your fellow steering council members who are not sure, despite all the previous discussions (of which Chuck surely has seen most - sorry, Chuck!).
It seems to me the only sure way in which we can avoid future discussions is to actually address the underlying problem. E.g., is the cost of deprecating & changing .T truly that much more than even having this discussion?
To me, I think what we have here is simply that if we want to do it, it will be an uphill battle. And an uphill battle may mean that we have to write something close to an NEP. Including seeing how much code blows up, e.g. by providing an environment variable switchable behaviour or so. I think it would be better to approach it from that side: What is necessary to be convincing enough? The problem of going from one behaviour to another (at least without an many-year waiting period) is real though, it not uncommon to leave scripts lying around for 5 years... So in that sense, I would agree that to really switch behaviour (not just error), it would need extremely careful analysis, which may not be feasible. OTOH, some other options, such as a new name or deprecations (or warning) do not have such fundamental problems. Quite honestly, I am not sure that deprecating `.T` completely for high dimensions is much more painful then e.g. the move of factorial in scipy, which forced me to modify a lot of my scripts (ok, its search+replace instead of replacing one line). We could go further of course, and say we do a "painful major" release at some point with things like py3k warnings and all. But we probably need more good reasons than a `.T`, and in-person discussions before even considering it. Best, Sebastian
All the best,
Marten
On Wed, Jun 26, 2019 at 4:18 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Jun 26, 2019 at 10:04 PM Kirill Balunov < kirillbalunov@gmail.com> wrote:
Only concerns #4 from Ilhan's list.
ср, 26 июн. 2019 г. в 00:01, Ralf Gommers <ralf.gommers@gmail.com
: [....]
Perhaps not full consensus between the many people with different opinions and interests. But for the first one, arr.T change: it's clear that this won't happen.
To begin with, I must admit that I am not familiar with the accepted policy of introducing changes to NumPy. But I find it quite nonconstructive just to say - it will not happen. What then is the point in the discussion?
There has been a *very* long discussion already, and several others on the same topic before. There are also long-standing ways of dealing with backwards compatibility - e.g. what Matthew said is not new, it's an agreed upon way of working. http://www.numpy.org/neps/nep-0023-backwards-compatibility.html lists some principles. That NEP is not yet accepted (it needs rework), but it gives a good idea of what does and does not go.
Between Juan's examples of valid use, and what Stephan and Matthew said, there's not much more to add. We're not going to change correct code for minor benefits.
I fully agree that any feature can find its use, valid or not is another question. Juan did not present these examples, but I will allow myself to assume that it is more correct to describe what is being done there as a permutation, and not a transpose. In addition, in the very next sentence, Juan adds that "These could be easily changed to .transpose() (honestly they probably should!)"
We're not going to change correct code for minor benefits.
It's fair, I personally have no preferences in both cases, the most important thing for me is that in the 2d case it works correctly. To be honest, until today, I thought that `.T` will raise for` ndim > 2`. At least that's what my experience told me. For example in
Matlab - Error using .' Transpose on ND array is not defined. Use PERMUTE instead.
Julia - transpose not defined for Array(Float64, 3). Consider using permutedims for higher-dimensional arrays.
Sympy - raise ValueError("array rank not 2")
Here, I agree with the authors that, to begin with, `transpose` is not the best name, since in general it doesn’t fit as an any mathematical definition (of course it will depend on what we take as an element) or a definition from linear algebra. Thus the name `transpose` only leads to confusion.
For a note about another suggestion - `.T` to mean a transpose of the last two dimensions, in Mathematica authors for some reason did the opposite (personally, I could not understand why they made such a choice :) ):
Transpose[list] transposes the first two levels in list.
I feel strongly that we should have the following policy:
* Under no circumstances should we make changes that mean that correct old code will give different results with new Numpy.
I find this overly strict rules that do not allow to evolve. I completely agree that a silent change in behavior is a disaster, that changing behavior (if it is not an error) in the same minor version (1.X.Y) is not acceptable, but I see no reason to extend this rule for a major version bump (2.A.B.), especially if it allows something to improve.
I'm sorry, you'll have to live with this rule. We've had lots of discussion about this rule in many concrete cases. When existing code is buggy or is consistently confusing many users, we can discuss. But in general changing old code to do something else is a terrible idea.
I would see such a rough version of a roadmap of change (I foresee my loneliness in this :)) Also considering this comment
Personally I would find any divergence between a.T and a.transpose() to be rather surprising.
it will be as follows:
1. in 1.18 add the `.permute` method to the array, with the same semantics as `.transpose`. 2. Starting from 1.18, emit `FutureWarning`, ` DeprectationWarning` for `.transpose` and advise replacing it with `.permute`. 3. Starting from 1.18 for `.T` with` ndim> 2`, emit a `FutureWarning`, with a note that in future versions the behavior will change. 4. In version 2, remove the `.transpose` and change the behavior for `.T`.
This is simply not enough. Many users will skip versions when upgrading. There must be an exceptionally good reason to change numerical results, and this simply is not one.
Cheers, Ralf
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion