[Numpy-discussion] Syntax Improvement for Array Transpose

Wed Jun 26 17:22:47 EDT 2019

Hi Ralf,

I realize you feel strongly that this whole thread is rehashing history,
but I think it is worth pointing out that many seem to consider that the
criterion for allowing backward incompatible changes, i.e., that "existing
code is buggy or is consistently confusing many users", is actually
fulfilled here.

Indeed, this appears true to such an extent that even those among the
steering council do not agree: while the topic of this thread was about
introducing *new* properties (because in the relevant issue I had suggested
to Steward it was not possible to change .T), it was Eric who brought up
the question whether we shouldn't just change `.T` after all. And in the
relevant issue, Sebastian noted that "I am not quite convinced that we
cannot change .T (at least in the sense of deprecation) myself", with Chuck
chiming in that "I don't recall being in opposition, and I also think the
current transpose is not what we want."

That makes three of your fellow steering council members who are not sure,
despite all the previous discussions (of which Chuck surely has seen most -
sorry, Chuck!).

It seems to me the only sure way in which we can avoid future discussions
is to actually address the underlying problem. E.g., is the cost of
deprecating & changing .T truly that much more than even having this
discussion?

All the best,

Marten

On Wed, Jun 26, 2019 at 4:18 PM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Wed, Jun 26, 2019 at 10:04 PM Kirill Balunov <kirillbalunov at gmail.com>
> wrote:
>
>> Only concerns #4 from Ilhan's list.
>>
>> ср, 26 июн. 2019 г. в 00:01, Ralf Gommers <ralf.gommers at gmail.com>:
>>
>>>
>>> [....]
>>>
>>> Perhaps not full consensus between the many people with different
>>> opinions and interests. But for the first one, arr.T change: it's clear
>>> that this won't happen.
>>>
>>
>> To begin with, I must admit that I am not familiar with the accepted
>> policy of introducing changes to NumPy. But I find it quite
>> nonconstructive just to say - it will not happen. What then is the point
>> in the discussion?
>>
>
> There has been a *very* long discussion already, and several others on the
> same topic before. There are also long-standing ways of dealing with
> backwards compatibility - e.g. what Matthew said is not new, it's an agreed
> upon way of working.
> http://www.numpy.org/neps/nep-0023-backwards-compatibility.html lists
> some principles. That NEP is not yet accepted (it needs rework), but it
> gives a good idea of what does and does not go.
>
>
>>
>>
>>> Between Juan's examples of valid use, and what Stephan and Matthew said,
>>> there's not much more to add. We're not going to change correct code for
>>> minor benefits.
>>>
>>
>> I fully agree that any feature can find its use, valid or not is another
>> question. Juan did not present these examples, but I will allow myself
>> to assume that it is more correct to describe what is being done there as a
>> permutation, and not a transpose. In addition, in the very next
>> sentence, Juan adds that "These could be easily changed to .transpose()
>> (honestly they probably should!)"
>>
>> We're not going to change correct code for minor benefits.
>>>
>>
>> It's fair, I personally have no preferences in both cases, the most
>> important thing for me is that in the 2d case it works correctly. To be
>> honest, until today, I thought that `.T` will raise for` ndim > 2`. At
>> least that's what my experience told me. For example in
>>
>>     Matlab - Error using  .' Transpose on ND array is not defined. Use
>> PERMUTE instead.
>>
>>     Julia - transpose not defined for Array(Float64, 3). Consider using
>> permutedims for higher-dimensional arrays.
>>
>>     Sympy - raise ValueError("array rank not 2")
>>
>> Here, I agree with the authors that, to begin with, `transpose` is not
>> the best name, since in general it doesn’t fit as an any mathematical
>> definition (of course it will depend on what we take as an element) or a
>> definition from linear algebra. Thus the name `transpose` only leads to
>> confusion.
>>
>> For a note about another suggestion - `.T` to mean a transpose of the
>> last two dimensions, in Mathematica authors for some reason did the
>> opposite (personally, I could not understand why they made such a choice
>> :) ):
>>
>>     Transpose[list]
>>         transposes the first two levels in list.
>>
>>     I feel strongly that we should have the following policy:
>>>
>>>     * Under no circumstances should we make changes that mean that
>>> correct
>>>     old code will give different results with new Numpy.
>>>
>>
>> I find this overly strict rules that do not allow to evolve. I
>> completely agree that a silent change in behavior is a disaster, that
>> changing behavior (if it is not an error) in the same minor version (1.X.Y)
>> is not acceptable, but I see no reason to extend this rule for a major
>> version bump (2.A.B.),  especially if it allows something to improve.
>>
>
> I'm sorry, you'll have to live with this rule. We've had lots of
> discussion about this rule in many concrete cases. When existing code is
> buggy or is consistently confusing many users, we can discuss. But in
> general changing old code to do something else is a terrible idea.
>
>
>> I would see such a rough version of a roadmap of change (I foresee my
>> loneliness in this :)) Also considering this comment
>>
>>     Personally I would find any divergence between a.T and a.transpose()
>>>     to be rather surprising.
>>>
>>
>> it will be as follows:
>>
>> 1. in 1.18 add the `.permute` method to the array, with the same
>> semantics as `.transpose`.
>> 2. Starting from 1.18, emit  `FutureWarning`, ` DeprectationWarning` for
>> `.transpose` and advise replacing it with `.permute`.
>> 3. Starting from 1.18 for `.T` with` ndim> 2`, emit a `FutureWarning`,
>> with a note that in future versions the behavior will change.
>> 4. In version 2, remove the `.transpose` and change the behavior for `.T`.
>>
>
> This is simply not enough. Many users will skip versions when upgrading.
> There must be an exceptionally good reason to change numerical results, and
> this simply is not one.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190626/be3d86fc/attachment-0001.html>