[Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering

[Ralf Gommers]

On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett <matthew.brett at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg
> >> <sebastian at sipsolutions.net> wrote:
> >> > Hey
> >> >
> >> > On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote:
> >> >> Hi,
> >> >>
> >> >> On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >> <snip>
> >> >> > Maybe we should go through and rename "order" to something more
> >> >> > descriptive
> >> >> > in each case, so we'd have
> >> >> >   a.reshape(..., index_order="C")
> >> >> >   a.copy(memory_order="F")
> >> >> > etc.?
> >> >>
> >> >> I'd like to propose this instead:
> >> >>
> >> >> a.reshape(..., order="C")
> >> >> a.copy(layout="F")
> >> >>
> >> >
> >> > I actually like this, makes the point clearer that it has to do with
> >> > memory layout and implies contiguity, plus it is short and from the
> >> > numpy perspective copy, etc. are the ones that add additional info to
> >> > "order" and not reshape (because IMO memory order is something new
> users
> >> > should not worry about at first). A and K orders will still have their
> >> > quirks with np.array and copy=True/False, but for many functions they
> >> > are esoteric anyway.
> >> >
> >> > It will be one hell of a deprecation though, but I am +0.5 for adding
> an
> >> > alias for now (maybe someone knows an even better name?), but I think
> >> > that in this case, it probably really is better to wait with actual
> >> > deprecation warnings for a few versions, since it touches a *lot* of
> >> > code. Plus I think at the point of starting deprecation warnings (and
> >> > best earlier) numpy should provide an automatic fixer script...
> >> >
> >> > The only counter point that remains for me is the difficulty of
> >> > deprecation, since I think the new name idea is very clean. And this
> is
> >> > unfortunately even more invasive then the index_order proposal.
> >>
> >> I completely agree that we'd have to be gentle with the change.  The
> >> problem we'd want to avoid is people innocently using 'layout' and
> >> finding to their annoyance that the code doesn't work with other
> >> people's numpy.
> >>
> >> How about:
> >>
> >> Step 1:  'order' remains as named keyword, layout added as alias,
> >> comment on the lines of "layout will become the default keyword for
> >> this option in later versions of numpy; please consider updating any
> >> code that does not need to remain backwards compatible'.
> >>
> >> Step 2: default keyword becomes 'layout' with 'order' as alias,
> >> comment like "order is an alias for 'layout' to maintain backwards
> >> compatibility with numpy <= 1.7.1', please update any code that does
> >> not need to maintain backwards compatibility with these numpy
> >> versions'
> >>
> >> Step 3: Add deprecation warning for 'order', "order will be removed as
> >> an alias in future versions of numpy"
> >>
> >> Step 4: (distant future) Remove alias
> >>
> >> ?
> >
> >
> > A very strong -1 from me. Now we're talking about deprecation warnings
> and a
> > backwards compatibility break after all. I thought we agreed that this
> was a
> > very bad idea, so why are you proposing it now?
> >
> > Here's how I see it: deprecation of "order" is a no go. Therefore we have
> > two choices here:
> > 1. Simply document the current "order" keyword better and leave it at
> that.
> > 2. Add a "layout" (or "index_order") keyword, and live with both "order"
> and
> > "layout" keywords forever.
> >
> > (2) is at least as confusing as (1), more work and poor design.
> Therefore I
> > propose to go with (1).
>
> You are saying that deprecation of 'order' at any stage in the next 10
> years of numpy's lifetime is a no go?
>

For something like this? Yes.


> I think that is short-sighted and I think it will damage numpy.
>

It will damage numpy to be conservative and not change a name for a little
bit of clarity for some people that avoids reading the docs maybe a little
more carefully? There's a lot of things that can damage numpy, but this
isn't even close in my book. Too few developers, continuous backwards
compatibility issues, faster alternative libraries surpassing numpy -
that's the kind of thing that causes damage.


> Believe me, I have as much investment in backward compatibility as you
> do.  All the three libraries that I spend a long time maintaining need
> to test against old numpy versions - but - for heaven's sake - only
> back to numpy 1.2 or numpy 1.3.  We don't support Python 2.5 any more,
> and I don't think we need to maintain compatibility with Numeric
> either.
>

Really? This is from 3 months ago:
http://article.gmane.org/gmane.comp.python.numeric.general/52632. It's now
2013, we are probably dropping numarray compat in 1.8. Not exactly 10
years, but of the same order.


> If you are saying that we need to maintain compatibility for 10 years
> at a stretch, then we will have to accept that numpy will gradually
> decay into a legacy library, because it is certain that, if we stay
> static, someone else with more ambition will do a better job.
>
> There is a cost to being averse to any change at all, no matter how
> gradually it is managed.
>

It's a cost/benefit trade-off, yes. Breaking backwards compatibility for a
big step forward is sometimes necessary, in order to avoid decay as you
say. You seem to have lost sight of the little thing you're arguing for
though. There simply is no big step forward here.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130405/17f21705/attachment.html>