[Numpy-discussion] The future of ndarray.diagonal()

[Benjamin Root]

Wasn't all of this discussed way back when the deprecation plan was made?
This was known to happen and was entirely the intent, right? What new
argument is there to deviate from the plan?

As for that particular blog post, I remember reading it back when it was
posted. I, again, sympathize with the author's plight, but I pointed out
that the reason for some of the changes he noted was because they could
cause bugs, which would mean that results could be wrong. Reproducibility
is nigh useless without a test suite to ensure the component parts are
reproducible on their own.

OTOH, there is an argument for slow, carefully-considered changes to APIs
(which I think the diagonal() changes were). As an example of a potentially
poor change is in matplotlib. We are starting to move to using properties,
away from get/setters(). In my upcoming book, I ran into a problem where I
needed to use an Artist's get_axes() or use its property "axes", but there
will only be one release of matplotlib where both of them will be valid. I
was faced with either using the get_axes() and have my code obsolete
sometime in the summer, use the propery, and have my code invalid for all
but the most recent version of matplotlib, or to have some version checking
code that would distract from the lesson at hand. I now think that a single
release cycle for deprecation of get_axes() was not a wise decision,
especially since the old code was merely verbose, not buggy.

To conclude, unless someone can present a *new* argument to deviate from
the diagonal() plan that was set a couple years ago, I don't see any reason
why the decisions that were agreed upon then are invalid now. The
pros-and-cons were weighed, and this particular con was known then and was
considered acceptable at that time.

Cheers!
Ben Root

On Sat, Jan 3, 2015 at 2:49 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On 1 Jan 2015 21:35, "Alexander Belopolsky" <ndarray at mac.com> wrote:
> >
> > A discussion [1] is currently underway at GitHub which will benefit from
> a larger forum.
> >
> > In version 1.9, the diagonal() method was changed to return a read-only
> (non-contiguous) view into the original array instead of a plain copy.
> Also, it has been announced [2] that in 1.10 the view will become
> read/write.
> >
> > A concern has now been raised [3] that this change breaks backward
> compatibility too much.
> >
> > Consider the following code:
> >
> > x = numy.eye(2)
> > d = x.diagonal()
> > d[0] = 2
> >
> > In 1.8, this code runs without errors and results in [2, 1] stored in
> array d.  In 1.9, this is an error.  With the current plan, in 1.10 this
> will become valid again, but the result will be different: x[0,0] will be 2
> while it is 1 in 1.8.
>
> Further context:
>
> In 1.7 and 1.8, the code above works as described, but also issues a
> visible-by-default warning:
>
> >>> np.__version__
> '1.7.2'
> >>> x = np.eye(2)
> >>> x.diagonal()[0] = 2
> __main__:1: FutureWarning: Numpy has detected that you (may be) writing to
> an array returned
> by numpy.diagonal or by selecting multiple fields in a record
> array. This code will likely break in the next numpy release --
> see numpy.diagonal or arrays.indexing reference docs for details.
> The quick fix is to make an explicit copy (e.g., do
> arr.diagonal().copy() or arr[['f0','f1']].copy()).
>
> 1.7 was released in Feb. 2013, ~22 months ago. (I'm not implying this
> number is particularly large or small, it's just something that I find
> useful to calculate when thinking about things like this.)
>
> The choice of "1.10" as the target for completing this change is
> more-or-less a strawman and we shouldn't feel bound by it. The schedule was
> originally written in between the 1.6 and 1.7 releases, when our release
> process was kinda broken and we had no idea what the future release
> schedule would look like (1.6 -> 1.7 ultimately ended up being a ~21 month
> gap). We've already adjusted the schedule for this deprecation once before
> (see issue #596: The original schedule called for the change to returning a
> ro-view to happen in 1.8, rather than 1.9 as it actually did). Now that our
> release frequency is higher, 1.11 might well be a more reasonable target
> than 1.10.
>
> As for the overall question, this is really a bigger question about what
> strategy we should use in general to balance between conservatism (which is
> a Good Thing) and making improvements (which is also a Good Thing). The
> post you cite brings this up explicitly:
>
> > [3] http://khinsen.wordpress.com/2014/09/12/the-state-of-numpy/
>
> I have huge respect for the problems and pain that Konrad describes in
> this blog post, but I really can't agree with the argument or the
> conclusions. His conclusion is that when it comes to compatibility breaks,
> slow-incremental-change is bad, and that we should instead prefer big
> all-at-once compatibility breaks like the Numeric->Numpy or Py2->Py3
> transitions. But when describing his own experiences that he uses to
> motivate this, he says:
>
> *"The two main dependencies of my code, NumPy and Python itself, did
> sometimes introduce incompatible changes (by design or as consequences of
> bug fixes) that required changes on my own code base, but they were
> surprisingly minor and never required more than about a day of work."*
>
> i.e., slow-incremental-change has actually worked well in his experience.
> (And in particular, the np.diagonal issue only comes in as an example to
> illustrate what he means by the phrase "slow continuous change" -- this
> particular change hasn't actually broken anything in his code.) OTOH the
> big problem that motivated his post was that his code is all written
> against the APIs of the ancient and long-abandoned Numeric project, and he
> finds the costs of transitioning them to the "new" numpy APIs to be
> prohibitively expensive, i.e. this big-bang transition broke his code. (It
> did manage to limp on for some years b/c numpy used to contain some
> compatibility code to emulate the Numeric API, but this doesn't really
> change the basic situation: there were two implementations of the API he
> needed -- numpy.numeric and Numeric itself -- and both implementations
> still exist in the sense that you can download them, but neither is usable
> because no-one's willing to maintain them anymore.) Maybe I'm missing
> something, but his data seems to be pi radians off from his conclusion.
>
> -n
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150103/509d14ff/attachment.html>