[Numpy-discussion] backwards compatibility and deprecation policy NEP

[Ralf Gommers]

On Tue, Jul 24, 2018 at 8:07 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Sun, Jul 22, 2018 at 12:28 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> > On Sat, Jul 21, 2018 at 7:15 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >> Speaking of examples: I hate to say this because in general I think
> >> using examples is a great idea. But... I think you should delete most
> >> of these examples. The problem is scope creep: the goal for this NEP
> >> (IMO) should be to lay out the principles we use to think about these
> >> issues in general, but right now it comes across as trying to lay down
> >> a final resolution on lots of specific issues (including several where
> >> there are ongoing conversations). It ends up like trying to squish
> >> multiple NEPs into one, which makes it hard to discuss, and also
> >> distracts from the core purpose.
> >
> >
> > I'm not sure this is the best thing to do. I can remove a couple, but
> aiming
> > to be "totally uncontroversial" is almost impossible given the topic of
> the
> > NEP.
>
> Of course the NEP itself will have some things to discuss – but I
> think the discussion will be more productive if we can stay focused on
> the core part of the NEP, which is the general principles we use to
> evaluate each specific situation as it comes up. Look at how much of
> the discussion so far has gotten derailed onto topics like
> subclassing, submodules, etc.
>

The subclassing discussion was actually illuminating and useful. Maybe it
does deserve its own write-up somewhere though. Happy to remove that too.
Would then like to put it somewhere else - in the docs, another NEP, ...?

The submodules one I'd really like to keep.


> > The diag view example is important I think, it's the second most
> > discussed backwards compatibility issue next to histogram. I'm happy to
> > remove the statement on what should happen with it going forward though.
>
> It's the most discussed issue because it was the test case where we
> developed all these policies in the first place :-).


Pretty sure that's not true, we had policies long before that plus it was
not advertised as a test case for backwards compat (it's just an
improvement that someone wanted to implement). But well, I don't care
enough about this particular one to argue about it - I'll remove it.

I'm not sure it's
> particularly interesting aside from that, and that specific history
> ("let's come up with a transition plan for this feature that no-one
> actually cares about, b/c no-one cares about it so it's a good thing
> to use as a test case") is unlikely to be repeated.
>
> > Then, I think it's not unreasonable to draw a couple of hard lines. For
> > example, removing complete submodules like linalg or random has ended up
> on
> > some draft brainstorm roadmap list because someone (no idea who) put it
> > there after a single meeting. Clearly the cost-benefit of that is such
> that
> > there's no point even discussing that more, so I'd rather draw that line
> > here than every time someone open an issue. Very recent example:
> > https://github.com/numpy/numpy/issues/11457 (remove auto-import of
> > numpy.testing).
>
> I can see an argument for splitting random and linalg into their own
> modules, which numpy depends on and imports so that existing code
> doesn't break.


Me too, that could happen. But that's unrelated to backwards compatibility.

E.g. this might let people install an old version of
> random if they needed to reproduce some old results, or help us merge
> numpy and scipy's linalg modules into a single package. I agree though
> that making 'np.linalg' start raising AttributeError is a total
> non-starter.
>

It is, hence why I say above that I'd like to keep that example.


> >> Regarding the major version number thing: ugh do we really want to
> >> talk about this more. I'd probably leave it out of the NEP entirely.
> >> If it stays in, I think it needs a clearer description of what counts
> >> as a "major" change.
> >
> >
> > I think it has value to keep it, and that it's not really possible to
> come
> > up with a very clear description of "major". In particular, I'd like
> every
> > deprecation message to say "this deprecated feature will be removed by
> > release X.Y.0". At the moment we don't do that, so if users see a message
> > they don't know if a removal will happen next year, in the far future
> (2.0),
> > or never. The major version thing is quite useful to signal our intent.
> > Doesn't mean we need to exhaustively discuss when to do a 2.0 though, I
> > agree that that's not a very useful discussion right now.
>
> The problem is that "2.0" means a lot of different things to different
> people, not just "some future date to be determined", so using it that
> way will confuse people. Also, it's hard to predict when a deprecation
> will actually happen... it's very common that we adjust the schedule
> as we go (e.g. when we try to remove it and then discover it breaks
> everyone so we have to put it back for a while).
>
> I feel like it would be better to do this based on time


This does make sense to me.

-- like say
> "this will be removed <today + 1 year>" or something, and then it
> might take longer but not shorter?
>

You can't practically do "today", should be <version number of next release
when PR is merged + at least N years>. But yes that is useful, the point is
to give a clear indication and it's then easy for the user to figure out
when the earliest date is that the removal could happen.

Given that this is clear and avoids the version number discussion, I'm
happy to go with that and remove the major/minor version text.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180724/3f1af24c/attachment.html>