[Numpy-discussion] Changes to generalized ufunc core dimension checking

[Nathaniel Smith]

On Wed, Mar 16, 2016 at 7:32 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Mar 16, 2016 at 3:45 PM, Steve Waterbury <waterbug at pangalactic.us>
> wrote:
>>
>> On 03/16/2016 06:28 PM, Nathaniel Smith wrote:
>>>
>>> ... Sounds like a real deprecation cycle would have been better.
>>
>>
>> IMHO for a library as venerable and widely-used as Numpy, a
>> deprecation cycle is almost always better ... consider this a
>> lesson learned.
>
>
> Mandatory XKCD - https://xkcd.com/1172
>
> We recently had a discussion about a similar "nobody we know uses nor should
> use this api" situation in IPython, and ultimately decided that xkcd 1172
> would hit us, so opted in this case just for creating new cleaner APIs +
> possibly doing slow deprecation of the old stuff.
>
> For a widely used library, if the code exists then someone, somewhere
> depends on it, regardless of how broken or obscure you think the feature is.
> We just have to live with that.

Sure, the point is well taken, and we've been working hard to make
numpy's change policy more consistent, rigorous, and useful -- and
this remains very much a work in progress.

But engineering is fundamentally the art of making trade-offs, which
are always messy and context-dependent. I actually rather like XKCD
1172 because it's a Rorschach test -- you can just as easily read it
as saying that you should start by accepting that all changes will
break something, and then move on to the more interesting discussion
of which things are you going to break and what are the trade-offs.
:-)

Like, in this case: Our general policy is definitely to use a
deprecation cycle. And if you look at the discussions back in
September, we also considered options like deprecating-then-removing
the broadcasting, or adding a new gufunc-registration-API that would
enable the new behavior while preserving the old behavior for old
users. Both sound appealing initially. But they both also have serious
downsides: they mean preserving broken behavior, possibly
indefinitely, which *also* breaks users' code. Notice that if we add a
new API in 1.10, then most people won't actually switch until 1.10
becomes the *minimum* version they support, except they probably will
forget then too. And we have to maintain both APIs indefinitely. And
I'm dubious that the kind of anonymous corporate users who got broken
by this would have noticed a deprecation warning -- during the 1.11
cycle we had to go around pointing out some year+ old deprecation
warnings to all our really active and engaged downstreams, never mind
the ones we only hear about months later through Travis :-/. In this
particular case, as far as we could tell, all single existing users
were actually *broken* by the current behavior, so it was a question
of whether we should fix a bunch of real cases but risk breaking some
rare/theoretical ones, or should we leave a bunch of real cases broken
for a while in order to be gentler to the rare/theoretical ones. And
would we have ever even learned about these cases that Travis's
clients are running into if we hadn't broken things? And so forth.

It's obviously true that we make mistakes and should try to learn from
them to do better in the future, but I object to the idea that there
are simple and neat answers available :-).

(And sometimes in my more pessimistic moments I feel like a lot of the
conventional rules for change management are really technology for
shifting around blame :-/. "We had a deprecation period that you
didn't notice; your code broke the same either way, but our use of a
deprecation period means that now it's *your* fault". Or, in the
opposite situation: "Sure, this API doesn't work in the way that
anyone would actually expect or want, but if we fix it then it will be
*our* fault when existing code breaks -- OTOH if we leave the
brokenness there but document it, then it'll be *your* fault when you
-- totally predictably -- fall into the trap we've left for you". IMO
in both situations it's much healthier to skip the blame games and
take responsibility for the actual end result whatever it is, and if
that means that some kind of failure is inevitable, then oh well, lets
think about how to optimize our process for minimum net failure given
imperfect information and finite resources :-). Is there some way we
can help downstreams notice deprecations earlier? It's a lot easier to
measure the cost of making a change than of not making a change -- is
there some way we can gather more data to correct for this bias? IMO
*these* are the really interesting questions, and they're ones that
we've been actively working on.)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org