On Thu, Apr 9, 2020 at 6:54 PM Sebastian Berg
On Thu, 2020-04-09 at 13:52 +0200, Ralf Gommers wrote:
On Wed, Mar 4, 2020 at 1:22 AM Sebastian Berg < sebastian@sipsolutions.net> wrote:
The NumPy version should normally be older than other libraries, and if
NumPy updates the API so do the downstream implementers. E.g. dask may have to provide multiple version of the same function depending on the installed NumPy version, but that seems OK to me?
That seems unworkable, and I don't think any libraries do this. Coupling the semantics of a single Dask function to the installed numpy version is odd.
Is it all that odd? Libraries (not array providers) already need to test for NumPy version occasionally due to API changes, so they also have two versions of the same thing around (e.g. a fallback).
That's completely different, it's internal to a library and not visible to end users via different signatures/behavior. This simply would move the burden to the array-object implementer to
some degree. Assume that we have a versioned API in some form or another, it seems to me we either require:
module = np.get_array_module(..., api_version=2)
Yes, this is the version I was thinking about.
or define `module.__api_version__`.
Where the latter means that sklearn/SciPy may have to check `__api_version__` on every function call, while currently such checks usually happen at import time. On the other hand, the former means that sklearn/scipy can only opt-in to new API after 3+ years easily?
That's anyway the case, has very little to do with API versioning I think - it's simply determined by minimum NumPy version supported.
Saying that the NumPy version is what pins the api-version, is not much more than assuming/requiring that NumPy will be the least up-to-date package?
Of course it is unworkable to get 100% right in practice but are you saying that because it seems like an impractical approach,
Yes this, impractical and undesired. or because
the API surface is currently so large that, of course, we will never get it 100% right (but that is generally true, nobody will be able to implement NumPy 100% compatible)?
That's true too, we *don't want* anyone to start adding compat features for outdated or "wish we could deprecate" NumPy features.
`__array_function__` has same issue? If we change our API, Dask has to catch up.
Yes, that's true. The restricted API should be more stable than the whole NumPy API, otherwise no one will be able to be fully compatible. If SciPy expects it to be the old version though (based on
the NumPy import) it will incorrectly assume the old-api will be used.
That's not incorrect unless it's a backwards-incompatible change, which should be rare. Cheers, Ralf