[Numpy-discussion] Put type annotations in NumPy proper?

[Roman Yurchak]

Thanks for re-starting this discussion, Stephan! I think there is 
definitely significant interest in this topic: 
https://github.com/numpy/numpy/issues/7370 is the issue with the largest 
number of user likes in the issue tracker (FWIW).

Having them in numpy, as opposed to a separate numpy-stubs repository 
would indeed be ideal from a user perspective. When looking into it in 
the past, I was never sure how well in sync numpy-stubs was. Putting 
aside ndarray, as more challenging, even annotations for numpy functions 
and method parameters with built-in types would help, as a start.

To add to the previously listed projects that would benefit from this, 
we are currently considering to start using some (minimal) type 
annotations in scikit-learn.

-- 
Roman Yurchak

On 24/03/2020 18:00, Stephan Hoyer wrote:
> When we started numpy-stubs [1] a few years ago, putting type 
> annotations in NumPy itself seemed premature. We still supported Python 
> 2, which meant that we would need to use awkward comments for type 
> annotations.
> 
> Over the past few years, using type annotations has become increasingly 
> popular, even in the scientific Python stack. For example, off-hand I 
> know that at least SciPy, pandas and xarray have at least part of their 
> APIs type annotated. Even without annotations for shapes or dtypes, it 
> would be valuable to have near complete annotations for NumPy, the 
> project at the bottom of the scientific stack.
> 
> Unfortunately, numpy-stubs never really took off. I can think of a few 
> reasons for that:
> 1. Missing high level guidance on how to write type annotations, 
> particularly for how (or if) to annotate particularly dynamic parts of 
> NumPy (e.g., consider __array_function__), and whether we should 
> prioritize strictness or faithfulness [2].
> 2. We didn't have a good experience for new contributors. Due to the 
> relatively low level of interest in the project, when a contributor 
> would occasionally drop in, I often didn't even notice their PR for a 
> few weeks.
> 3. Developing type annotations separately from the main codebase makes 
> them a little harder to keep in sync. This means that type annotations 
> couldn't serve their typical purpose of self-documenting code. Part of 
> this may be necessary for NumPy (due to our use of C extensions), but 
> large parts of NumPy's user facing APIs are written in Python. We no 
> longer support Python 2, so at least we no longer need to worry about 
> putting annotations in comments.
> 
> We eventually could probably use a formal NEP (or several) on how we 
> want to use type annotations in NumPy, but I think a good first step 
> would be to think about how to start moving the annotations from 
> numpy-stubs into numpy proper.
> 
> Any thoughts? Anyone interested in taking the lead on this?
> 
> Cheers,
> Stephan
> 
> [1] https://github.com/numpy/numpy-stubs
> [2] https://github.com/numpy/numpy-stubs/issues/12
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>