[Numpy-discussion] Is there any official position on PEP484/mypy?

Fri Jul 29 09:31:16 EDT 2016

One thing that I have always wished for from a project like mypy is the
ability to annotate what the expected shape should be. Too often, I get a
piece of code from a coworker and it comes with no docstring explaining the
expected dimensions of the input arrays and what the output array is going
to be. What would be really awesome is the ability to do something like
annotate that "a" is MxN, and "b" is NxP, and that "c" is Px3. Even if the
linter can't really check to make sure that the shapes would be respected,
it would still be nice to have a common way of expressing the expected
shapes in this annotation format.

As for matplotlib, we would need to express much more complicated
annotations, because our API is so flexible. It would be useful to keep an
eye out to those needs as well.

Cheers!
Ben Root

On Fri, Jul 29, 2016 at 5:33 AM, Daniel Moisset <dmoisset at machinalis.com>
wrote:

> Hi Sebastian, thanks for your reply
>
> I'm glad to hear that you see value in having type annotations. Just to
> clarify, my question was aimed at surveying if there was interest in
> accepting the work we're already doing if we contribute it and if it has
> value for the numpy project. I'm aware there's effort involved; some
> colleagues and me are already involved doing that at
> https://github.com/machinalis/mypy-data because it's valuable for
> ourselves, so the volunteers are already here. You of course are invited to
> comment on the existing code and try it :) (or joining the effort, goes
> without saying)
>
> Running the checker on the test suite is probably the best way to validate
> the annotations (the normal way would be checking the annotations against
> the code, but that doesn't work with C extensions like numpy). That's
> something we haven't been doing yet but it's an obvious next step now that
> some simple examples are working.
> WRT "I wonder if all or most of numpy can be easily put into it.", we've
> covered ndarray (and matrix soon) which are the core types, things built
> upon that shouldn't be too hard. We found some snags along the way [1] [2],
> but no of it is a showstopper and I'm quite sure we'll fix those in time.
> But of course, if someone wants to try it out it will be a better
> validation than my optimism to see if this makes sense :)
>
> Thanks again and I'd be happy to hear more opinions from other numpy devs!
>
> Best,
>    D.
>
> [1] http://www.machinalis.com/blog/writing-type-stubs-for-numpy/
> [2] https://github.com/machinalis/mypy-data/issues
>
>
> On 29 Jul 2016 08:31, "Sebastian Berg" <sebastian at sipsolutions.net> wrote:
>
>> On Mi, 2016-07-27 at 20:07 +0100, Daniel Moisset wrote:
>> >
>> > Hi,
>> >
>> > I work at Machinalis were we use a lot of numpy (and the pydata stack
>> > in general). Recently we've also been getting involved with mypy,
>> > which is a tool to type check (not on runtime, think of it as a
>> > linter) annotated python code (the way of annotating python types has
>> > been recently standarized in PEP 484).
>> >
>> > As part of that involvement we've started creating type annotations
>> > for the Python libraries we use most, which include numpy. Mypy
>> > provides a way to specify types with annotations in separate files in
>> > case you don't have control over a library, so we have created an
>> > initial proof of concept at [1], and we are actively improving it.
>> > You can find some additional information about it and some problems
>> > we've found on the way at this blogpost [2].
>> >
>> > What I wanted to ask is if the people involved on the numpy project
>> > are aware of PEP484 and if you have some interest in starting using
>> > them. The main benefit is that annotations serve as clear (and
>> > automatically testable) documentation for users, and secondary
>> > benefits is that users discovers bugs more quickly and that some IDEs
>> > (like pycharm) are starting to use this information for smart editor
>> > features (autocompletion, online checking, refactoring tools);
>> > eventually tools like jupyter could take advantage of these
>> > annotations in the future. And the cost of writing and including
>> > these are relatively low.
>> >
>>
>> There is currently no plan to do it as far as I know, but with these
>> things it is often more of a problem that someone volunteers to
>> maintain it then to convince everyone that it is a good idea.
>> If there is enough interest we could talk about hosting it on the numpy
>> github group as a separate project to make it a bit more
>> visible/obvious that such a project exists.
>>
>> For inclusion in numpy, it seems to me that currently this would
>> probably be better of improved separately? In the long run, would it be
>> possible to do something like run all numpy tests and then check
>> whether the definitions cover (almost) everything, or test against the
>> documentation or so? Otherwise it might get tricky to keep things quite
>> up to date, at least until these type checks are very widely used. Also
>> I wonder if all or most of numpy can be easily put into it.
>>
>> Anyway, it seems like a great project to have as much support for type
>> annotations as possible. I have never used them, but with editors
>> picking up on these things it sounds like it could be very useful in
>> the future.
>>
>> - Sebastian
>>
>>
>> > We're doing the work anyway, but contributing our typespecs back
>> > could make it easier for users to benefit from this, and for us to
>> > maintain it and keep it in sync with future releases.
>> >
>> > If you've never heard about PEP484 or mypy (it happens a lot) I'll be
>> > happy to clarify anything about it that might helpunderstand this
>> > situation
>> >
>> > Thanks!
>> >
>> > D.
>> >
>> >
>> > [1] https://github.com/machinalis/mypy-data
>> > [2] http://www.machinalis.com/blog/writing-type-stubs-for-numpy/
>> >
>> > --
>> > Daniel F. Moisset - UK Country Manager
>> > www.machinalis.com
>> > Skype: @dmoisset
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160729/25709619/attachment.html>