Re: [code-quality] Python skeletons: type hinting for third-party libraries
Hello, This seems like a great project. I would *love* to see a more widespread type language for Python. I am commenting less on the skeleton collection because I think just mimicking DefinitelyTyped and Homebrew is such a good way to do it that I support it completely. I will definitely contribute skeletons for my dependencies in order to get better checking for my projects. For the type language... I am *less* concerned with: - *The exact concrete syntax for types.* I think Foo -> Bar and [Foo] and{Foo:Bar}are all nicer than Function[Foo,Bar] and List[Foo] and dict[Foo,Bar], but as long as the underlying type system has these concepts and there is *some* syntax for them (which there is not in typeannotations, mypy, or any contractl ibrary that I know of) then everything is fine. And we can always support shorthands in addition to the direct __repr__ style of writing these types. - *How the types get into Python code.* This is probably best done a few different way. The way that typeannotations and mypy do it with Python 3 annotations is lovely, but we cannot wait for Python 3 support everywhere. Docstrings in a various of ways are fine. This decision should be independent from the type language itself. - *How the types are checked*. Static checking is super, and dynamic checking is really helpful too, but as the docstring formats show it is really amazing to even just have the types for a human to read, and document your intent in a rigorous way. Wild monkey patching and reflection will probably always make it hard to be actually type safe and that would be unpythonic anyhow. I am *most* concerned with the *abstract* type language. Here are notes on the current proposal (and some context): - *The type language must have function types.* It is really disappointing when the only when I can describe a function's type is interleaved with its definition. It is very important to be able to say "method some_method has type (int, datetime, {int:str}, *str) -> str. Without this, I don't even have the words to describe the method. - *The type language should really have "static duck types"*. These were mentioned briefly by Vladimir. The technical term is row types, and they are only currently in python-rightarrow with the concrete syntax like object(self_type, field: type, field: type, ...). Without this, I don't have the words to describe an object. It is not really Pythonic to describe it in terms of its implementation when all that matters is that it has certain methods (of certain types). - *The type language should allow types for *args and **kwargs*. I do not see this in the current proposal. The current python-rightarrow contains one idea but I think it can be simplified to e.g. (Foo, *Bar, **Bizzle) -> Baz (since kwargs has to have str keys -- I think? -- you only give the type of its values.) - *Generics are just variables and should allow self-documenting names. *Just a little thing; Reserving T-Z for generics makes it so you cannot give them useful names. To be very clear: I believe these things about the whole project: - *I think the abstract type language can be describe without being too rigid about the concrete syntax.* The actual ways in which Python data structure are put together is not going to change any time soon, so if we get the abstract part right, there can be innovation in the concrete syntax, shorthands, etc. In fact, tuples, lists, dicts, and functions are all "real" things that exist pretty much independently of Python. I feel sort of the same way about row types, etc. So we may as well use the universal abstract syntax for them and give it a Pythonic concrete syntax. - *I think all of that can be defined without having a type checker (or even a type system) to handle all of them perfectly. *Any type that is too hard for a type checker can be treated as "unknown". Questions: - The type language proposal should go in its own repo perhaps? - How will license be handled for the major central repo of skeletons? I would prefer a global BSD/Apache/MIT/whatever style license and make that clear in CONTRIBUTING. Disclaimer: I am the author of python-rightarrow which is really an unimportant and barely-started side project to develop a simple type language for Python. I have a lot of experience with programming language and type checking / type inference / static-dynamic hybrid type implementation. I hope the result of this discussion makes my project obsolete. Cheers, - Kenn
Hello again,
Perhaps I missed if it was already mentioned, but I thought I would ensure
that PySonar was in the conversation as it was only recently released as
open source.
It seems to share goals with mypy and PyCharm, and has perhaps already been
used by SourceGraph to do a light & broad analysis of many project on
github.
- https://github.com/yinwang0/pysonar2/
- https://sourcegraph.com (unfortunately lacking the type information on
the site like Hoogle has, but it seems like the tool does types as well)
- (photo of slide from which I make quick guesses)
https://twitter.com/sqs/status/400834847855611904/photo/1
It would be nice to get all these project collaborating on the skeletons.
- Kenn
On Fri, Nov 8, 2013 at 1:50 PM, Kenn Knowles
Hello,
This seems like a great project. I would *love* to see a more widespread type language for Python. I am commenting less on the skeleton collection because I think just mimicking DefinitelyTyped and Homebrew is such a good way to do it that I support it completely. I will definitely contribute skeletons for my dependencies in order to get better checking for my projects.
For the type language...
I am *less* concerned with:
- *The exact concrete syntax for types.* I think Foo -> Bar and [Foo]and{Foo:Bar}are all nicer than Function[Foo,Bar] and List[Foo] and dict[Foo,Bar], but as long as the underlying type system has these concepts and there is *some* syntax for them (which there is not in typeannotations, mypy, or any contractl ibrary that I know of) then everything is fine. And we can always support shorthands in addition to the direct __repr__ style of writing these types.
- *How the types get into Python code.* This is probably best done a few different way. The way that typeannotations and mypy do it with Python 3 annotations is lovely, but we cannot wait for Python 3 support everywhere. Docstrings in a various of ways are fine. This decision should be independent from the type language itself.
- *How the types are checked*. Static checking is super, and dynamic checking is really helpful too, but as the docstring formats show it is really amazing to even just have the types for a human to read, and document your intent in a rigorous way. Wild monkey patching and reflection will probably always make it hard to be actually type safe and that would be unpythonic anyhow.
I am *most* concerned with the *abstract* type language. Here are notes on the current proposal (and some context):
- *The type language must have function types.* It is really disappointing when the only when I can describe a function's type is interleaved with its definition. It is very important to be able to say "method some_method has type (int, datetime, {int:str}, *str) -> str. Without this, I don't even have the words to describe the method.
- *The type language should really have "static duck types"*. These were mentioned briefly by Vladimir. The technical term is row types, and they are only currently in python-rightarrow with the concrete syntax like object(self_type, field: type, field: type, ...). Without this, I don't have the words to describe an object. It is not really Pythonic to describe it in terms of its implementation when all that matters is that it has certain methods (of certain types).
- *The type language should allow types for *args and **kwargs*. I do not see this in the current proposal. The current python-rightarrow contains one idea but I think it can be simplified to e.g. (Foo, *Bar, **Bizzle) -> Baz (since kwargs has to have str keys -- I think? -- you only give the type of its values.)
- *Generics are just variables and should allow self-documenting names. *Just a little thing; Reserving T-Z for generics makes it so you cannot give them useful names.
To be very clear: I believe these things about the whole project:
- *I think the abstract type language can be describe without being too rigid about the concrete syntax.* The actual ways in which Python data structure are put together is not going to change any time soon, so if we get the abstract part right, there can be innovation in the concrete syntax, shorthands, etc. In fact, tuples, lists, dicts, and functions are all "real" things that exist pretty much independently of Python. I feel sort of the same way about row types, etc. So we may as well use the universal abstract syntax for them and give it a Pythonic concrete syntax.
- *I think all of that can be defined without having a type checker (or even a type system) to handle all of them perfectly. *Any type that is too hard for a type checker can be treated as "unknown".
Questions:
- The type language proposal should go in its own repo perhaps?
- How will license be handled for the major central repo of skeletons? I would prefer a global BSD/Apache/MIT/whatever style license and make that clear in CONTRIBUTING.
Disclaimer: I am the author of python-rightarrow which is really an unimportant and barely-started side project to develop a simple type language for Python. I have a lot of experience with programming language and type checking / type inference / static-dynamic hybrid type implementation. I hope the result of this discussion makes my project obsolete.
Cheers,
- Kenn
participants (1)
-
Kenn Knowles