What's missing in PEP-484 (Type hints)
# Syntactic sugar "Beautiful is better than ugly, " thus nice syntax is needed. Current syntax is very mechanical. Syntactic sugar is needed on top of current PEP. # internal vs external @intify def foo() -> int: b = "42" return b # check 1 x = foo() // 2 # check 2 Does the return type apply to implementation (str) or decorated callable (int)? How can same annotation or a pair of annotations be used to: * validate return statement type * validate subsequent use * look reasonable in the source code # lambda Not mentioned in the PEP, omitted for convenience or is there a rationale? f = lambda x: None if x is None else str(x ** 2) Current syntax seems to preclude annotation of `x` due to colon. Current syntax sort of allows lamba return type annotation, but it's easy to confuse with `f`. # local variables Not mentioned in the PEP Non-trivial code could really use these. # global variables Not mentioned in the PEP Module-level globals are part of API, annotation is welcome. What is the syntax? # comprehensions [3 * x.data for x in foo if "bar" in x.type] Arguable, perhaps annotation is only needed on `foo` here, but then how complex comprehensions, e.g. below, the intermediate comprehension could use an annotation [xx for y in [...] if ...] # class attributes s = socket.socket(...) s.type, s.family, s.proto # int s.fileno # callable If annotations are only available for methods, it will lead to Java-style explicit getters and setters. Python language and data model prefers properties instead, thus annotations are needed on attributes. # plain data user1 = dict(id=123, # always int name="uuu", # always str ...) # other fields possible smth = [42, "xx", ...] (why not namedtuple? b/c extensible, mutable) At least one PHP IDE allows to annotate PDO. Perhaps it's just bad taste in Python? Or is there a valid use-case? # personal note I think it's amazing how much thought has already been put into this proposal. The foundation is pretty solid (per Guido talk). I am not at all opposed to software that infers types (like jedi), or reads user-specified types (like phpstorm and pep 484) and does something good with that. In fact I'm ambivalent to current proposal, standard and promise of better tools on one hand; narrow scope, debatable looks on the other. -- dima
* Dima Tisnek <dimaqq@gmail.com> [2015-04-30 13:41:53 +0200]:
# lambda Not mentioned in the PEP, omitted for convenience or is there a rationale? f = lambda x: None if x is None else str(x ** 2) Current syntax seems to preclude annotation of `x` due to colon. Current syntax sort of allows lamba return type annotation, but it's easy to confuse with `f`.
Not sure if you'd really want to stuff type annotations into a lambda... at that point you'd IMHO be better off by using a real function.
# local variables Not mentioned in the PEP Non-trivial code could really use these.
# global variables Not mentioned in the PEP Module-level globals are part of API, annotation is welcome. What is the syntax?
# comprehensions [3 * x.data for x in foo if "bar" in x.type] Arguable, perhaps annotation is only needed on `foo` here, but then how complex comprehensions, e.g. below, the intermediate comprehension could use an annotation [xx foj y in [...] if ...]
# class attributes s = socket.socket(...) s.type, s.family, s.proto # int s.fileno # callable If annotations are only available for methods, it will lead to Java-style explicit getters and setters. Python language and data model prefers properties instead, thus annotations are needed on attributes.
# plain data user1 = dict(id=123, # always int name="uuu", # always str ...) # other fields possible smth = [42, "xx", ...] (why not namedtuple? b/c extensible, mutable) At least one PHP IDE allows to annotate PDO. Perhaps it's just bad taste in Python? Or is there a valid use-case?
Most (all?) of this is actually mentioned in the PEP: https://www.python.org/dev/peps/pep-0484/#type-comments Florian -- http://www.the-compiler.org | me@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc I love long mails! | http://email.is-not-s.ms/
# plain data user1 = dict(id=123, # always int name="uuu", # always str ...) # other fields possible smth = [42, "xx", ...] (why not namedtuple? b/c extensible, mutable) At least one PHP IDE allows to annotate PDO. Perhaps it's just bad taste in Python? Or is there a valid use-case?
Most (all?) of this is actually mentioned in the PEP: https://www.python.org/dev/peps/pep-0484/#type-comments
The question was about mixed containers, e.g.: x = [12, "Jane"] y = [13, "John", 99.9] There first element is always int, second always str, and rest is variable.
On Thu, Apr 30, 2015 at 01:41:53PM +0200, Dima Tisnek wrote:
# Syntactic sugar "Beautiful is better than ugly, " thus nice syntax is needed. Current syntax is very mechanical. Syntactic sugar is needed on top of current PEP.
I think the annotation syntax is beautiful. It reminds me of Pascal.
# internal vs external @intify def foo() -> int: b = "42" return b # check 1 x = foo() // 2 # check 2
Does the return type apply to implementation (str) or decorated callable (int)?
I would expect that a static type checker would look at foo, and flag this as an error. The annotation says that foo returns an int, but it clearly returns a string. That's an obvious error. Here is how I would write that: # Perhaps typing should have a Function type? def intify(func: Callable[[], str]) -> Callable[[], int]: @functools.wraps(func) def inner() -> int: return int(func()) return inner @intify def foo() -> str: b = "42" return b That should, I hope, pass the type check, and without lying about the signature of *undecorated* foo. The one problem with this is that naive readers will assume that *decorated* foo also has a return type of str, and be confused. That's a problem. One solution might be, "don't write decorators that change the return type", but that seems horribly restrictive. Another solution might be to write a comment: @intify # changes return type to int def foo() -> str: ... but that's duplicating information already in the intify decorator, and it relies on the programmer writing a comment, which people don't do unless they really need to. I think that the only solution is education: given a decorator, you cannot assume that the annotations still apply unless you know what the decorator does.
How can same annotation or a pair of annotations be used to: * validate return statement type * validate subsequent use * look reasonable in the source code
# lambda Not mentioned in the PEP, omitted for convenience or is there a rationale? f = lambda x: None if x is None else str(x ** 2) Current syntax seems to preclude annotation of `x` due to colon. Current syntax sort of allows lamba return type annotation, but it's easy to confuse with `f`.
I don't believe that you can annotate lambda functions with current syntax. For many purposes, I do not think that is important: a good type checker will often be able to infer the return type of the lambda, and from that infer what argument types are permitted: lambda arg: arg + 1 Obviously arg must be a Number, since it has to support addition with ints.
# local variables Not mentioned in the PEP Non-trivial code could really use these.
Normally local variables will have their type inferred from the operations done to them: s = arg[1:] # s has the same type as arg When that is not satisfactory, you can annotate variables with a comment: s = arg[1:] #type: List[int] https://www.python.org/dev/peps/pep-0484/#id24
# global variables Not mentioned in the PEP Module-level globals are part of API, annotation is welcome. What is the syntax?
As above.
# comprehensions [3 * x.data for x in foo if "bar" in x.type] Arguable, perhaps annotation is only needed on `foo` here, but then how complex comprehensions, e.g. below, the intermediate comprehension could use an annotation [xx for y in [...] if ...]
A list comprehension is obviously of type List. If you need to give a more specific hint: result = [expr for x in things if cond(x)] #type: List[Whatever] See also the discussion of "cast" in the PEP. https://www.python.org/dev/peps/pep-0484/#id25
# class attributes s = socket.socket(...) s.type, s.family, s.proto # int s.fileno # callable If annotations are only available for methods, it will lead to Java-style explicit getters and setters. Python language and data model prefers properties instead, thus annotations are needed on attributes.
class Thing: a = 42 # can be inferred b = [] # inferred as List[Any] c = [] #type: List[float] -- Steve
On 30 April 2015 at 14:33, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Apr 30, 2015 at 01:41:53PM +0200, Dima Tisnek wrote:
# Syntactic sugar "Beautiful is better than ugly, " thus nice syntax is needed. Current syntax is very mechanical. Syntactic sugar is needed on top of current PEP.
I think the annotation syntax is beautiful. It reminds me of Pascal.
Haha, good one!
# internal vs external @intify def foo() -> int: b = "42" return b # check 1 x = foo() // 2 # check 2
Does the return type apply to implementation (str) or decorated callable (int)?
I would expect that a static type checker would look at foo, and flag this as an error. The annotation says that foo returns an int, but it clearly returns a string. That's an obvious error.
Is this per PEP, or just a guess? I think PEP needs to be explicit about this. [snipped]
lambda arg: arg + 1
Obviously arg must be a Number, since it has to support addition with ints.
Well, no, it can be any custom type that implements __add__ Anyhow, the question was about non-trivial lambdas.
# local variables Not mentioned in the PEP Non-trivial code could really use these.
Normally local variables will have their type inferred from the operations done to them:
s = arg[1:] # s has the same type as arg
Good point, should be mentioned in PEP. Technically, type can be empty list, mixed list or custom return type for overloaded __getitem__ that accepts slices.
# comprehensions [3 * x.data for x in foo if "bar" in x.type] Arguable, perhaps annotation is only needed on `foo` here, but then how complex comprehensions, e.g. below, the intermediate comprehension could use an annotation [xx for y in [...] if ...]
A list comprehension is obviously of type List. If you need to give a more specific hint:
result = [expr for x in things if cond(x)] #type: List[Whatever]
See also the discussion of "cast" in the PEP.
Good point for overall comprehension type. re: cast, personally I have some reservations against placing `cast()` into runtime path. I'm sorry if I was not clear. My question was how should type of ephemeral `x` be specified. In other words, can types be specified on anything inside a comprehension?
# class attributes s = socket.socket(...) s.type, s.family, s.proto # int s.fileno # callable If annotations are only available for methods, it will lead to Java-style explicit getters and setters. Python language and data model prefers properties instead, thus annotations are needed on attributes.
class Thing: a = 42 # can be inferred b = [] # inferred as List[Any] c = [] #type: List[float]
Good point, I suppose comments + stub file may be sufficient. Stub is better (required?) because it allows to specify types of attributes that are not assigned in class scope, but that are expected to be there as result of __init__ or because it's a C extension. An example in PEP would be good.
On Thu, May 7, 2015 at 7:25 AM, Dima Tisnek <dimaqq@gmail.com> wrote:
On 30 April 2015 at 14:33, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Apr 30, 2015 at 01:41:53PM +0200, Dima Tisnek wrote:
# internal vs external @intify def foo() -> int: b = "42" return b # check 1 x = foo() // 2 # check 2
Does the return type apply to implementation (str) or decorated callable (int)?
I would expect that a static type checker would look at foo, and flag this as an error. The annotation says that foo returns an int, but it clearly returns a string. That's an obvious error.
Is this per PEP, or just a guess?
I think PEP needs to be explicit about this.
The PEP shouldn't have to explain all the rules for type inferencing. There's a section "What is checked?" that says (amongst other things): The body of a checked function is checked for consistency with the given annotations. The annotations are also used to check correctness of calls appearing in other checked functions.
Normally local variables will have their type inferred from the
operations done to them:
s = arg[1:] # s has the same type as arg
Good point, should be mentioned in PEP.
Again, what do you want the PEP to say? I am trying to keep the PEP shorter than the actual code that implements the type checker. :-)
Technically, type can be empty list, mixed list or custom return type for overloaded __getitem__ that accepts slices.
I'm sorry if I was not clear. My question was how should type of ephemeral `x` be specified. In other words, can types be specified on anything inside a comprehension?
That's actually a good question; the PEP shows some examples of #type: comments in peculiar places, but there's no example using list comprehensions. Your best bet is to leave it to the type inferencer; if your comprehension is so complex that need to put type annotations on parts of it, you may be better off rewriting it as a regular for-loop, which offers more options for annotations.
Stub is better (required?) because it allows to specify types of attributes that are not assigned in class scope, but that are expected to be there as result of __init__ or because it's a C extension.
Note that the PEP does not explicitly say whether the information of a stub might be *merged* with the information gleaned from the source code. The basic model is that if a stub is present the implementation source code is not read at all by the type checker (and, conversely, information from stubs is not available at all at runtime). But it is possible for some type checker to improve upon this model.
An example in PEP would be good.
Can you give an example that I can edit and put in the PEP? -- --Guido van Rossum (python.org/~guido)
re: comprehension Perhaps PEP can, at least, have a short list/summary of limitations? I recall something was mentioned, but I can't find a section like that in PEP. re: example following https://github.com/JukkaL/mypy/blob/master/stubs/3.2/socket.py # socket.pyi python2 class _socketobject: family = 0 # inferred from initializer (?) type = 0 # type: int # explicit # socket.pyi python3 class socket: family = AddressFamily.AF_UNSPEC # inferred I presume? def settimeout(timeout: Union[int, float, None]) -> None: pass # negative arguments illegal timeout = -1.0 # yet, that's what you get by default (set None) Perhaps, after all, socket module is a bad example. I suppose you have a point that well-written modules are self-documenting anyway... Here's another try: # _sqlite3.pyi python2 version # warning, readonly: module allows reassignment, but you really shouldn't! # instead use sqlite3.register_xxx functions converters = {} # type: Dict[str, Callable[[str], Any]] adapters = {} # type: Dict[Tuple[Type, SomethingInternal], Callable[[Any], str]] On 7 May 2015 at 17:39, Guido van Rossum <guido@python.org> wrote:
On Thu, May 7, 2015 at 7:25 AM, Dima Tisnek <dimaqq@gmail.com> wrote:
On 30 April 2015 at 14:33, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Apr 30, 2015 at 01:41:53PM +0200, Dima Tisnek wrote:
# internal vs external @intify def foo() -> int: b = "42" return b # check 1 x = foo() // 2 # check 2
Does the return type apply to implementation (str) or decorated callable (int)?
I would expect that a static type checker would look at foo, and flag this as an error. The annotation says that foo returns an int, but it clearly returns a string. That's an obvious error.
Is this per PEP, or just a guess?
I think PEP needs to be explicit about this.
The PEP shouldn't have to explain all the rules for type inferencing. There's a section "What is checked?" that says (amongst other things):
The body of a checked function is checked for consistency with the given annotations. The annotations are also used to check correctness of calls appearing in other checked functions.
Normally local variables will have their type inferred from the operations done to them:
s = arg[1:] # s has the same type as arg
Good point, should be mentioned in PEP.
Again, what do you want the PEP to say? I am trying to keep the PEP shorter than the actual code that implements the type checker. :-)
Technically, type can be empty list, mixed list or custom return type for overloaded __getitem__ that accepts slices.
I'm sorry if I was not clear. My question was how should type of ephemeral `x` be specified. In other words, can types be specified on anything inside a comprehension?
That's actually a good question; the PEP shows some examples of #type: comments in peculiar places, but there's no example using list comprehensions. Your best bet is to leave it to the type inferencer; if your comprehension is so complex that need to put type annotations on parts of it, you may be better off rewriting it as a regular for-loop, which offers more options for annotations.
Stub is better (required?) because it allows to specify types of attributes that are not assigned in class scope, but that are expected to be there as result of __init__ or because it's a C extension.
Note that the PEP does not explicitly say whether the information of a stub might be *merged* with the information gleaned from the source code. The basic model is that if a stub is present the implementation source code is not read at all by the type checker (and, conversely, information from stubs is not available at all at runtime). But it is possible for some type checker to improve upon this model.
An example in PEP would be good.
Can you give an example that I can edit and put in the PEP?
-- --Guido van Rossum (python.org/~guido)
participants (4)
-
Dima Tisnek -
Florian Bruhin -
Guido van Rossum -
Steven D'Aprano