Trial balloon: adding variable type declarations in support of PEP 484
PEP 484 doesn't change Python's syntax. Therefore it has no good syntax to offer for declaring the type of variables, and instead you have to write e.g. a = 0 # type: float b = [] # type: List[int] c = None # type: Optional[str] I'd like to address this in the future, and I think the most elegant syntax would be to let you write these as follows: a: float = 0 b: List[int] = [] c: Optional[str] = None (I've considered a 'var' keyword in the past, but there just are too many variables named 'var' in my code. :-) There are some corner cases to consider. First, to declare a variable's type without giving it an initial value, we can write this: a: float Second, when these occur in a class body, they can define either class variables or instance variables. Do we need to be able to specify which? Third, there's an annoying thing with tuples/commas here. On the one hand, in a function declaration, we may see (a: int = 0, b: str = ''). On the other hand, in an assignment, we may see a, b = 0, '' Suppose we wanted to add types to the latter. Would we write this as a, b: int, str = 0, '' or as a: int, b: str = 0, '' ??? Personally I think neither is acceptable, and we should just write it as a: int = 0 b: str = '' but this is a slight step back from a, b = 0, '' # type: (int, str) -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 2, 2016 at 7:31 AM, Guido van Rossum <guido@python.org> wrote:
Additional case, unless it's patently obvious to someone with more 484 experience than I: what happens with chained assignment? a = b = c = 0 Does each variable get separately tagged, or does one tag apply to all? ChrisA
On 08/01/2016 02:40 PM, Chris Angelico wrote:
On Tue, Aug 2, 2016 at 7:31 AM, Guido van Rossum wrote:
I would think each name would need to be typed: a: int = b: int = c: int = 0 However, if somebody was doing typing from the get-go I would imagine they would do: a, b, c: float -- ~Ethan~
On Mon, Aug 1, 2016 at 2:46 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
The problem with this is that the relative priorities of '=' and ',' are inverted between argument lists and assignments. And the expression on the right might be a single variable whose type is a tuple. So we'd get a: int, b: str = x But the same thing in a function definition already has a different meaning: def foo(a: int, b: str = x): ... -- --Guido van Rossum (python.org/~guido)
On 2016-08-01 15:58, Guido van Rossum wrote:
Is that really a big deal? Insofar as it's a problem, it already exists for function arguments vs. assignments without any type annotation. That is: a, b = x already means something different from def foo(a, b = x): ... So in one sense, keeping type annotations with the variables would actually maintain the current convention (namely, that assignment and argument defaults have different conventions). -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Mon, Aug 1, 2016 at 2:46 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
But what would this do? a: int, b: str = x Does the x get distributed over (a, b) or does a remain unset? The analogy with assignment suggest that x gets distributed, but the analogy with function definitions says x only goes to b. -- --Guido van Rossum (python.org/~guido)
On 08/01/2016 04:31 PM, Guido van Rossum wrote:
When speaking of the function header I meant that we already have one way to say which name has which type, and it would be simpler (at least to understand) if we stick with one way to specify type. As far as questions such as "what would this do" I would say it should do the exact same thing as if the type info wasn't there: a: int, b: str = x simplifies to a, b = x and so x had better be a two-item iterable; on the other hand: def func(a: int, b: str = x): simplifies to def func(a, b=x): and so parameter a is mandatory while parameter b has a default and so is optional. -- ~Ethan~
On 2 August 2016 at 00:41, Ethan Furman <ethan@stoneleaf.us> wrote:
From what Guido says, the main use case is class variables, where complicated initialisations are extremely rare, so this shouldn't be a
For me, a: int, b: str = x immediately says "leave a unassigned, and assign x to b". Maybe that's my C experience at work. But a, b = x immediately says tuple unpacking. So in my view, allowing annotations on unpacking syntax is going to cause a *lot* of confusion, and I'd strongly argue for only allowing type annotations on single variables: VAR [: TYPE] [= INITIAL_VALUE] For multiple variables, just use multiple lines: a: int b: str = x or a: int b: str a, b = x depending on your intention. problem in practice. Paul
Just to add some data points, here is how it is done in some other languages which have both types and tuple unpacking. In all cases I added the minimum of parentheses required. Scala: val (x : Int, y : String) = (42, "Hello") Ocaml: let (x: int), (y : string) = 42, "Hello" alternatively: let (x, y) : int * string = 42, "Hello" Note that if we omit the types we can do the Python-esque: let x, y = 42, "Hello" Stephan 2016-08-02 9:54 GMT+02:00 Paul Moore <p.f.moore@gmail.com>:
On 2 August 2016 at 17:54, Paul Moore <p.f.moore@gmail.com> wrote:
+1 from me for this view: only allow syntactic annotation of single names without parentheses. That one rule eliminates all the cases where unpacking assignment and parameter assignment do starkly different things. *However*, even with that restriction, unambiguously annotated unpacking could still be supported via: (a, b) : Tuple[int, str] = x (a, b, *c) : Tuple[int, int, Tuple[int]] = y Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Aug 1, 2016 at 3:31 PM, Guido van Rossum <guido@python.org> wrote:
As noted by someone else, what about "local": local a: float It seems like nonlocal and global would need the same treatment: nonlocal b: List[int] global c: Optional[str]
Isn't that currently a NameError? Is it worth making this work while preserving the error case? A "local" keyword would solve the problem, no?
In the immediate case I'd expect the former. We don't currently have a canonical way to "declare" instance attributes in the class definition. It may be worth sorting that out separately and addressing the PEP 484 aspect at that point.
Third, there's an annoying thing with tuples/commas here.
What about unpacking into explicit displays? For example: (a, b) = 0, '' [a, b] = 0, '' -eric
On Mon, Aug 1, 2016 at 11:40 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
The last 2 examples are actually wrong. The use of "nonlocal" and "global" is inside functions, to refer to variables that were originally defined *outside* the function. The type declaration should be near the definition (in the global/outer scope), not on its use. If we're looking for a keyword-based syntax for empty definitions, one option would be: def a: float It uses an existing keyword so no risk of name collissions, the meaning is quite accurate (you're defining a new variable), and my guess is that the syntax is distinct enough (no parenthesis or final colons which would indicate a function definition). OTOH it might break some usages of grep to look for functions, and autoindenters on code editors. -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Mon, Aug 1, 2016 at 3:40 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
A common use of `# type:` is in class bodies. But `local` is a bad idea there.
Well, they don't support assignment either, so I'd like to keep them out of this if possible.
Currently looks like a syntax error to me.
My observation is that most uses of `# type: ...` in class bodies is used to declare an instance variable.
Yeah. -- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016 at 10:31 PM, Guido van Rossum <guido@python.org> wrote:
I'd say that if I have a class C with a class variable cv, instance variable iv, a good type checking system should detect: C.cv # ok C.iv # error! C().iv # ok which is something that PEP484 doesn't clarify much (and mypy flags all 3 as valid) So in short, I think it is relevant to specify differently class vs instance vars. Suppose we wanted to add types to the latter. Would we write this as
I'm not specially fond of the «# type: (int, str)». It works ok for 2 variables, but for more it is hard to mentally "align" the variable names to the type names, for example in: kind, text, start, end, line = token # type: (int, str, Tuple[int, int], Tuple[int, int], str) it's not easy to quickly answer "what is the type of 'end' here?". So I wouldn't miss that notation a lot if it went away Given that, I'm happier with both the 2-line solution and the second one-liner, which both keep the types closer to the names. But given that as you mentioned: kind: int, text:str, start: Tuple[int, int], end: Tuple[int, int], line: str = token looks a bit misleading (looks more like an assignment to token), perhaps it would avoid errors to accpt as valid only: (kind: int, text:str, start: Tuple[int, int], end: Tuple[int, int], line: str) = token other possibility if you really love the current mypy syntax (perhaps both could be valid): (kind, text, start, end, line):(int, str, Tuple[int, int], Tuple[int, int], str) = token I don't like that one very much, but perhaps it inspires ideas on someone here. Other places to think about are: * Multiple assignment (Chris mentioned these) * loop variables (in a for statement, comprehensions, generator expressions) * lambda arguments -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Mon, Aug 1, 2016 at 4:35 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
Yeah, this is all because you can't express that in Python either. When you see an assignment in a class body you can't tell if it's meant as an instance variable default or a class variable (except for some specific cases -- e.g. nested class definitions are pretty obvious :-).
So in short, I think it is relevant to specify differently class vs instance vars.
Agreed, we need to invent a workable proposal for this. Here's a strawman: - The default is an instance variable (backed by a class variable as default if there's an initial value) - To define a class variable, prefix the type with 'class` Example: class C: a: int # instance var b: List[int] = None # instance var c: class List[int] = [] # class var Class variables must come with an initializer; instance variables may or may not have an initializer. (Bonus: instance variable initializers must be immutable values.) Regarding everything involving multiple variables (either `a = b = 0` or `a, b = 0, 0`) I propose that those cannot be combined with types. Period. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 2, 2016 at 11:09 PM, Guido van Rossum <guido@python.org> wrote:
May be I've gotten wrong my python style for many years, but I always considered that the "proper" way to create instance variables was inside the initializer (or in rare occasions, some other method/classmethod). For me, an assignment at a class body is a class variable/constant. So I was going to propose "type declarations at class level are always for class variables, and inside methods (preceded by "self.") are for instance variables". Using class level variables for defaults always seemed unpythonic and error prone (when mutable state is involved) to me. I felt that was common practice but can't find documentation to support it, so I'd like to hear if I'm wrong there :)
--
Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On 3 August 2016 at 11:51, Daniel Moisset <dmoisset@machinalis.com> wrote:
You've just missed one of the most powerful side-effects of how Python's class and instance variables interact: class Spaceship: color = RED hitpoints = 50 def powerup(self): self.color = BLUE self.hitpoints += 100 Above: the defaults are good for almost all spaceship instaces - but when one of them is "powered up", and only them, that instance values are defined to be different than the defaults specified in the class. At that point proper instance variables are created in the instanceś __dict__, but for all other instances, the defaults, living in the instance's".__class__.__dict__" are just good enough.
I may have missed that on the message deluge so far - but would the type annotations be available at runtime, like parameter annotations live in "__annotations__"? An __annotations__ dict straight in the class itself? I thinkt hat is the obvious thing - although I did not see it in the above messages. (A s I said, I may have missed its mention) That is ratehr important because then, beyond enabling static code analysis it is easy to have third party frameworks that enforce typing in runtime. On 3 August 2016 at 12:13, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On Wed, Aug 3, 2016 at 11:31 AM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
It was mentioned that var annotations would be evaluated and discarded, but I agree that this is less than ideal. I agree that it is better to store var annotations in the namespace that they appear in. Maybe something like this: a: int = 0 b: List[float] = [] would result in a namespace { 'a': 0, 'b': [], '__var_annotations__': { 'a': int, 'b': List[float], }, }
On 3 August 2016 at 16:40, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Agreed, it seems a shame that this would be an area where it's not possible to introspect data that was available at compile time. (I'd say it doesn't feel Pythonic, except that claiming that Guido's proposing something non-Pythonic is self-contradictory :-)) However, I have no expectation of ever needing this data for anything I'd write, so it's not actually something that would matter to me one way or the other. Paul
Guido is the BDFL of CPython, not the arbiter of what the community thinks is 'pythonic'. It is yet to be seen if type annotations are considered pythonic at all. Under the assumption that they are, yes, annotations that also come with initialization would be the more pythonic way of doing things.
On Thu, Aug 4, 2016 at 1:40 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
So... the maligned statement "a: int" would actually mean "__var_annotations__['a'] = int", which is a perfectly valid executable statement. This might solve the problem? Question, though: Does __var_annotations__ always exist (as a potentially empty dict), or is it created on first use? A function's __annotations__ is always present, but functions have lots of attributes that we normally don't care about. Polluting every single namespace seems wasteful; magically creating a new local variable when you first hit an annotated 'declaration' seems odd. ChrisA
On Wed, Aug 3, 2016 at 8:31 AM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Actually we've hardly touched on that yet. For classes I think it would be nice to make these available. For locals I think we should not evaluate the type at all, otherwise the cost of using a variable declaration would be too high.
Although, frankly, that's not something that PEP 484 particularly cares about. Runtime type checking has very different requirements -- would you really want to check that a List[int] contains only ints if the list has a million items? -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 16:13, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Yes, but I view that as "when you ask for an instance variable, if there isn't one you get the class variable as the default - nested namespaces basically, just like local variables of a function and global variables. So to me class Spaceship: hitpoints: int = 50 declares a class Spaceship, with a *class* variable hitpoints. Instances will get that value by default, but can assign something different.
class Whatever: myval : int = 20 def __init__(self): self.myval = "Fooled you!" The problem is of course that there's no way to attach a type declaration to an instance variable, unless you allow self.myval : str = "Fooled you" which opens up the possibility of declaring types for (in theory) arbitrary expressions. In summary, I think type annotations on variables declared at class scope should describe the type of the class variable - because that's what the assignment is creating. That leaves no obvious means of declaring the type of an instance variable (can you put a comment-style type annotation on the self.x = whatever line at the moment?) Which is a problem, but not one (IMO) that should be solved by somehow pretending that when you declare a class variable you're actually declaring an instance variable. Paul
On Wed, Aug 3, 2016 at 8:35 AM, Paul Moore <p.f.moore@gmail.com> wrote: [...]
But here you're thinking with your runtime hat on. The type checker would actually like to understand what you mean here -- a class variable (maybe it can be updated by saying `Spaceship.hitpoints += 10`) or an instance variable. Which is why I proposed that you can make it a class variable by saying `hitpoints: class int = 50`, and the default would be for it to be an instance variable. I propose that the difference is that class variables cannot be updated through the instance (as that would make it an instance variable, which you've explicitly promised not to do by using the `class int` type).
Again you're thinking with your runtime hat on. If you were to run this through a type checker you'd *want* to get an error here (since realistically there is probably a lot more going on in that class and the type clash is a symptom of a poor choice for a variable name or a bad initializer).
We should probably allow that too, but only for assignment to instance variables using `self` (it's easy for the type checker to only allow certain forms -- the runtime should be more lenient). A type checker that warns about instance variables used or set without a declaration could be very useful indeed (catching many typos).
I look at it from a more pragmatic side. What do I want my type checker to check? The annotations need to be useful to help me catch bugs but not so annoying that I have to constantly override the type checker. From this perspective I definitely want a way to declare the types of instance variables at the class level. Packages like SQLAlchemy and Django and traits, that have each developed their own, quite sophisticated machinery for declaring instance variables, seem to support this perspective. -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 17:26, Guido van Rossum <guido@python.org> wrote:
But here you're thinking with your runtime hat on
Yep, that's the mistake I was making. Thanks for clarifying. I've yet to actually use a type checker[1], so I'm still not used to thinking in terms of "typecheck-time" behaviour. Paul [1] Off-topic, but I'm not sure if type annotations are powerful enough (yet) to track bytes vs str sufficiently to help with resolving Python 2 -> Python 3 string handling errors. When I get a chance to confirm that's possible, I definitely have some candidate codebases I'd like to try my hand with :-)
On Wed, Aug 3, 2016 at 10:29 AM, Paul Moore <p.f.moore@gmail.com> wrote:
That's unfortunate -- the feature I'm developing here is only of interest to people using a type checker, and the many details of what it should look like and how it should work will benefit most from feedback from people who have actually dealt with the existing way of declaring variables.
Not yet, but we're working on it. See https://github.com/python/typing/issues/208 -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 19:01, Guido van Rossum <guido@python.org> wrote:
Understood. Mostly I'm keeping out of it - I chimed in because I (mistakenly) thought the way of declaring instance variables clashed with a non-(typing)-expert's understanding of what was going on (defining a class variable which is used as a fallback value if there's no instance variable of the same name). I still think the notation could be confusing, but the confusion can be fixed with documentation, and I'm happy to propose fixes for documentation when I finally get round to using the feature. Sorry for the noise.
Great :-) Paul
There's something I found a bit confusing in the runtime semantic of defaults in your strawman proposal: On Tue, Aug 2, 2016 at 11:09 PM, Guido van Rossum <guido@python.org> wrote:
Or you are proposing that in runtime these are equivalent to "b, c = None, []" ? If that's the case, I find it misleading to call them "instance variables" vs "class variables", given that those concepts are supposed to have different runtime semantics, not just for the typechecker. -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 7:12 AM, Daniel Moisset <dmoisset@machinalis.com> wrote:
The latter (except possibly for also storing the types in __annotations__). I'm a little worried by your claim that it's misleading to distinguish between instance and class variables. There really are three categories: - pure class variable -- exists in class __dict__ only - pure instance variable -- exists in instance __dict__ only - instance variable with class default -- in class __dict__ and maybe in instance __dict__ (Note that even a pure class variable can be referenced as an instance attribute -- this is the mechanism that enables the third category.) Pragmatically, the two categories of instance variables together are wildly more common than pure class variables. I also believe that the two categories of instance variables are both pretty common. So I want the notation to support all three: class Starship: stats: class Dict[str, int] = {} # Pure class variable damage: class int = 0 # Hybrid class/instance variable captain: str # Pure instance variable The intention is that the captain's name must always be set when a Starship is initialized, but all Starships start their life with zero damage, and stats is merely a place where various Starship methods can leave behind counters logging how often various events occur. (In real code it would probably be a defaultdict.) -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 4, 2016 at 5:40 PM, Guido van Rossum <guido@python.org> wrote:
For me what's misleading is the runtime behaviour for b, which has na initializer but is not flagged as class variable... see below
I follow the example perfectly. Now suppose a reader finds the following piece of code: class Starship: stats: class Dict[str, int] = {} # Pure class variable damage: class int = 0 # Hybrid class/instance variable captain: str # Pure instance variable speed: float = 0 I added a new attribute (similar to b in your original example). Given that the type declaration doesn't say "class",the reader might be inclined to think it's an instance variable. But in runtime (if I got you right), that variable will be stored in "Starship.__dict__" and writing "Starship.speed = 3" will change the speed of those starship instances that still haven't set the attribute. So in the end both "damage" and "speed" have "class variable" runtime semantics, even when one is flagged as "class" and the other isn't. The other combination that feels a bit confusing when adding "class" tags is saying "attr: class T", without an initializer.... in what case would someone do that? what does it mean if I find some code saying that about the class, that it might get that attribute set somewhere else? -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 12:11 PM, Daniel Moisset <dmoisset@machinalis.com> wrote: [...]
We may have to debate more what the checker should allow here. (Since it's okay for a type checker to disallow things that might actually work at runtime.) I'm inclined to allow it, as long as the value assigned to Starship.speed is compatible with float.
Agreed that looks silly. We probably should make that a syntax error. -- --Guido van Rossum (python.org/~guido)
On 5 August 2016 at 02:40, Guido van Rossum <guido@python.org> wrote:
I'm mostly on board with the proposal now, but have one last niggle: What do you think of associating the "class" more with the variable name than the type definition for class declarations by putting to the left of the colon? That is: class Starship: stats class: Dict[str, int] = {} # Pure class variable damage class: int = 0 # Hybrid class/instance variable captain: str # Pure instance variable Pronounced as: "stats is declared on the class as a dict mapping from strings to integers and is initialised as an empty dict" "damage is declared on the class as an integer and is initialised as zero" "captain is declared on instances as an integer" Just a minor thing, but the closer association with the name reads better to me since "Class attribute or instance attribute?" is really a property of the name binding, rather than of the permitted types that can be bound to that name Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 5, 2016 at 12:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Hmm... But the type is *also* a property of the name binding. And I think the "class-var-ness" needs to be preserved in the __annotations__ dict somehow, so that's another reason why it "belongs" to the type rather than to the name binding (a nebulous concept to begin with). Also, I like the idea that everything between the ':' and the '=' (or the end of the line) belongs to the type checker. I expect that'll be easier for people who aren't interested in the type checker. -- --Guido van Rossum (python.org/~guido)
On 6 August 2016 at 02:12, Guido van Rossum <guido@python.org> wrote:
Fair point, although this and the __var_annotations__ discussion also raises the point that annotations have to date always been valid Python expressions. So perhaps rather than using the class keyword, it would make sense to riff off classmethod and generic types to propose: class Starship: stats: ClassAttr[Dict[str, int]] = {} # Pure class variable damage: DefaultAttr[int] = 0 # Hybrid class/instance variable captain: str # Pure instance variable Pronounced as: "stats is a class attribute mapping from strings to integers and is initialised as an empty dict" "damage is an instance attribute declared as an integer with a default value of zero defined on the class" "captain is an instance attribute declared as a string" In addition to clearly distinguishing class-only attributes from class-attribute-as-default-value, this would also mean that the specific meaning of those annotations could be documented in typing.ClassAttr and typing.DefaultAttr, rather than needing to be covered in the documentation of the type annotation syntax itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 5, 2016 at 1:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Same here. The only thing I'd like considered further is exposing the annotations at runtime. Others have already suggested this and Guido has indicated that they will not be evaluated at runtime. I think that's fine in the short term, but would still like future support considered (or at least not ruled out) for runtime evaluation (and availability on e.g. func/module/cls.__var_annotations__). If performance is the main concern, we should be able to add compiler support to evaluate/store the annotations selectively on a per-file basis, like we do for __future__ imports. Alternately, for functions we could evaluate the annotations in global/non-local scope at the time the code object is created (i.e. when handling the MAKE_FUNCTION op code), rather than as part of the code object's bytecode. Again, not adding the support right now is fine but we should be careful to not preclude later support. Of course, this assumes sufficient benefit from run-time access variable type annotations, which I think is weaker than the argument for function annotations. We should still keep this possible future aspect in mind though. Relatedly, it would be nice to address the future use of this syntax for more generic variable annotations (a la function annotations), but that's less of a concern for me. The only catch is that making "class" an optional part of the syntax impacts the semantics of the more generic "variable annotations". However, I don't see "class" as a problem, particularly if it is more strongly associated with the name rather than the annotation, as you've suggested below. If anything it's an argument *for* your recommendation. :)
+1 One question: Will the use of the "class" syntax only be allowed at the top level of class definition blocks? -eric
On Fri, Aug 5, 2016 at 12:23 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
I had exactly the same thoughts, but I wonder how likely the type annotations would need local scope. For example, is something like this def f(x): v: type(x) ... going to be supported by type checkers?
On Fri, Aug 5, 2016 at 9:23 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Actually my current leaning is as follows: - For annotated module globals, the type annotations will be evaluated and collected in a dict named __annotations__ (itself a global variable in the module). - Annotations in class scope are evaluated and collected in a dict named __annotations__ on the class (itself a class attribute). This will contain info about annotated class variables as well as instance variables. I'm thinking that the class variable annotations will be wrapped in a `ClassVar[...]` object. - Annotations on local variables are *not* evaluated (because it would be too much of a speed concern) but the presence of an annotation still tells Python that it's a local variable, so a "local variable slot" is reserved for it (even if no initial value is present), and if you annotate it without initialization you may get an UnboundLocalError, e.g. here: def foo(): x: int print(x) # UnboundLocalError This is the same as e.g. def bar(): if False: x = 0 print(x) # UnboundLocalError
I considered that, but before allowing that complexity, I think we should come up with a compelling use case (not a toy example). This also produces some surprising behavior, e.g. what would the following do: def foo(): T = List[int] a: T = [] # etc. If we evaluate the annotation `T` in the surrounding scope, it would be a NameError, but a type checker should have no problem with this (it's just a local type alias).
The problem with such a promise is that it has no teeth, until the future behavior is entirely specified, and then we might as well do it now. My current proposal (no evaluation of annotations for locals) means that you can write complete nonsense there (as long as it is *syntactically* correct) and Python would allow it. Surely somebody is going to come up with a trick to rely on that and then the future development would break their code.
I'm unclear on what you mean by "more generic variable annotations". Do you have an example?
One question: Will the use of the "class" syntax only be allowed at the top level of class definition blocks?
Oooh, very good question. I think that's the idea. Enforcement can't happen directly at the syntactic level, but it can be checked in the same way that we check that e.g. `return` only occurs in a function or `break` and `continue` only in a loop. -- --Guido van Rossum (python.org/~guido)
On 05.08.2016 18:41, Guido van Rossum wrote:
Will arbitrary expressions work or only type declarations? a: <expression> I am asking because https://www.python.org/dev/peps/pep-3107/#parameters is not limited to types. -- Sven
On Fri, Aug 05, 2016 at 11:03:55PM +0200, Sven R. Kunze wrote:
Will arbitrary expressions work or only type declarations?
a: <expression>
My understanding is that the Python interpreter will treat the part after the colon as a no-op. So long as it is syntactically valid, it will be ignored, and never evaluated at all. # Okay spam: fe + fi * fo - fum # SyntaxError spam: fe fi fo fum But the type-checker (if any) should be expected to complain bitterly about anything it doesn't understand, and rightly so. I think that this form of variable annotation should be officially reserved for type hints, and only type hints. -- Steve
On Fri, Aug 5, 2016 at 10:41 AM, Guido van Rossum <guido@python.org> wrote:
Sounds good. I don't think it's likely to be a problem for code that expects __annotations__ only on functions (if such code exists).
I considered that, but before allowing that complexity, I think we should come up with a compelling use case (not a toy example).
Agreed.
Yeah, I think your current approach is good enough.
I'm talking about the idea of using variable annotations for more than just type declarations, just as there are multiple uses in the wild for function annotations. As I said, I'm not terribly interested in the use case and just wanted to point it out. :)
Sounds good. -eric
On Fri, 5 Aug 2016 at 15:27 Eric Snow <ericsnowcurrently@gmail.com> wrote:
Since the information will be in the AST, it would be a more generic solution to come up with a way to keep the AST that was used to generate a code object around. I'm not saying that this is a specific reason to try and tackle that idea (although it does come up regularly), but I think it's a reasonable thing to punt on if this is a better all-around solution in this case.
On Fri, Aug 5, 2016 at 3:37 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
My current bias is towards not using annotations in function bodies unless mypy insists on them (as it does in some corner cases where the type inference falls short or requires too much looking ahead). In classes my bias is towards fully specifying all instance variables, because they serve an important documentation purpose. -- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016 at 2:32 PM Guido van Rossum <guido@python.org> wrote:
My first impression of this given the trivial int and str examples is... Why are you declaring types for things that are plainly obvious? I guess that's a way of saying pick better examples. :) Ones where the types aren't implied by obvious literals on the RHS. Your examples using complex types such as List[int] and Optional[str] are already good ones as that can't be immediately inferred. b: str = module.something(a) is a better example as without knowledge of module.something we cannot immediately infer the type and thus the type declaration might be considered useful to prevent bugs rather than annoying read and keep up to date. I predict it will be more useful for people to declare abstract interface-like types rather than concrete ones such as int or str anyways. (duck typing ftw) But my predictions shouldn't be taken too seriously. I want to see what happens.
I don't like this at all. We only allow pre-declaration without an assignment using keywords today. the 'local' suggestion others have mentioned is worth consideration but I worry any time we add a keyword as that breaks a lot of existing code. Cython uses 'cdef' for this but we obviously don't want that as it implies much more and isn't obvious outside of the cython context. You could potentially reuse the 'def' keyword for this. def a: List[float]. This would be a surprising new syntax for many who are used to searching code for r'^\s*def' to find function definitions. Precedent: Cython already overloads its own 'cdef' concept for both variable and function/method use. Potential alternative to the above def (ab)use: def a -> List[float] def a List[float] def List[float] a # copies the Cython ordering which seems to derive from C syntax for obvious reasons But the -> token really implies return value while the : token already implies variable type annotation. At first glance I'm not happy with these but arguments could be made. Second, when these occur in a class body, they can define either class
Disallowing ": type" syntax in the presence of tuple assignment seems simple and wise to me. Easy to parse. But I understand if people disagree and want a defined way to do it. but this is a slight step back from
When thinking about how to spell this out in a PEP, it is worth taking into account existing ways of declaring types on variables in Python. Cython took the "Keyword Type Name" approach with "cdef double j" syntax. http://cython.readthedocs.io/en/latest/src/quickstart/cythonize.html Is it an error to write the following (poor style) code declaring a type for the same variable multiple times: c: int = module.count_things(x) compute_thing(c) if c > 3: c: str = module.get_thing(3) logging.info('end of thing 3: %s', c[-5:]) do_something(c) where c takes on multiple types within a single scope? static single assignment form would generate a c', c'', and union of c' and c'' types for the final do_something call to reason about that code. but it is entirely doable in Python and does happen in unfortunately real world messy code as variables are reused in bad ways. My preference would be to make it an error for more than one type to be declared for the same variable. First type ever mentioned within the scope wins and all others are SyntaxError worthy. Assigning to a variable in a scope before an assignment that declares its type should probably also be a SyntaxError. -gps
What makes you respond so vehemently to `a: float`? The `def` keyword has been proposed before, but *I* have a vehement response to it; `def` is for functions. The Cython syntax may live forever in Cython, but I don't want to add it to Python, especially since we already have function annotations using `var: type = default` -- variable declarations must somehow rhyme with this. The `local` keyword isn't horrible but would still require a `__future__` import and a long wait, which is why I'm exploring just `var: type = value`. I think we can pull it off syntactically, using a trick no more horrible than the one we already employ to restrict the LHS of an assignment even though the grammar seen by the parser is something like `expr_stmt: testlist ('=' testlist)*` (at least it was something like this long ago -- it's more complex now but the same idea still applies, since the official parser is still LR(1)). Regarding scopes, I like the way mypy currently does this -- you can only have a `# type` comment on the first assignment of a variable, and scopes are flat as they are in Python. (Mypy is really anticipating a syntax for variable declarations here.) Seems we agree on this, at least. On Mon, Aug 1, 2016 at 4:39 PM, Gregory P. Smith <greg@krypto.org> wrote:
-- --Guido van Rossum (python.org/~guido)
On 8/2/2016 1:14 AM, Guido van Rossum wrote:
What makes you respond so vehemently to `a: float`?
I am not Gregory, but I also had a negative reaction. It makes no sense to me. By your rule that annotations are optional and ignorable, 'a: float' is a statement expression consisting of the identifier 'a'. If one ignores or removes the annotations of a function header, one is left with a valid function header with the same meaning. The only runtime effect is on the .annotations attribute, and any consequence of that. If one were to do the same with a (proposed) annotated assignment, would be left with a valid assignment, and there should be no runtime effect either way. If one removes ': float' from 'a: float', one is left with 'a', a single-name expression statement. To be consistent, the addition or removed of the annotation should have no runtime effect here either. The meaning is the statement is 'if a is not bound to anything in the namespace stack, raise NameError'. In batch mode, the else part is ignore it, ie, 'pass'. I have never seen a name expression statement used this way in non-interactive code. It would be an obscure way to control program flow. Interactive mode adds 'else print(a)', which is useful, hence '>>> a' is common. This is not relevant to the offline use of annotations. If 'a' is bound to something, then the annotation belongs on the binding statement. If it is not, then the annotation is irrelevant unless added to the NameError message. If, as I suspect. upi meant 'a: float' to be a different kind of statement, such as a static type declaration for the name 'a', it would be a major change to Python, unlike adding type hints to existing statements. It would make the annotation required, not optional. It would complicate an annotation stripper, as 'a: float' would have to be handled differently from 'a: float = 1.0'. The existing scope declarations are a bit jarring also, but they have the runtime effect of enabling non-local name binding. The best alternative to 'global n' I can think of would have been a tagged '=', such as '=*' meaning 'bind in the global namespace. But this would have had problems both initially and when adding augmented assignments and non-local binding. -- Terry Jan Reedy
On Tue, Aug 2, 2016 at 3:55 PM, Terry Reedy <tjreedy@udel.edu> wrote:
You're taking the "optional and ignorable" too literally. My strawman proposal for the semantics of a: float is that the interpreter should evaluate the expression `float` and then move on to the next line. That's the same as what happens with annotations in signatures. (However this is a potential slow-down and maybe we need to skip it.) The idea of this syntax is that you can point to function definitions and hand-wave a bit and say "just like an argument has an optional type and an optional default, a variable can have either a type or an initializer or both" (and then in very small print continue to explain that if you leave out both the two situations differ, and ditto for multiple assignment and unpacking).
None of that is relevant, really. :-)
But there are no annotation strippers, only parsers that understand the various annotation syntaxes and ignore the annotations.
This is entirely different from global/nonlocal -- the latter are references to previously declared/initialized variables. Here we are declaring new local/class/instance variables. Pretty much the opposite. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 2, 2016 at 6:07 PM, Guido van Rossum <guido@python.org> wrote:
The criticism I would make about allowing variables without assignments like a: float is that it makes my mental model of a variable a little bit more complicated than it is currently. If I see "a" again a few lines below, it can either be pointing to some object or be un-initialized. Maybe the benefits are worth it, but I don't really see it, and I wanted to point out this "cost". Semi-related: what should happen if I do "del a" and then re-use the variable name a few lines below? My feeling is that the type annotation should also get discarded when the variable is deleted.
On 3 August 2016 at 10:48, Alvaro Caceres via Python-ideas <python-ideas@python.org> wrote:
This concern rings true for me as well - "I'm going to be defining a variable named 'a' later and it will be a float" isn't a concept Python has had before. I *have* that concept in my mental model of C/C++, but trying to activate for Python has my brain going "Wut? No.". I'd be much happier if we made initialisation mandatory, so the above would need to be written as either: a: float = 0.0 # Or other suitable default value or: a: Optional[float] = None The nebulous concept (and runtime loophole) where you can see: class Example: a: float ... but still have Example().a throw AttributeError would also be gone. (Presumably this approach would also simplify typechecking inside __new__ and __init__ implementations, as the attribute will reliably be defined the moment the instance is created, even if it hasn't been set to an appropriate value yet) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Aug 3, 2016 at 8:24 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Have you annotated a large code base yet? This half of the proposal comes from over six months of experience annotating large amounts of code (Dropbox code and mypy itself). We commonly see situations where a variable is assigned on each branch of an if/elif/etc. structure. If you need to annotate that variable, mypy currently requires that you put the annotation on the first assignment to the variable, which is in the first branch. It would be much cleaner if you could declare the variable before the first `if`. But picking a good initializer is tricky, especially if you have a type that does not include None. As an illustration, I found this code: https://github.com/python/mypy/blob/master/mypy/checkexpr.py#L1152 if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() # type: Type elif op == '-': method_type = self.analyze_external_member_access('__neg__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type elif op == '+': method_type = self.analyze_external_member_access('__pos__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type else: assert op == '~', "unhandled unary operator" method_type = self.analyze_external_member_access('__invert__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type return result Look at the annotation of `result` in the if-block. We need an annotation because the first functions used to assign it returns a subclasses of `Type`, and the type inference engine will assume the variable's type is that of the first assignment. Be that as it may, given that we need the annotation, I think the code would be clearer if we could set the type *before* the `if` block. But we really don't want to set a value, and in particular we don't want to set it to None, since (assuming strict None-checking) None is not a valid value for this type -- we don't want the type to be `Optional[Type]`. IOW I want to be able to write this code as result: Type if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() elif op == '-': # etc.
That's an entirely different issue though -- PEP 484 doesn't concern itself with whether variables are always initialized (though it's often easy for a type checker to check that). If we wrote that using `__init__` we could still have such a bug: class Example: def __init__(self, n: int) -> None: for i in range(n): self.a = 0.0 # type: float But the syntax used for declaring types is not implicated in this bug.
But, again, real problems arise when the type of an *initialized* instance must always be some data structure (and not None), but you can't come up with a reasonable default initializer that has the proper type. Regarding the question of whether it's better to declare the types of instance variables in `__init__` (or `__new__`) or at the class level: for historical reasons, mypy uses both idioms in different places, and when exploring the code I've found it much more helpful to see the types declared in the class rather than in `__init__`. Compare for yourself: https://github.com/python/mypy/blob/master/mypy/build.py#L84 (puts the types in `__init__`) https://github.com/python/mypy/blob/master/mypy/build.py#L976 (puts the types in the class) -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 03, 2016 at 09:11:42AM -0700, Guido van Rossum wrote:
Just playing Devil's Advocate here, could you use a type hint *comment*? #type result: Type if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() elif op == '-': # etc. Advantages: - absolutely no runtime cost, not even to evaluate and discard the name following the colon; - doesn't (wrongly) imply that the name "result" exists yet; Disadvantages: - can't build a runtime __annotations__ dict; [...]
Again, playing Devil's Advocate... maybe we want a concept of "undefined" like in Javascipt. result: Type = undef which would make it clear that this is a type declaration, and that result is still undefined. Disadvantages: - Javascript programmers will think you can write `print(result)` and get "undef" (or similar); - requires a new keyword. -- Steve
On Wed, Aug 3, 2016 at 10:35 AM, Steven D'Aprano <steve@pearwood.info> wrote:
I like this -- and we need to change the interpreter anyway. I take it this would be a no-op at run time? Though I'm still on the fence -- it's a common idiom to use None to mean undefined. For example, I at least, always thought is was better style to do: class Something(): an_attribute = None and then: if self.an_attribute is None: .... Than not predefine it, and do: if not hasattr(self, 'an_attribute'): .... granted, I only do that for class (or instance) attributes in real code, not all names. So do we really need to support "this variable can be undefined, but if it is defined in can NOT be None? Are there really cases where a variable can't be None, but there is NO reasonable default value? Or are folks hitting a limitation in the type checker's ability to deal with None?
Do we care about that???? BTW, does JS have a None? or is 'undef' its None? -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Chris, From: Python-ideas <python-ideas-bounces+kevin-lists=theolliviers.com@python.org> on behalf of Chris Barker <chris.barker@noaa.gov> Date: Wednesday, August 3, 2016 at 11:33 AM To: Steven D'Aprano <steve@pearwood.info> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484 On Wed, Aug 3, 2016 at 10:35 AM, Steven D'Aprano <steve@pearwood.info> wrote: Again, playing Devil's Advocate... maybe we want a concept of "undefined" like in Javascipt. result: Type = undef which would make it clear that this is a type declaration, and that result is still undefined. I like this -- and we need to change the interpreter anyway. I take it this would be a no-op at run time? Though I'm still on the fence -- it's a common idiom to use None to mean undefined. For example, I at least, always thought is was better style to do: class Something(): an_attribute = None and then: if self.an_attribute is None: .... Than not predefine it, and do: if not hasattr(self, 'an_attribute'): .... granted, I only do that for class (or instance) attributes in real code, not all names. This is SOP for me, too. I think the hassle of dealing with uninitialized variables (and the bugs that result when not being diligent about them) naturally discourages their use. So do we really need to support "this variable can be undefined, but if it is defined in can NOT be None? Are there really cases where a variable can't be None, but there is NO reasonable default value? Or are folks hitting a limitation in the type checker's ability to deal with None? Disadvantages: - Javascript programmers will think you can write `print(result)` and get "undef" (or similar); Do we care about that???? BTW, does JS have a None? or is 'undef' its None? JavaScript has the null keyword in addition to undefined. In my experience with JavaScript, in practice having both a null keyword and an undefined keyword almost always ends up being a source of bugs. Consider the non-intuitive results of the following: var value = undefined; If (value == null) // returns true if value is undefined thanks to type coercion, since null is coerced to false var value = null; If (value == undefined) // returns true if value is null, again thanks to type coercion This is one reason why JS needs the === / !== operators in order to short-curcuit type coercion, and the need to do this is a common source of mistakes among JS developers. I do think Python is better equipped to make this less confusing thanks to the is keyword, etc., but no matter how you look at it, undef, if created, will be something distinct from None and yet sometimes devs will see the two as equivalent. This opens up questions like "do we want a way for None and undef to be equivalent sometimes, but in other cases consider them distinct cases?" I'd say keep it simple. Have developers use None for uninitialized variables if we require assignment, or simply make: a: int implicitly set the value to None if we allow no explicit assignment. Regards, Kevin -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Aug 03, 2016 at 11:33:34AM -0700, Chris Barker wrote:
That was my intention. result: Type = undef would be a no-op at runtime, but the type-checker could take the type-hint from it. There's no actual "undef" value. If you followed that line by: print(result) you would get a NameError, same as today. To answer Chris A's objection in a later post, this is *not* an alternative to the existing `del` keyword. You can't use it to delete an existing name: result = 23 result = undef # SyntaxError, no type given result: Type = undef # confuse the type-checker AND a runtime no-op
Though I'm still on the fence -- it's a common idiom to use None to mean undefined.
Yes, but None is an actual value. "undef" would not be, it would just be syntax to avoid giving an actual value. I don't actually want to introduce a special "Undefined" value like in Javascript, I just want something to say to the program "this is just a type declaration, no value has been defined as yet".
Yes. See Guido's responses. The use-case is: (1) you have a variable that needs a type-hint for a specific type, excluding None; (2) there's no default value you can give it; (3) and it's set in different places (such as in different branches of an if...elif... block). Currently, you have to write: if condition: spam: Widget = Widget(arg) # lots more code here elif other_condition: spam: Widget = load(storage) # lots more code here elif another_condition spam: Widget = eggs.lookup('cheese')['aardvark'] # lots more code here else: spam: Widget = create_widget(x, y, z) # lots more code here which repeats the Widget declaration multiple times. For readability and maintenance, Guido wants to pull the declaration out and put it at the top like this: spam = ???? #type spam:Widget if condition: spam = Widget(arg) # lots more code here # etc. but there's nothing he can put in place of the ???? placeholder. He can't use a specific Widget, because he doesn't know which Widget will be used. He can't use None, because None is not a valid value for spam. Neither of these options are right: spam: Widget = None # fails the type check spam: Optional[Widget] = None # passes the type check here # but allows spam to be None later, which is wrong.
Or are folks hitting a limitation in the type checker's ability to deal with None?
No. The limitation is that there's no way to declare a type-hint without declaring a value at the same time. Hence some suggestions: spam: Widget spam: Widget = undef #type spam: Widget
I shouldn't have specifically said "Javascript programmers". I think it applies to lots of people. And besides, there's probably already people who have a variable called "undef" and turning it into a keyword will break their code.
BTW, does JS have a None? or is 'undef' its None?
Javascript has a null, as well as an undefined value. Thanks to the magic of Javascript's type-coercion rules, they compare equal: [steve@ando ~]$ rhino Rhino 1.7 release 0.7.r2.3.el5_6 2011 05 04 js> a = null; null js> b = undefined; js> print(a, b) null undefined js> a == b; true js> a === b; false In case my intent is still not clear, I DON'T want to introduce an actual "undefined" value, as in Javascript. I only made this suggestion in the hope that it might spark a better idea in others. -- Steve
On Wed, Aug 3, 2016 at 5:52 PM, Steven D'Aprano <steve@pearwood.info> wrote: [...]
[...] Thanks for explaining my use case so well (in the text I snipped). However, I still feel that just result: Type is the better alternative. Syntactically there's no problem with it. It doesn't require a new `undef` keyword or constant, and it doesn't make people think there's an `undef` value. As you mentioned, this has been an unmitigated disaster in JS. -- --Guido van Rossum (python.org/~guido)
Concerning type annotations on local variables of a function: 1) Would they be stored anywhere? If so, where? 2) Would they be evaluated once, or every time the function is called? 3) If they're evaluated once, when and in what scope? -- Greg
On Thu, Aug 04, 2016 at 06:38:00PM +1200, Greg Ewing wrote:
I would think that annotations on local variables should not be evaluated or stored. I think it is reasonable to have annotations on global variables (or inside a class definition) stored somewhere for runtime analysis, like function annotations. -- Steve
On 04.08.2016 15:32, Steven D'Aprano wrote:
I don't know. That somehow would break expected Python semantics and would lead to big asymmetry. Most people don't think about the difference between module-level variables and variables in functions. Their code works regardlessly. The more I think about it, the more I like #comment-style annotations then. They are a clear sign to the developer >not at runtime<, and only available for static analysis. This might even be true for class/instance variables. -- Sven
However the presence of a local declaration like 'a: int' would create a local slot for 'a' as if it were assigned to somewhere in the function. --Guido (mobile)
On Thu, Aug 4, 2016 at 11:22 AM, Guido van Rossum <gvanrossum@gmail.com> wrote:
However the presence of a local declaration like 'a: int' would create a local slot for 'a' as if it were assigned to somewhere in the function.
Does this mean that the following code will raise a NameError? a = None def f(): a: int a (FWIW, I like the <var>: <type> notation more than any alternative proposed so far.)
On Thu, Aug 4, 2016 at 8:40 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
It will do exactly the same as a = None def f(): if False: a = ... a This raises UnboundLocalError (a subclass of NameError).
(FWIW, I like the <var>: <type> notation more than any alternative proposed so far.)
Me too. :-) -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 4, 2016 at 12:15 PM, Guido van Rossum <gvanrossum@gmail.com> wrote:
That's what I expected. Moreover, it looks like there is a precedent for such behavior: $ python3 -O
(Note that the if statement is optimized away.) BTW, a: int looks like an "incomplete" assignment to me. It's a statement that under-specifies a: gives it a type, but not a value. Visually, ':' is a shorter version of '=' and a half of the venerable Pascal's ':='. It all makes sense to me, but your milage may vary if your first language was JavaScript rather than ALGOL. :-)
On Thu, Aug 4, 2016 at 9:44 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Actually, I'm trying to keep some memories of Pascal. In Pascal you write: var age: integer; The family of languages derived from C is already doing enough to keep the ALGOL tradition alive. Anyways, "age: int" matches what you write in function signatures in Python, so I think our hands are tied here and we might as well make the most of it. -- --Guido van Rossum (python.org/~guido)
On 4 August 2016 at 02:11, Guido van Rossum <guido@python.org> wrote:
Ah, that makes sense - given that motivation, I agree it's worth introducing the concept. You'll probably want to make sure to give that example from the mypy code (or a simpler version), as I expect that pre-declaration aspect will be the most controversial part of the whole proposal. I wonder if there's some way we could make that new statement form trigger the following runtime behaviour: a : int a # This raises a new UnboundNameError at module scope, UnboundLocalError otherwise Otherwise we're at risk of allowing thoroughly confusing runtime behaviour like: >>> a = "Not an int" >>> def f(): ... # a: int would go here ... print(a) # This should really fail ... >>> f() Not an int The possibility that springs to mind is a new dedicated opcode, DECLARE_NAME, that works like an assignment that appears anywhere in the function for function namespaces, and does something new for module and class namespaces where it's like an assignment, but doesn't appear in locals(). Depending on how the latter work, we may even be able to raise a new UnboundAttributeError subclass for attempts to access declared-but-not-defined attributes. We'd also want the new syntax to conflict with both global and nonlocal, the same way they currently conflict with each other:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Aug 04, 2016 at 10:37:33PM +1000, Nick Coghlan wrote:
Why wouldn't it raise NameError, as it does now? I understand that `a: int` with no assignment will be treated as a no-op by the interpreter. a: int = 0 will be fine, of course, since that's assigning 0 to a.
I would expect that the type-checker will complain that you're declaring a local variable "a" but there's no local "a" in the function. If the checker isn't that smart, I expect it would complain that "a" is set to a string, but declared as an int. Either way, the type-checker ought to complain. -- Steve
On 4 August 2016 at 23:29, Steven D'Aprano <steve@pearwood.info> wrote:
Guido's reply clarified that he expects the compiler to be declaration aware (so it correctly adds "a" to the local symbol table when a type declaration is given), which means functions will be free of ambiguous behaviour - they'll throw UnboundLocalError if a name is declared and then referenced without first being defined. That means it's mainly in classes that oddities will still be possible:
That's probably acceptable to handle as "Don't do that" though, and leave it to typecheckers to pick it up as problematic "Referenced before definition" behaviour. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Aug 1, 2016 at 10:14 PM Guido van Rossum <guido@python.org> wrote:
What makes you respond so vehemently to `a: float`?
First reaction. It doesn't actually seem too bad now. It is already legal to just say `a` as a statement so this isn't much different. def f(): a is already legal. So `a: List[float]` would at least be meaningful rather than a meaningless statement that any linter should question. :)
I agree, cdef is their own mess. :)
-gps
On Mon, Aug 01, 2016 at 02:31:16PM -0700, Guido van Rossum wrote:
Those examples look reasonable to me. [...]
I would think so. Consider the case that you have Class.spam and Class().spam which may not be the same type. E.g. the class attribute (representing the default value used by all instances) might be a mandatory int, while the instance attribute might be Optional[int].
Require parens around the name:hint. (a:int), (b:str) = 0, '' Or just stick to a type hinting comment :-) What about this case? spam, eggs = [1, 2.0, 'foo'], (1, '') [a, b, c], [d, e] = spam, eggs That becomes: [(a:int), (b:float), (c:str)], [(x:int), (y:str)] = spam, eggs which is bearable, but just unpleasant enough to discourage people from doing it unless they really need to. -- Steve
Regarding class variables: in my experience instance variables way outnumber class variables, except when the latter are used as defaults for instance variables. See e.g. this bit of code from mypy (and many others, actually): https://github.com/python/mypy/blob/master/mypy/nodes.py#L154. All of these declare instance variables. Many of them have no need for a default in the class, but must give one anyway or else there's nothing to put the type comment on -- you can't write defs # type: List[Statement] (it would be a NameError) and you certainly don't want to write defs = [] # type: List[Statement] (else the gods of shared mutable state will curse you) so you have to make do with defs = None # type: List[Statement] But this would be totally reasonable as defs: List[Statement] under my proposal. On Mon, Aug 1, 2016 at 7:55 PM, Steven D'Aprano <steve@pearwood.info> wrote:
-- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016, at 02:31 PM, Guido van Rossum wrote:
To me, option A ("a, b: int, str") and option B ("a: int, b: str") are both... overly dense. If I had to choose between one, I'd choose B. Strongly. A is flatly wrong-- 'b' looks like it binds to 'int', but in fact, it binds to 'str'. This contradicts function annotations. But I still don't really like B. To me, I'd just disallow variable annotations in unpacking. Unpacking is a wonderful feature, I love unpacking, but to combine it with annotations is a cognitive overload. Maybe that means you don't use unpacking sometimes: that's okay. Unpacking is about being concise, to me at least, and if I'm doing annotations, that means I'm accepting being more verbose for the sake of static checking. You can't be all things to all people. In this case, I say, someone has to pick which their priority is: conciseness or static checking. -- Stephen Hansen m e @ i x o k a i . i o
On 01.08.2016 23:31, Guido van Rossum wrote:
I can't help but this seems like a short "key: value" dict declaration. Besides the fact that those examples don't really highlight the real use-cases. But others already covered that.
Then writing "a" should be allow, too, right? As a variable declaration without any type hint. That's usually a NameError. Put it differently, what is the use-case Python has missed so long in not providing a way of declaring an empty variable? Will it be filled with None?
Could be useful but might result in a lot of double maintenance work (class body + place of initilization).
Or using "a: float" from above: a: float b: str a, b = 0, '' So, ignoring commas for now and solving it later would also work. Sven
On Wed, Aug 3, 2016, 6:49 AM Sven R. Kunze <srkunze@mail.de> wrote:
Nope, it wouldn't do anything at all. No bytecode. No namespace updated. No None. It is merely an informational statement that an optional type checker pass may make use of. This definitely counts as a difference between a bare 'a' and 'a: SPAM'. The former is a name lookup while the latter is a no-op. Good or bad I'm undecided. At least the latter is useful while the former is more of a side effect of how Python works.
I'm behind Gregory here, as I'm not a big fan of the "a: float" syntax (but I don't have a strong opinion either way). I'm strongly against merely "a" being equal to "a = None", though. In my code, that's either a typo or I've wrapped that in a try: ... except NameError: ..., and I'd like Python to throw a NameError in that case. So either we deal with an asymmetry in how annotations work, or we disallow it. I'm likely not going to use that particular syntax even if it exists, so I'm -0. -Emanuel
From Gregory P. Smith <greg@krypto.org> Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484
On Wed, Aug 3, 2016, 6:49 AM Sven R. Kunze <srkunze@mail.de<mailto:srkunze@mail.de>> wrote: On 01.08.2016 23:31, Guido van Rossum wrote:
Then writing "a" should be allow, too, right? As a variable declaration without any type hint. That's usually a NameError. Put it differently, what is the use-case Python has missed so long in not providing a way of declaring an empty variable? Will it be filled with None? Nope, it wouldn't do anything at all. No bytecode. No namespace updated. No None. It is merely an informational statement that an optional type checker pass may make use of. This definitely counts as a difference between a bare 'a' and 'a: SPAM'. The former is a name lookup while the latter is a no-op. Good or bad I'm undecided. At least the latter is useful while the former is more of a side effect of how Python works.
If I understand this proposal then we probably need to consider this too: if something: a: float else: a: str a = 'what would static type-checker do here?' del a a: int = 0 def fnc(): global a: list a = 7 fnc() a = [1, 2, 3] # and this could be interesting for static type-checker too
On Wed, Aug 3, 2016 at 3:02 PM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
The beauty of it is that that's entirely up to the static checker. In mypy this would probably be an error. But at runtime we can make this totally well-defined.
del a a: int = 0
Ditto.
def fnc(): global a: list
I'm not proposing to add such syntax, and the right place for the type of a would be at the global level, not on the `global` syatement.
Indeed, but that's not what we're debating here. -- --Guido van Rossum (python.org/~guido)
On 8/4/16, Guido van Rossum <guido@python.org> wrote:
On Wed, Aug 3, 2016 at 3:02 PM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
Sorry but for me is really important where we are going (at least as inspiration). As I understand now this changes could end in code which could be pure python and on other hand also compilable by static typed compiler too. Maybe it is what we wanted (I prefer this one), maybe we doesnt care and maybe we want to avoid it. For example Cython's code: cdef int n could be writen: n: cdef.int or n: 'cdef int' and I think it could be good to see convergence here. And maybe this cdef int n, k, i could be inspiring too allow one to many possibility n, j, k: int # (1) I think it is natural and better than n, j, k: int, int, int But (1) would be discordant with this (from PEP484) def __init__(self, left: Node, right: Node)
On Thu, Aug 4, 2016 at 12:16 AM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
But you are thinking with your runtime hat on. To a static type checker, the call to fnc() is irrelevant to the type of the global variable `a`. It's like writing def foo(): return int def bar(a: foo()): return a+1 The static type checker rejects `foo()` as an invalid type, even though you know what it means at runtime. (But what if foo() were to flip a coin to choose between int and str?)
Yes, that should be possible. Just don't run mypy over that source code. :-)
And that's why I really don't want to go there. What if someone wrote T = Tuple[int, int, str] a, b, c: T Do we now have three Tuple[int, int, str] variables, or two ints and a str? -- --Guido van Rossum (python.org/~guido)
I'll post here another wild idea (based on the https://github.com/python/ mypy/blob/master/mypy/build.py#L976 example), just thinking on instance attributes. I myself am not fully sold on it but it's quite different so perhaps adds a new angle to the brainstorming and has some nice upsides: class State: def __instancevars__( manager: BuildManager, order: int, # Order in which modules were encountered id: str, # Fully qualified module name path: Optional[str], # Path to module source xpath: str, # Path or '<string>' source: Optional[str], # Module source code meta: Optional[CacheMeta], data: Optional[str], tree: Optional[MypyFile], dependencies: List[str], suppressed: List[str], # Suppressed/missing dependencies priorities: Dict[str, int] ): ... The benefits of this is: * No need to add new syntax or keywords here, this works in any 3.x * This part of the change can be done right now without waiting for python 3.6 * it's guaranteed to be consistent with function annotations * The instance vars annotations end up in a standard place on runtime if someone needs them (run time checking, documentation generators, etc). * No confusing access to a name that may look like a NameError * Very obvious runtime behaviour for python users even if they don't use a typechecker. * This can even be used in python 2.x, by changing the annotation to a "# type: T" along the argument. The downsides are: * There is a bit of boilerplate (the parenthesis, the ellipsis, the extra commas) * It's unclear what the body of this function is supposed to do, but probably the typechecker can ensure it's always "..." (I'm assuming that whoever writes this wants to use a type checker anyway) * It's also unclear what a default argument would mean here, or *args, or **kwargs (again, the typechecker could enforce proper usage here) * It doesn't cover locals and class variables, so it's still necessary the new syntax for "type declaration and initialization assignment" (but not the "declaration without assignment") As a second order iterations on the idea * If this idea works well now it's easier to add nicer syntax later that maps to this (like a block statement) and specify the runtime semantics in term of this (which is what I see the traditional way in python) * It could be possible to give semantic to the body (like a post __new__ thing where locals() from this function are copied to the instance dict), and this would allow setting real instance defaults () but this is new class creation semantics and can have a performance cost on instantiation when having types declared Hope this adds to the discussion, On Thu, Aug 4, 2016 at 9:46 AM, Sven R. Kunze <srkunze@mail.de> wrote:
-- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 4:03 AM, Daniel Moisset <dmoisset@machinalis.com> wrote:
That feels too hacky for a long-term solution, and some of your downsides are very real. The backwards compatible solution is to keep using type comments (with initial values set to None, or `...` if Python 2 isn't needed). But thanks for thinking creatively about the problem! -- --Guido van Rossum (python.org/~guido)
On 2016-08-01 14:31, Guido van Rossum wrote:
Let me ask a perhaps silly question. Reading a lot of subsequent messages in this thread, it seems that the main intended use for all this is external static type checkers. (Hence all the references to thinking with "runtime hats" on.) Given this, what is the benefit of making this change to Python syntax? If it's only going to be used by static checker tools, is there any difference between having those tools grab it from comments vs. from "real" syntax? This seems doubly questionable if, for local variables, the type annotations are totally unavailable at runtime (which I think is what was suggested). It seems odd to have Python syntax that not only doesn't do anything, but can't even be made to do anything by the program itself when it runs. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, Aug 4, 2016 at 2:33 PM, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
It seems odd to have Python syntax that not only doesn't do anything, but can't even be made to do anything by the program itself when it runs.
Why do you find this odd? Python already has #-comment syntax with exactly the property you complain about. Think of local variable annotation as a structured comment syntax.
On 2016-08-04 12:09, Alexander Belopolsky wrote:
Exactly. But the existing comment annotations (# type: float) already are structured comment syntax, so what does this new one add? (Note we already have at least one parallel --- although it is much more limited in scope --- namely encoding declarations, which are also done within existing comment syntax, rather than having their own "real" syntax.) -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, 4 Aug 2016 at 12:20 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
[I'm going to give an explanation one shot, then I will leave the topic alone as explaining the "why" definitely has the possibility of dragging on] You're right this could be viewed as syntactic sugar, but then again so is multiplication for integers as a short-hand for addition or the fact we even have infix operators which translate to method calls on objects. In all cases, the syntax is meant to either make people's lives easier to to promote their use. In this instance it both makes people's lives easier and promotes use. By making it syntax it becomes easier to use as the compiler can now detect when the information is improperly typed. And by making it syntax, not only does it make sure there's a strong standard but lets people know that this isn't some little feature but in fact is being promoted by the Python language itself. There's also the disconnect of having type annotations on function parameters but completely missing the OOP aspect of attributes on objects which leaves a gaping hole in terms of syntactic support for type hints. Hopefully that all makes sense as to why Guido has brought this up.
On Tue, Aug 2, 2016 at 7:31 AM, Guido van Rossum <guido@python.org> wrote:
Additional case, unless it's patently obvious to someone with more 484 experience than I: what happens with chained assignment? a = b = c = 0 Does each variable get separately tagged, or does one tag apply to all? ChrisA
On 08/01/2016 02:40 PM, Chris Angelico wrote:
On Tue, Aug 2, 2016 at 7:31 AM, Guido van Rossum wrote:
I would think each name would need to be typed: a: int = b: int = c: int = 0 However, if somebody was doing typing from the get-go I would imagine they would do: a, b, c: float -- ~Ethan~
On Mon, Aug 1, 2016 at 2:46 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
The problem with this is that the relative priorities of '=' and ',' are inverted between argument lists and assignments. And the expression on the right might be a single variable whose type is a tuple. So we'd get a: int, b: str = x But the same thing in a function definition already has a different meaning: def foo(a: int, b: str = x): ... -- --Guido van Rossum (python.org/~guido)
On 2016-08-01 15:58, Guido van Rossum wrote:
Is that really a big deal? Insofar as it's a problem, it already exists for function arguments vs. assignments without any type annotation. That is: a, b = x already means something different from def foo(a, b = x): ... So in one sense, keeping type annotations with the variables would actually maintain the current convention (namely, that assignment and argument defaults have different conventions). -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Mon, Aug 1, 2016 at 2:46 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
But what would this do? a: int, b: str = x Does the x get distributed over (a, b) or does a remain unset? The analogy with assignment suggest that x gets distributed, but the analogy with function definitions says x only goes to b. -- --Guido van Rossum (python.org/~guido)
On 08/01/2016 04:31 PM, Guido van Rossum wrote:
When speaking of the function header I meant that we already have one way to say which name has which type, and it would be simpler (at least to understand) if we stick with one way to specify type. As far as questions such as "what would this do" I would say it should do the exact same thing as if the type info wasn't there: a: int, b: str = x simplifies to a, b = x and so x had better be a two-item iterable; on the other hand: def func(a: int, b: str = x): simplifies to def func(a, b=x): and so parameter a is mandatory while parameter b has a default and so is optional. -- ~Ethan~
On 2 August 2016 at 00:41, Ethan Furman <ethan@stoneleaf.us> wrote:
From what Guido says, the main use case is class variables, where complicated initialisations are extremely rare, so this shouldn't be a
For me, a: int, b: str = x immediately says "leave a unassigned, and assign x to b". Maybe that's my C experience at work. But a, b = x immediately says tuple unpacking. So in my view, allowing annotations on unpacking syntax is going to cause a *lot* of confusion, and I'd strongly argue for only allowing type annotations on single variables: VAR [: TYPE] [= INITIAL_VALUE] For multiple variables, just use multiple lines: a: int b: str = x or a: int b: str a, b = x depending on your intention. problem in practice. Paul
Just to add some data points, here is how it is done in some other languages which have both types and tuple unpacking. In all cases I added the minimum of parentheses required. Scala: val (x : Int, y : String) = (42, "Hello") Ocaml: let (x: int), (y : string) = 42, "Hello" alternatively: let (x, y) : int * string = 42, "Hello" Note that if we omit the types we can do the Python-esque: let x, y = 42, "Hello" Stephan 2016-08-02 9:54 GMT+02:00 Paul Moore <p.f.moore@gmail.com>:
On 2 August 2016 at 17:54, Paul Moore <p.f.moore@gmail.com> wrote:
+1 from me for this view: only allow syntactic annotation of single names without parentheses. That one rule eliminates all the cases where unpacking assignment and parameter assignment do starkly different things. *However*, even with that restriction, unambiguously annotated unpacking could still be supported via: (a, b) : Tuple[int, str] = x (a, b, *c) : Tuple[int, int, Tuple[int]] = y Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Aug 1, 2016 at 3:31 PM, Guido van Rossum <guido@python.org> wrote:
As noted by someone else, what about "local": local a: float It seems like nonlocal and global would need the same treatment: nonlocal b: List[int] global c: Optional[str]
Isn't that currently a NameError? Is it worth making this work while preserving the error case? A "local" keyword would solve the problem, no?
In the immediate case I'd expect the former. We don't currently have a canonical way to "declare" instance attributes in the class definition. It may be worth sorting that out separately and addressing the PEP 484 aspect at that point.
Third, there's an annoying thing with tuples/commas here.
What about unpacking into explicit displays? For example: (a, b) = 0, '' [a, b] = 0, '' -eric
On Mon, Aug 1, 2016 at 11:40 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
The last 2 examples are actually wrong. The use of "nonlocal" and "global" is inside functions, to refer to variables that were originally defined *outside* the function. The type declaration should be near the definition (in the global/outer scope), not on its use. If we're looking for a keyword-based syntax for empty definitions, one option would be: def a: float It uses an existing keyword so no risk of name collissions, the meaning is quite accurate (you're defining a new variable), and my guess is that the syntax is distinct enough (no parenthesis or final colons which would indicate a function definition). OTOH it might break some usages of grep to look for functions, and autoindenters on code editors. -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Mon, Aug 1, 2016 at 3:40 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
A common use of `# type:` is in class bodies. But `local` is a bad idea there.
Well, they don't support assignment either, so I'd like to keep them out of this if possible.
Currently looks like a syntax error to me.
My observation is that most uses of `# type: ...` in class bodies is used to declare an instance variable.
Yeah. -- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016 at 10:31 PM, Guido van Rossum <guido@python.org> wrote:
I'd say that if I have a class C with a class variable cv, instance variable iv, a good type checking system should detect: C.cv # ok C.iv # error! C().iv # ok which is something that PEP484 doesn't clarify much (and mypy flags all 3 as valid) So in short, I think it is relevant to specify differently class vs instance vars. Suppose we wanted to add types to the latter. Would we write this as
I'm not specially fond of the «# type: (int, str)». It works ok for 2 variables, but for more it is hard to mentally "align" the variable names to the type names, for example in: kind, text, start, end, line = token # type: (int, str, Tuple[int, int], Tuple[int, int], str) it's not easy to quickly answer "what is the type of 'end' here?". So I wouldn't miss that notation a lot if it went away Given that, I'm happier with both the 2-line solution and the second one-liner, which both keep the types closer to the names. But given that as you mentioned: kind: int, text:str, start: Tuple[int, int], end: Tuple[int, int], line: str = token looks a bit misleading (looks more like an assignment to token), perhaps it would avoid errors to accpt as valid only: (kind: int, text:str, start: Tuple[int, int], end: Tuple[int, int], line: str) = token other possibility if you really love the current mypy syntax (perhaps both could be valid): (kind, text, start, end, line):(int, str, Tuple[int, int], Tuple[int, int], str) = token I don't like that one very much, but perhaps it inspires ideas on someone here. Other places to think about are: * Multiple assignment (Chris mentioned these) * loop variables (in a for statement, comprehensions, generator expressions) * lambda arguments -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Mon, Aug 1, 2016 at 4:35 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
Yeah, this is all because you can't express that in Python either. When you see an assignment in a class body you can't tell if it's meant as an instance variable default or a class variable (except for some specific cases -- e.g. nested class definitions are pretty obvious :-).
So in short, I think it is relevant to specify differently class vs instance vars.
Agreed, we need to invent a workable proposal for this. Here's a strawman: - The default is an instance variable (backed by a class variable as default if there's an initial value) - To define a class variable, prefix the type with 'class` Example: class C: a: int # instance var b: List[int] = None # instance var c: class List[int] = [] # class var Class variables must come with an initializer; instance variables may or may not have an initializer. (Bonus: instance variable initializers must be immutable values.) Regarding everything involving multiple variables (either `a = b = 0` or `a, b = 0, 0`) I propose that those cannot be combined with types. Period. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 2, 2016 at 11:09 PM, Guido van Rossum <guido@python.org> wrote:
May be I've gotten wrong my python style for many years, but I always considered that the "proper" way to create instance variables was inside the initializer (or in rare occasions, some other method/classmethod). For me, an assignment at a class body is a class variable/constant. So I was going to propose "type declarations at class level are always for class variables, and inside methods (preceded by "self.") are for instance variables". Using class level variables for defaults always seemed unpythonic and error prone (when mutable state is involved) to me. I felt that was common practice but can't find documentation to support it, so I'd like to hear if I'm wrong there :)
--
Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On 3 August 2016 at 11:51, Daniel Moisset <dmoisset@machinalis.com> wrote:
You've just missed one of the most powerful side-effects of how Python's class and instance variables interact: class Spaceship: color = RED hitpoints = 50 def powerup(self): self.color = BLUE self.hitpoints += 100 Above: the defaults are good for almost all spaceship instaces - but when one of them is "powered up", and only them, that instance values are defined to be different than the defaults specified in the class. At that point proper instance variables are created in the instanceś __dict__, but for all other instances, the defaults, living in the instance's".__class__.__dict__" are just good enough.
I may have missed that on the message deluge so far - but would the type annotations be available at runtime, like parameter annotations live in "__annotations__"? An __annotations__ dict straight in the class itself? I thinkt hat is the obvious thing - although I did not see it in the above messages. (A s I said, I may have missed its mention) That is ratehr important because then, beyond enabling static code analysis it is easy to have third party frameworks that enforce typing in runtime. On 3 August 2016 at 12:13, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
On Wed, Aug 3, 2016 at 11:31 AM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
It was mentioned that var annotations would be evaluated and discarded, but I agree that this is less than ideal. I agree that it is better to store var annotations in the namespace that they appear in. Maybe something like this: a: int = 0 b: List[float] = [] would result in a namespace { 'a': 0, 'b': [], '__var_annotations__': { 'a': int, 'b': List[float], }, }
On 3 August 2016 at 16:40, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Agreed, it seems a shame that this would be an area where it's not possible to introspect data that was available at compile time. (I'd say it doesn't feel Pythonic, except that claiming that Guido's proposing something non-Pythonic is self-contradictory :-)) However, I have no expectation of ever needing this data for anything I'd write, so it's not actually something that would matter to me one way or the other. Paul
Guido is the BDFL of CPython, not the arbiter of what the community thinks is 'pythonic'. It is yet to be seen if type annotations are considered pythonic at all. Under the assumption that they are, yes, annotations that also come with initialization would be the more pythonic way of doing things.
On Thu, Aug 4, 2016 at 1:40 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
So... the maligned statement "a: int" would actually mean "__var_annotations__['a'] = int", which is a perfectly valid executable statement. This might solve the problem? Question, though: Does __var_annotations__ always exist (as a potentially empty dict), or is it created on first use? A function's __annotations__ is always present, but functions have lots of attributes that we normally don't care about. Polluting every single namespace seems wasteful; magically creating a new local variable when you first hit an annotated 'declaration' seems odd. ChrisA
On Wed, Aug 3, 2016 at 8:31 AM, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Actually we've hardly touched on that yet. For classes I think it would be nice to make these available. For locals I think we should not evaluate the type at all, otherwise the cost of using a variable declaration would be too high.
Although, frankly, that's not something that PEP 484 particularly cares about. Runtime type checking has very different requirements -- would you really want to check that a List[int] contains only ints if the list has a million items? -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 16:13, Joao S. O. Bueno <jsbueno@python.org.br> wrote:
Yes, but I view that as "when you ask for an instance variable, if there isn't one you get the class variable as the default - nested namespaces basically, just like local variables of a function and global variables. So to me class Spaceship: hitpoints: int = 50 declares a class Spaceship, with a *class* variable hitpoints. Instances will get that value by default, but can assign something different.
class Whatever: myval : int = 20 def __init__(self): self.myval = "Fooled you!" The problem is of course that there's no way to attach a type declaration to an instance variable, unless you allow self.myval : str = "Fooled you" which opens up the possibility of declaring types for (in theory) arbitrary expressions. In summary, I think type annotations on variables declared at class scope should describe the type of the class variable - because that's what the assignment is creating. That leaves no obvious means of declaring the type of an instance variable (can you put a comment-style type annotation on the self.x = whatever line at the moment?) Which is a problem, but not one (IMO) that should be solved by somehow pretending that when you declare a class variable you're actually declaring an instance variable. Paul
On Wed, Aug 3, 2016 at 8:35 AM, Paul Moore <p.f.moore@gmail.com> wrote: [...]
But here you're thinking with your runtime hat on. The type checker would actually like to understand what you mean here -- a class variable (maybe it can be updated by saying `Spaceship.hitpoints += 10`) or an instance variable. Which is why I proposed that you can make it a class variable by saying `hitpoints: class int = 50`, and the default would be for it to be an instance variable. I propose that the difference is that class variables cannot be updated through the instance (as that would make it an instance variable, which you've explicitly promised not to do by using the `class int` type).
Again you're thinking with your runtime hat on. If you were to run this through a type checker you'd *want* to get an error here (since realistically there is probably a lot more going on in that class and the type clash is a symptom of a poor choice for a variable name or a bad initializer).
We should probably allow that too, but only for assignment to instance variables using `self` (it's easy for the type checker to only allow certain forms -- the runtime should be more lenient). A type checker that warns about instance variables used or set without a declaration could be very useful indeed (catching many typos).
I look at it from a more pragmatic side. What do I want my type checker to check? The annotations need to be useful to help me catch bugs but not so annoying that I have to constantly override the type checker. From this perspective I definitely want a way to declare the types of instance variables at the class level. Packages like SQLAlchemy and Django and traits, that have each developed their own, quite sophisticated machinery for declaring instance variables, seem to support this perspective. -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 17:26, Guido van Rossum <guido@python.org> wrote:
But here you're thinking with your runtime hat on
Yep, that's the mistake I was making. Thanks for clarifying. I've yet to actually use a type checker[1], so I'm still not used to thinking in terms of "typecheck-time" behaviour. Paul [1] Off-topic, but I'm not sure if type annotations are powerful enough (yet) to track bytes vs str sufficiently to help with resolving Python 2 -> Python 3 string handling errors. When I get a chance to confirm that's possible, I definitely have some candidate codebases I'd like to try my hand with :-)
On Wed, Aug 3, 2016 at 10:29 AM, Paul Moore <p.f.moore@gmail.com> wrote:
That's unfortunate -- the feature I'm developing here is only of interest to people using a type checker, and the many details of what it should look like and how it should work will benefit most from feedback from people who have actually dealt with the existing way of declaring variables.
Not yet, but we're working on it. See https://github.com/python/typing/issues/208 -- --Guido van Rossum (python.org/~guido)
On 3 August 2016 at 19:01, Guido van Rossum <guido@python.org> wrote:
Understood. Mostly I'm keeping out of it - I chimed in because I (mistakenly) thought the way of declaring instance variables clashed with a non-(typing)-expert's understanding of what was going on (defining a class variable which is used as a fallback value if there's no instance variable of the same name). I still think the notation could be confusing, but the confusion can be fixed with documentation, and I'm happy to propose fixes for documentation when I finally get round to using the feature. Sorry for the noise.
Great :-) Paul
There's something I found a bit confusing in the runtime semantic of defaults in your strawman proposal: On Tue, Aug 2, 2016 at 11:09 PM, Guido van Rossum <guido@python.org> wrote:
Or you are proposing that in runtime these are equivalent to "b, c = None, []" ? If that's the case, I find it misleading to call them "instance variables" vs "class variables", given that those concepts are supposed to have different runtime semantics, not just for the typechecker. -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 7:12 AM, Daniel Moisset <dmoisset@machinalis.com> wrote:
The latter (except possibly for also storing the types in __annotations__). I'm a little worried by your claim that it's misleading to distinguish between instance and class variables. There really are three categories: - pure class variable -- exists in class __dict__ only - pure instance variable -- exists in instance __dict__ only - instance variable with class default -- in class __dict__ and maybe in instance __dict__ (Note that even a pure class variable can be referenced as an instance attribute -- this is the mechanism that enables the third category.) Pragmatically, the two categories of instance variables together are wildly more common than pure class variables. I also believe that the two categories of instance variables are both pretty common. So I want the notation to support all three: class Starship: stats: class Dict[str, int] = {} # Pure class variable damage: class int = 0 # Hybrid class/instance variable captain: str # Pure instance variable The intention is that the captain's name must always be set when a Starship is initialized, but all Starships start their life with zero damage, and stats is merely a place where various Starship methods can leave behind counters logging how often various events occur. (In real code it would probably be a defaultdict.) -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 4, 2016 at 5:40 PM, Guido van Rossum <guido@python.org> wrote:
For me what's misleading is the runtime behaviour for b, which has na initializer but is not flagged as class variable... see below
I follow the example perfectly. Now suppose a reader finds the following piece of code: class Starship: stats: class Dict[str, int] = {} # Pure class variable damage: class int = 0 # Hybrid class/instance variable captain: str # Pure instance variable speed: float = 0 I added a new attribute (similar to b in your original example). Given that the type declaration doesn't say "class",the reader might be inclined to think it's an instance variable. But in runtime (if I got you right), that variable will be stored in "Starship.__dict__" and writing "Starship.speed = 3" will change the speed of those starship instances that still haven't set the attribute. So in the end both "damage" and "speed" have "class variable" runtime semantics, even when one is flagged as "class" and the other isn't. The other combination that feels a bit confusing when adding "class" tags is saying "attr: class T", without an initializer.... in what case would someone do that? what does it mean if I find some code saying that about the class, that it might get that attribute set somewhere else? -- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 12:11 PM, Daniel Moisset <dmoisset@machinalis.com> wrote: [...]
We may have to debate more what the checker should allow here. (Since it's okay for a type checker to disallow things that might actually work at runtime.) I'm inclined to allow it, as long as the value assigned to Starship.speed is compatible with float.
Agreed that looks silly. We probably should make that a syntax error. -- --Guido van Rossum (python.org/~guido)
On 5 August 2016 at 02:40, Guido van Rossum <guido@python.org> wrote:
I'm mostly on board with the proposal now, but have one last niggle: What do you think of associating the "class" more with the variable name than the type definition for class declarations by putting to the left of the colon? That is: class Starship: stats class: Dict[str, int] = {} # Pure class variable damage class: int = 0 # Hybrid class/instance variable captain: str # Pure instance variable Pronounced as: "stats is declared on the class as a dict mapping from strings to integers and is initialised as an empty dict" "damage is declared on the class as an integer and is initialised as zero" "captain is declared on instances as an integer" Just a minor thing, but the closer association with the name reads better to me since "Class attribute or instance attribute?" is really a property of the name binding, rather than of the permitted types that can be bound to that name Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 5, 2016 at 12:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Hmm... But the type is *also* a property of the name binding. And I think the "class-var-ness" needs to be preserved in the __annotations__ dict somehow, so that's another reason why it "belongs" to the type rather than to the name binding (a nebulous concept to begin with). Also, I like the idea that everything between the ':' and the '=' (or the end of the line) belongs to the type checker. I expect that'll be easier for people who aren't interested in the type checker. -- --Guido van Rossum (python.org/~guido)
On 6 August 2016 at 02:12, Guido van Rossum <guido@python.org> wrote:
Fair point, although this and the __var_annotations__ discussion also raises the point that annotations have to date always been valid Python expressions. So perhaps rather than using the class keyword, it would make sense to riff off classmethod and generic types to propose: class Starship: stats: ClassAttr[Dict[str, int]] = {} # Pure class variable damage: DefaultAttr[int] = 0 # Hybrid class/instance variable captain: str # Pure instance variable Pronounced as: "stats is a class attribute mapping from strings to integers and is initialised as an empty dict" "damage is an instance attribute declared as an integer with a default value of zero defined on the class" "captain is an instance attribute declared as a string" In addition to clearly distinguishing class-only attributes from class-attribute-as-default-value, this would also mean that the specific meaning of those annotations could be documented in typing.ClassAttr and typing.DefaultAttr, rather than needing to be covered in the documentation of the type annotation syntax itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Aug 5, 2016 at 1:40 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Same here. The only thing I'd like considered further is exposing the annotations at runtime. Others have already suggested this and Guido has indicated that they will not be evaluated at runtime. I think that's fine in the short term, but would still like future support considered (or at least not ruled out) for runtime evaluation (and availability on e.g. func/module/cls.__var_annotations__). If performance is the main concern, we should be able to add compiler support to evaluate/store the annotations selectively on a per-file basis, like we do for __future__ imports. Alternately, for functions we could evaluate the annotations in global/non-local scope at the time the code object is created (i.e. when handling the MAKE_FUNCTION op code), rather than as part of the code object's bytecode. Again, not adding the support right now is fine but we should be careful to not preclude later support. Of course, this assumes sufficient benefit from run-time access variable type annotations, which I think is weaker than the argument for function annotations. We should still keep this possible future aspect in mind though. Relatedly, it would be nice to address the future use of this syntax for more generic variable annotations (a la function annotations), but that's less of a concern for me. The only catch is that making "class" an optional part of the syntax impacts the semantics of the more generic "variable annotations". However, I don't see "class" as a problem, particularly if it is more strongly associated with the name rather than the annotation, as you've suggested below. If anything it's an argument *for* your recommendation. :)
+1 One question: Will the use of the "class" syntax only be allowed at the top level of class definition blocks? -eric
On Fri, Aug 5, 2016 at 12:23 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
I had exactly the same thoughts, but I wonder how likely the type annotations would need local scope. For example, is something like this def f(x): v: type(x) ... going to be supported by type checkers?
On Fri, Aug 5, 2016 at 9:23 AM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
Actually my current leaning is as follows: - For annotated module globals, the type annotations will be evaluated and collected in a dict named __annotations__ (itself a global variable in the module). - Annotations in class scope are evaluated and collected in a dict named __annotations__ on the class (itself a class attribute). This will contain info about annotated class variables as well as instance variables. I'm thinking that the class variable annotations will be wrapped in a `ClassVar[...]` object. - Annotations on local variables are *not* evaluated (because it would be too much of a speed concern) but the presence of an annotation still tells Python that it's a local variable, so a "local variable slot" is reserved for it (even if no initial value is present), and if you annotate it without initialization you may get an UnboundLocalError, e.g. here: def foo(): x: int print(x) # UnboundLocalError This is the same as e.g. def bar(): if False: x = 0 print(x) # UnboundLocalError
I considered that, but before allowing that complexity, I think we should come up with a compelling use case (not a toy example). This also produces some surprising behavior, e.g. what would the following do: def foo(): T = List[int] a: T = [] # etc. If we evaluate the annotation `T` in the surrounding scope, it would be a NameError, but a type checker should have no problem with this (it's just a local type alias).
The problem with such a promise is that it has no teeth, until the future behavior is entirely specified, and then we might as well do it now. My current proposal (no evaluation of annotations for locals) means that you can write complete nonsense there (as long as it is *syntactically* correct) and Python would allow it. Surely somebody is going to come up with a trick to rely on that and then the future development would break their code.
I'm unclear on what you mean by "more generic variable annotations". Do you have an example?
One question: Will the use of the "class" syntax only be allowed at the top level of class definition blocks?
Oooh, very good question. I think that's the idea. Enforcement can't happen directly at the syntactic level, but it can be checked in the same way that we check that e.g. `return` only occurs in a function or `break` and `continue` only in a loop. -- --Guido van Rossum (python.org/~guido)
On 05.08.2016 18:41, Guido van Rossum wrote:
Will arbitrary expressions work or only type declarations? a: <expression> I am asking because https://www.python.org/dev/peps/pep-3107/#parameters is not limited to types. -- Sven
On Fri, Aug 05, 2016 at 11:03:55PM +0200, Sven R. Kunze wrote:
Will arbitrary expressions work or only type declarations?
a: <expression>
My understanding is that the Python interpreter will treat the part after the colon as a no-op. So long as it is syntactically valid, it will be ignored, and never evaluated at all. # Okay spam: fe + fi * fo - fum # SyntaxError spam: fe fi fo fum But the type-checker (if any) should be expected to complain bitterly about anything it doesn't understand, and rightly so. I think that this form of variable annotation should be officially reserved for type hints, and only type hints. -- Steve
On Fri, Aug 5, 2016 at 10:41 AM, Guido van Rossum <guido@python.org> wrote:
Sounds good. I don't think it's likely to be a problem for code that expects __annotations__ only on functions (if such code exists).
I considered that, but before allowing that complexity, I think we should come up with a compelling use case (not a toy example).
Agreed.
Yeah, I think your current approach is good enough.
I'm talking about the idea of using variable annotations for more than just type declarations, just as there are multiple uses in the wild for function annotations. As I said, I'm not terribly interested in the use case and just wanted to point it out. :)
Sounds good. -eric
On Fri, 5 Aug 2016 at 15:27 Eric Snow <ericsnowcurrently@gmail.com> wrote:
Since the information will be in the AST, it would be a more generic solution to come up with a way to keep the AST that was used to generate a code object around. I'm not saying that this is a specific reason to try and tackle that idea (although it does come up regularly), but I think it's a reasonable thing to punt on if this is a better all-around solution in this case.
On Fri, Aug 5, 2016 at 3:37 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
My current bias is towards not using annotations in function bodies unless mypy insists on them (as it does in some corner cases where the type inference falls short or requires too much looking ahead). In classes my bias is towards fully specifying all instance variables, because they serve an important documentation purpose. -- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016 at 2:32 PM Guido van Rossum <guido@python.org> wrote:
My first impression of this given the trivial int and str examples is... Why are you declaring types for things that are plainly obvious? I guess that's a way of saying pick better examples. :) Ones where the types aren't implied by obvious literals on the RHS. Your examples using complex types such as List[int] and Optional[str] are already good ones as that can't be immediately inferred. b: str = module.something(a) is a better example as without knowledge of module.something we cannot immediately infer the type and thus the type declaration might be considered useful to prevent bugs rather than annoying read and keep up to date. I predict it will be more useful for people to declare abstract interface-like types rather than concrete ones such as int or str anyways. (duck typing ftw) But my predictions shouldn't be taken too seriously. I want to see what happens.
I don't like this at all. We only allow pre-declaration without an assignment using keywords today. the 'local' suggestion others have mentioned is worth consideration but I worry any time we add a keyword as that breaks a lot of existing code. Cython uses 'cdef' for this but we obviously don't want that as it implies much more and isn't obvious outside of the cython context. You could potentially reuse the 'def' keyword for this. def a: List[float]. This would be a surprising new syntax for many who are used to searching code for r'^\s*def' to find function definitions. Precedent: Cython already overloads its own 'cdef' concept for both variable and function/method use. Potential alternative to the above def (ab)use: def a -> List[float] def a List[float] def List[float] a # copies the Cython ordering which seems to derive from C syntax for obvious reasons But the -> token really implies return value while the : token already implies variable type annotation. At first glance I'm not happy with these but arguments could be made. Second, when these occur in a class body, they can define either class
Disallowing ": type" syntax in the presence of tuple assignment seems simple and wise to me. Easy to parse. But I understand if people disagree and want a defined way to do it. but this is a slight step back from
When thinking about how to spell this out in a PEP, it is worth taking into account existing ways of declaring types on variables in Python. Cython took the "Keyword Type Name" approach with "cdef double j" syntax. http://cython.readthedocs.io/en/latest/src/quickstart/cythonize.html Is it an error to write the following (poor style) code declaring a type for the same variable multiple times: c: int = module.count_things(x) compute_thing(c) if c > 3: c: str = module.get_thing(3) logging.info('end of thing 3: %s', c[-5:]) do_something(c) where c takes on multiple types within a single scope? static single assignment form would generate a c', c'', and union of c' and c'' types for the final do_something call to reason about that code. but it is entirely doable in Python and does happen in unfortunately real world messy code as variables are reused in bad ways. My preference would be to make it an error for more than one type to be declared for the same variable. First type ever mentioned within the scope wins and all others are SyntaxError worthy. Assigning to a variable in a scope before an assignment that declares its type should probably also be a SyntaxError. -gps
What makes you respond so vehemently to `a: float`? The `def` keyword has been proposed before, but *I* have a vehement response to it; `def` is for functions. The Cython syntax may live forever in Cython, but I don't want to add it to Python, especially since we already have function annotations using `var: type = default` -- variable declarations must somehow rhyme with this. The `local` keyword isn't horrible but would still require a `__future__` import and a long wait, which is why I'm exploring just `var: type = value`. I think we can pull it off syntactically, using a trick no more horrible than the one we already employ to restrict the LHS of an assignment even though the grammar seen by the parser is something like `expr_stmt: testlist ('=' testlist)*` (at least it was something like this long ago -- it's more complex now but the same idea still applies, since the official parser is still LR(1)). Regarding scopes, I like the way mypy currently does this -- you can only have a `# type` comment on the first assignment of a variable, and scopes are flat as they are in Python. (Mypy is really anticipating a syntax for variable declarations here.) Seems we agree on this, at least. On Mon, Aug 1, 2016 at 4:39 PM, Gregory P. Smith <greg@krypto.org> wrote:
-- --Guido van Rossum (python.org/~guido)
On 8/2/2016 1:14 AM, Guido van Rossum wrote:
What makes you respond so vehemently to `a: float`?
I am not Gregory, but I also had a negative reaction. It makes no sense to me. By your rule that annotations are optional and ignorable, 'a: float' is a statement expression consisting of the identifier 'a'. If one ignores or removes the annotations of a function header, one is left with a valid function header with the same meaning. The only runtime effect is on the .annotations attribute, and any consequence of that. If one were to do the same with a (proposed) annotated assignment, would be left with a valid assignment, and there should be no runtime effect either way. If one removes ': float' from 'a: float', one is left with 'a', a single-name expression statement. To be consistent, the addition or removed of the annotation should have no runtime effect here either. The meaning is the statement is 'if a is not bound to anything in the namespace stack, raise NameError'. In batch mode, the else part is ignore it, ie, 'pass'. I have never seen a name expression statement used this way in non-interactive code. It would be an obscure way to control program flow. Interactive mode adds 'else print(a)', which is useful, hence '>>> a' is common. This is not relevant to the offline use of annotations. If 'a' is bound to something, then the annotation belongs on the binding statement. If it is not, then the annotation is irrelevant unless added to the NameError message. If, as I suspect. upi meant 'a: float' to be a different kind of statement, such as a static type declaration for the name 'a', it would be a major change to Python, unlike adding type hints to existing statements. It would make the annotation required, not optional. It would complicate an annotation stripper, as 'a: float' would have to be handled differently from 'a: float = 1.0'. The existing scope declarations are a bit jarring also, but they have the runtime effect of enabling non-local name binding. The best alternative to 'global n' I can think of would have been a tagged '=', such as '=*' meaning 'bind in the global namespace. But this would have had problems both initially and when adding augmented assignments and non-local binding. -- Terry Jan Reedy
On Tue, Aug 2, 2016 at 3:55 PM, Terry Reedy <tjreedy@udel.edu> wrote:
You're taking the "optional and ignorable" too literally. My strawman proposal for the semantics of a: float is that the interpreter should evaluate the expression `float` and then move on to the next line. That's the same as what happens with annotations in signatures. (However this is a potential slow-down and maybe we need to skip it.) The idea of this syntax is that you can point to function definitions and hand-wave a bit and say "just like an argument has an optional type and an optional default, a variable can have either a type or an initializer or both" (and then in very small print continue to explain that if you leave out both the two situations differ, and ditto for multiple assignment and unpacking).
None of that is relevant, really. :-)
But there are no annotation strippers, only parsers that understand the various annotation syntaxes and ignore the annotations.
This is entirely different from global/nonlocal -- the latter are references to previously declared/initialized variables. Here we are declaring new local/class/instance variables. Pretty much the opposite. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 2, 2016 at 6:07 PM, Guido van Rossum <guido@python.org> wrote:
The criticism I would make about allowing variables without assignments like a: float is that it makes my mental model of a variable a little bit more complicated than it is currently. If I see "a" again a few lines below, it can either be pointing to some object or be un-initialized. Maybe the benefits are worth it, but I don't really see it, and I wanted to point out this "cost". Semi-related: what should happen if I do "del a" and then re-use the variable name a few lines below? My feeling is that the type annotation should also get discarded when the variable is deleted.
On 3 August 2016 at 10:48, Alvaro Caceres via Python-ideas <python-ideas@python.org> wrote:
This concern rings true for me as well - "I'm going to be defining a variable named 'a' later and it will be a float" isn't a concept Python has had before. I *have* that concept in my mental model of C/C++, but trying to activate for Python has my brain going "Wut? No.". I'd be much happier if we made initialisation mandatory, so the above would need to be written as either: a: float = 0.0 # Or other suitable default value or: a: Optional[float] = None The nebulous concept (and runtime loophole) where you can see: class Example: a: float ... but still have Example().a throw AttributeError would also be gone. (Presumably this approach would also simplify typechecking inside __new__ and __init__ implementations, as the attribute will reliably be defined the moment the instance is created, even if it hasn't been set to an appropriate value yet) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, Aug 3, 2016 at 8:24 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Have you annotated a large code base yet? This half of the proposal comes from over six months of experience annotating large amounts of code (Dropbox code and mypy itself). We commonly see situations where a variable is assigned on each branch of an if/elif/etc. structure. If you need to annotate that variable, mypy currently requires that you put the annotation on the first assignment to the variable, which is in the first branch. It would be much cleaner if you could declare the variable before the first `if`. But picking a good initializer is tricky, especially if you have a type that does not include None. As an illustration, I found this code: https://github.com/python/mypy/blob/master/mypy/checkexpr.py#L1152 if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() # type: Type elif op == '-': method_type = self.analyze_external_member_access('__neg__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type elif op == '+': method_type = self.analyze_external_member_access('__pos__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type else: assert op == '~', "unhandled unary operator" method_type = self.analyze_external_member_access('__invert__', operand_type, e) result, method_type = self.check_call(method_type, [], [], e) e.method_type = method_type return result Look at the annotation of `result` in the if-block. We need an annotation because the first functions used to assign it returns a subclasses of `Type`, and the type inference engine will assume the variable's type is that of the first assignment. Be that as it may, given that we need the annotation, I think the code would be clearer if we could set the type *before* the `if` block. But we really don't want to set a value, and in particular we don't want to set it to None, since (assuming strict None-checking) None is not a valid value for this type -- we don't want the type to be `Optional[Type]`. IOW I want to be able to write this code as result: Type if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() elif op == '-': # etc.
That's an entirely different issue though -- PEP 484 doesn't concern itself with whether variables are always initialized (though it's often easy for a type checker to check that). If we wrote that using `__init__` we could still have such a bug: class Example: def __init__(self, n: int) -> None: for i in range(n): self.a = 0.0 # type: float But the syntax used for declaring types is not implicated in this bug.
But, again, real problems arise when the type of an *initialized* instance must always be some data structure (and not None), but you can't come up with a reasonable default initializer that has the proper type. Regarding the question of whether it's better to declare the types of instance variables in `__init__` (or `__new__`) or at the class level: for historical reasons, mypy uses both idioms in different places, and when exploring the code I've found it much more helpful to see the types declared in the class rather than in `__init__`. Compare for yourself: https://github.com/python/mypy/blob/master/mypy/build.py#L84 (puts the types in `__init__`) https://github.com/python/mypy/blob/master/mypy/build.py#L976 (puts the types in the class) -- --Guido van Rossum (python.org/~guido)
On Wed, Aug 03, 2016 at 09:11:42AM -0700, Guido van Rossum wrote:
Just playing Devil's Advocate here, could you use a type hint *comment*? #type result: Type if op == 'not': self.check_usable_type(operand_type, e) result = self.chk.bool_type() elif op == '-': # etc. Advantages: - absolutely no runtime cost, not even to evaluate and discard the name following the colon; - doesn't (wrongly) imply that the name "result" exists yet; Disadvantages: - can't build a runtime __annotations__ dict; [...]
Again, playing Devil's Advocate... maybe we want a concept of "undefined" like in Javascipt. result: Type = undef which would make it clear that this is a type declaration, and that result is still undefined. Disadvantages: - Javascript programmers will think you can write `print(result)` and get "undef" (or similar); - requires a new keyword. -- Steve
On Wed, Aug 3, 2016 at 10:35 AM, Steven D'Aprano <steve@pearwood.info> wrote:
I like this -- and we need to change the interpreter anyway. I take it this would be a no-op at run time? Though I'm still on the fence -- it's a common idiom to use None to mean undefined. For example, I at least, always thought is was better style to do: class Something(): an_attribute = None and then: if self.an_attribute is None: .... Than not predefine it, and do: if not hasattr(self, 'an_attribute'): .... granted, I only do that for class (or instance) attributes in real code, not all names. So do we really need to support "this variable can be undefined, but if it is defined in can NOT be None? Are there really cases where a variable can't be None, but there is NO reasonable default value? Or are folks hitting a limitation in the type checker's ability to deal with None?
Do we care about that???? BTW, does JS have a None? or is 'undef' its None? -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Chris, From: Python-ideas <python-ideas-bounces+kevin-lists=theolliviers.com@python.org> on behalf of Chris Barker <chris.barker@noaa.gov> Date: Wednesday, August 3, 2016 at 11:33 AM To: Steven D'Aprano <steve@pearwood.info> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484 On Wed, Aug 3, 2016 at 10:35 AM, Steven D'Aprano <steve@pearwood.info> wrote: Again, playing Devil's Advocate... maybe we want a concept of "undefined" like in Javascipt. result: Type = undef which would make it clear that this is a type declaration, and that result is still undefined. I like this -- and we need to change the interpreter anyway. I take it this would be a no-op at run time? Though I'm still on the fence -- it's a common idiom to use None to mean undefined. For example, I at least, always thought is was better style to do: class Something(): an_attribute = None and then: if self.an_attribute is None: .... Than not predefine it, and do: if not hasattr(self, 'an_attribute'): .... granted, I only do that for class (or instance) attributes in real code, not all names. This is SOP for me, too. I think the hassle of dealing with uninitialized variables (and the bugs that result when not being diligent about them) naturally discourages their use. So do we really need to support "this variable can be undefined, but if it is defined in can NOT be None? Are there really cases where a variable can't be None, but there is NO reasonable default value? Or are folks hitting a limitation in the type checker's ability to deal with None? Disadvantages: - Javascript programmers will think you can write `print(result)` and get "undef" (or similar); Do we care about that???? BTW, does JS have a None? or is 'undef' its None? JavaScript has the null keyword in addition to undefined. In my experience with JavaScript, in practice having both a null keyword and an undefined keyword almost always ends up being a source of bugs. Consider the non-intuitive results of the following: var value = undefined; If (value == null) // returns true if value is undefined thanks to type coercion, since null is coerced to false var value = null; If (value == undefined) // returns true if value is null, again thanks to type coercion This is one reason why JS needs the === / !== operators in order to short-curcuit type coercion, and the need to do this is a common source of mistakes among JS developers. I do think Python is better equipped to make this less confusing thanks to the is keyword, etc., but no matter how you look at it, undef, if created, will be something distinct from None and yet sometimes devs will see the two as equivalent. This opens up questions like "do we want a way for None and undef to be equivalent sometimes, but in other cases consider them distinct cases?" I'd say keep it simple. Have developers use None for uninitialized variables if we require assignment, or simply make: a: int implicitly set the value to None if we allow no explicit assignment. Regards, Kevin -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Aug 03, 2016 at 11:33:34AM -0700, Chris Barker wrote:
That was my intention. result: Type = undef would be a no-op at runtime, but the type-checker could take the type-hint from it. There's no actual "undef" value. If you followed that line by: print(result) you would get a NameError, same as today. To answer Chris A's objection in a later post, this is *not* an alternative to the existing `del` keyword. You can't use it to delete an existing name: result = 23 result = undef # SyntaxError, no type given result: Type = undef # confuse the type-checker AND a runtime no-op
Though I'm still on the fence -- it's a common idiom to use None to mean undefined.
Yes, but None is an actual value. "undef" would not be, it would just be syntax to avoid giving an actual value. I don't actually want to introduce a special "Undefined" value like in Javascript, I just want something to say to the program "this is just a type declaration, no value has been defined as yet".
Yes. See Guido's responses. The use-case is: (1) you have a variable that needs a type-hint for a specific type, excluding None; (2) there's no default value you can give it; (3) and it's set in different places (such as in different branches of an if...elif... block). Currently, you have to write: if condition: spam: Widget = Widget(arg) # lots more code here elif other_condition: spam: Widget = load(storage) # lots more code here elif another_condition spam: Widget = eggs.lookup('cheese')['aardvark'] # lots more code here else: spam: Widget = create_widget(x, y, z) # lots more code here which repeats the Widget declaration multiple times. For readability and maintenance, Guido wants to pull the declaration out and put it at the top like this: spam = ???? #type spam:Widget if condition: spam = Widget(arg) # lots more code here # etc. but there's nothing he can put in place of the ???? placeholder. He can't use a specific Widget, because he doesn't know which Widget will be used. He can't use None, because None is not a valid value for spam. Neither of these options are right: spam: Widget = None # fails the type check spam: Optional[Widget] = None # passes the type check here # but allows spam to be None later, which is wrong.
Or are folks hitting a limitation in the type checker's ability to deal with None?
No. The limitation is that there's no way to declare a type-hint without declaring a value at the same time. Hence some suggestions: spam: Widget spam: Widget = undef #type spam: Widget
I shouldn't have specifically said "Javascript programmers". I think it applies to lots of people. And besides, there's probably already people who have a variable called "undef" and turning it into a keyword will break their code.
BTW, does JS have a None? or is 'undef' its None?
Javascript has a null, as well as an undefined value. Thanks to the magic of Javascript's type-coercion rules, they compare equal: [steve@ando ~]$ rhino Rhino 1.7 release 0.7.r2.3.el5_6 2011 05 04 js> a = null; null js> b = undefined; js> print(a, b) null undefined js> a == b; true js> a === b; false In case my intent is still not clear, I DON'T want to introduce an actual "undefined" value, as in Javascript. I only made this suggestion in the hope that it might spark a better idea in others. -- Steve
On Wed, Aug 3, 2016 at 5:52 PM, Steven D'Aprano <steve@pearwood.info> wrote: [...]
[...] Thanks for explaining my use case so well (in the text I snipped). However, I still feel that just result: Type is the better alternative. Syntactically there's no problem with it. It doesn't require a new `undef` keyword or constant, and it doesn't make people think there's an `undef` value. As you mentioned, this has been an unmitigated disaster in JS. -- --Guido van Rossum (python.org/~guido)
Concerning type annotations on local variables of a function: 1) Would they be stored anywhere? If so, where? 2) Would they be evaluated once, or every time the function is called? 3) If they're evaluated once, when and in what scope? -- Greg
On Thu, Aug 04, 2016 at 06:38:00PM +1200, Greg Ewing wrote:
I would think that annotations on local variables should not be evaluated or stored. I think it is reasonable to have annotations on global variables (or inside a class definition) stored somewhere for runtime analysis, like function annotations. -- Steve
On 04.08.2016 15:32, Steven D'Aprano wrote:
I don't know. That somehow would break expected Python semantics and would lead to big asymmetry. Most people don't think about the difference between module-level variables and variables in functions. Their code works regardlessly. The more I think about it, the more I like #comment-style annotations then. They are a clear sign to the developer >not at runtime<, and only available for static analysis. This might even be true for class/instance variables. -- Sven
However the presence of a local declaration like 'a: int' would create a local slot for 'a' as if it were assigned to somewhere in the function. --Guido (mobile)
On Thu, Aug 4, 2016 at 11:22 AM, Guido van Rossum <gvanrossum@gmail.com> wrote:
However the presence of a local declaration like 'a: int' would create a local slot for 'a' as if it were assigned to somewhere in the function.
Does this mean that the following code will raise a NameError? a = None def f(): a: int a (FWIW, I like the <var>: <type> notation more than any alternative proposed so far.)
On Thu, Aug 4, 2016 at 8:40 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
It will do exactly the same as a = None def f(): if False: a = ... a This raises UnboundLocalError (a subclass of NameError).
(FWIW, I like the <var>: <type> notation more than any alternative proposed so far.)
Me too. :-) -- --Guido van Rossum (python.org/~guido)
On Thu, Aug 4, 2016 at 12:15 PM, Guido van Rossum <gvanrossum@gmail.com> wrote:
That's what I expected. Moreover, it looks like there is a precedent for such behavior: $ python3 -O
(Note that the if statement is optimized away.) BTW, a: int looks like an "incomplete" assignment to me. It's a statement that under-specifies a: gives it a type, but not a value. Visually, ':' is a shorter version of '=' and a half of the venerable Pascal's ':='. It all makes sense to me, but your milage may vary if your first language was JavaScript rather than ALGOL. :-)
On Thu, Aug 4, 2016 at 9:44 AM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Actually, I'm trying to keep some memories of Pascal. In Pascal you write: var age: integer; The family of languages derived from C is already doing enough to keep the ALGOL tradition alive. Anyways, "age: int" matches what you write in function signatures in Python, so I think our hands are tied here and we might as well make the most of it. -- --Guido van Rossum (python.org/~guido)
On 4 August 2016 at 02:11, Guido van Rossum <guido@python.org> wrote:
Ah, that makes sense - given that motivation, I agree it's worth introducing the concept. You'll probably want to make sure to give that example from the mypy code (or a simpler version), as I expect that pre-declaration aspect will be the most controversial part of the whole proposal. I wonder if there's some way we could make that new statement form trigger the following runtime behaviour: a : int a # This raises a new UnboundNameError at module scope, UnboundLocalError otherwise Otherwise we're at risk of allowing thoroughly confusing runtime behaviour like: >>> a = "Not an int" >>> def f(): ... # a: int would go here ... print(a) # This should really fail ... >>> f() Not an int The possibility that springs to mind is a new dedicated opcode, DECLARE_NAME, that works like an assignment that appears anywhere in the function for function namespaces, and does something new for module and class namespaces where it's like an assignment, but doesn't appear in locals(). Depending on how the latter work, we may even be able to raise a new UnboundAttributeError subclass for attempts to access declared-but-not-defined attributes. We'd also want the new syntax to conflict with both global and nonlocal, the same way they currently conflict with each other:
Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Aug 04, 2016 at 10:37:33PM +1000, Nick Coghlan wrote:
Why wouldn't it raise NameError, as it does now? I understand that `a: int` with no assignment will be treated as a no-op by the interpreter. a: int = 0 will be fine, of course, since that's assigning 0 to a.
I would expect that the type-checker will complain that you're declaring a local variable "a" but there's no local "a" in the function. If the checker isn't that smart, I expect it would complain that "a" is set to a string, but declared as an int. Either way, the type-checker ought to complain. -- Steve
On 4 August 2016 at 23:29, Steven D'Aprano <steve@pearwood.info> wrote:
Guido's reply clarified that he expects the compiler to be declaration aware (so it correctly adds "a" to the local symbol table when a type declaration is given), which means functions will be free of ambiguous behaviour - they'll throw UnboundLocalError if a name is declared and then referenced without first being defined. That means it's mainly in classes that oddities will still be possible:
That's probably acceptable to handle as "Don't do that" though, and leave it to typecheckers to pick it up as problematic "Referenced before definition" behaviour. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Aug 1, 2016 at 10:14 PM Guido van Rossum <guido@python.org> wrote:
What makes you respond so vehemently to `a: float`?
First reaction. It doesn't actually seem too bad now. It is already legal to just say `a` as a statement so this isn't much different. def f(): a is already legal. So `a: List[float]` would at least be meaningful rather than a meaningless statement that any linter should question. :)
I agree, cdef is their own mess. :)
-gps
On Mon, Aug 01, 2016 at 02:31:16PM -0700, Guido van Rossum wrote:
Those examples look reasonable to me. [...]
I would think so. Consider the case that you have Class.spam and Class().spam which may not be the same type. E.g. the class attribute (representing the default value used by all instances) might be a mandatory int, while the instance attribute might be Optional[int].
Require parens around the name:hint. (a:int), (b:str) = 0, '' Or just stick to a type hinting comment :-) What about this case? spam, eggs = [1, 2.0, 'foo'], (1, '') [a, b, c], [d, e] = spam, eggs That becomes: [(a:int), (b:float), (c:str)], [(x:int), (y:str)] = spam, eggs which is bearable, but just unpleasant enough to discourage people from doing it unless they really need to. -- Steve
Regarding class variables: in my experience instance variables way outnumber class variables, except when the latter are used as defaults for instance variables. See e.g. this bit of code from mypy (and many others, actually): https://github.com/python/mypy/blob/master/mypy/nodes.py#L154. All of these declare instance variables. Many of them have no need for a default in the class, but must give one anyway or else there's nothing to put the type comment on -- you can't write defs # type: List[Statement] (it would be a NameError) and you certainly don't want to write defs = [] # type: List[Statement] (else the gods of shared mutable state will curse you) so you have to make do with defs = None # type: List[Statement] But this would be totally reasonable as defs: List[Statement] under my proposal. On Mon, Aug 1, 2016 at 7:55 PM, Steven D'Aprano <steve@pearwood.info> wrote:
-- --Guido van Rossum (python.org/~guido)
On Mon, Aug 1, 2016, at 02:31 PM, Guido van Rossum wrote:
To me, option A ("a, b: int, str") and option B ("a: int, b: str") are both... overly dense. If I had to choose between one, I'd choose B. Strongly. A is flatly wrong-- 'b' looks like it binds to 'int', but in fact, it binds to 'str'. This contradicts function annotations. But I still don't really like B. To me, I'd just disallow variable annotations in unpacking. Unpacking is a wonderful feature, I love unpacking, but to combine it with annotations is a cognitive overload. Maybe that means you don't use unpacking sometimes: that's okay. Unpacking is about being concise, to me at least, and if I'm doing annotations, that means I'm accepting being more verbose for the sake of static checking. You can't be all things to all people. In this case, I say, someone has to pick which their priority is: conciseness or static checking. -- Stephen Hansen m e @ i x o k a i . i o
On 01.08.2016 23:31, Guido van Rossum wrote:
I can't help but this seems like a short "key: value" dict declaration. Besides the fact that those examples don't really highlight the real use-cases. But others already covered that.
Then writing "a" should be allow, too, right? As a variable declaration without any type hint. That's usually a NameError. Put it differently, what is the use-case Python has missed so long in not providing a way of declaring an empty variable? Will it be filled with None?
Could be useful but might result in a lot of double maintenance work (class body + place of initilization).
Or using "a: float" from above: a: float b: str a, b = 0, '' So, ignoring commas for now and solving it later would also work. Sven
On Wed, Aug 3, 2016, 6:49 AM Sven R. Kunze <srkunze@mail.de> wrote:
Nope, it wouldn't do anything at all. No bytecode. No namespace updated. No None. It is merely an informational statement that an optional type checker pass may make use of. This definitely counts as a difference between a bare 'a' and 'a: SPAM'. The former is a name lookup while the latter is a no-op. Good or bad I'm undecided. At least the latter is useful while the former is more of a side effect of how Python works.
I'm behind Gregory here, as I'm not a big fan of the "a: float" syntax (but I don't have a strong opinion either way). I'm strongly against merely "a" being equal to "a = None", though. In my code, that's either a typo or I've wrapped that in a try: ... except NameError: ..., and I'd like Python to throw a NameError in that case. So either we deal with an asymmetry in how annotations work, or we disallow it. I'm likely not going to use that particular syntax even if it exists, so I'm -0. -Emanuel
From Gregory P. Smith <greg@krypto.org> Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484
On Wed, Aug 3, 2016, 6:49 AM Sven R. Kunze <srkunze@mail.de<mailto:srkunze@mail.de>> wrote: On 01.08.2016 23:31, Guido van Rossum wrote:
Then writing "a" should be allow, too, right? As a variable declaration without any type hint. That's usually a NameError. Put it differently, what is the use-case Python has missed so long in not providing a way of declaring an empty variable? Will it be filled with None? Nope, it wouldn't do anything at all. No bytecode. No namespace updated. No None. It is merely an informational statement that an optional type checker pass may make use of. This definitely counts as a difference between a bare 'a' and 'a: SPAM'. The former is a name lookup while the latter is a no-op. Good or bad I'm undecided. At least the latter is useful while the former is more of a side effect of how Python works.
If I understand this proposal then we probably need to consider this too: if something: a: float else: a: str a = 'what would static type-checker do here?' del a a: int = 0 def fnc(): global a: list a = 7 fnc() a = [1, 2, 3] # and this could be interesting for static type-checker too
On Wed, Aug 3, 2016 at 3:02 PM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
The beauty of it is that that's entirely up to the static checker. In mypy this would probably be an error. But at runtime we can make this totally well-defined.
del a a: int = 0
Ditto.
def fnc(): global a: list
I'm not proposing to add such syntax, and the right place for the type of a would be at the global level, not on the `global` syatement.
Indeed, but that's not what we're debating here. -- --Guido van Rossum (python.org/~guido)
On 8/4/16, Guido van Rossum <guido@python.org> wrote:
On Wed, Aug 3, 2016 at 3:02 PM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
Sorry but for me is really important where we are going (at least as inspiration). As I understand now this changes could end in code which could be pure python and on other hand also compilable by static typed compiler too. Maybe it is what we wanted (I prefer this one), maybe we doesnt care and maybe we want to avoid it. For example Cython's code: cdef int n could be writen: n: cdef.int or n: 'cdef int' and I think it could be good to see convergence here. And maybe this cdef int n, k, i could be inspiring too allow one to many possibility n, j, k: int # (1) I think it is natural and better than n, j, k: int, int, int But (1) would be discordant with this (from PEP484) def __init__(self, left: Node, right: Node)
On Thu, Aug 4, 2016 at 12:16 AM, Pavol Lisy <pavol.lisy@gmail.com> wrote:
But you are thinking with your runtime hat on. To a static type checker, the call to fnc() is irrelevant to the type of the global variable `a`. It's like writing def foo(): return int def bar(a: foo()): return a+1 The static type checker rejects `foo()` as an invalid type, even though you know what it means at runtime. (But what if foo() were to flip a coin to choose between int and str?)
Yes, that should be possible. Just don't run mypy over that source code. :-)
And that's why I really don't want to go there. What if someone wrote T = Tuple[int, int, str] a, b, c: T Do we now have three Tuple[int, int, str] variables, or two ints and a str? -- --Guido van Rossum (python.org/~guido)
I'll post here another wild idea (based on the https://github.com/python/ mypy/blob/master/mypy/build.py#L976 example), just thinking on instance attributes. I myself am not fully sold on it but it's quite different so perhaps adds a new angle to the brainstorming and has some nice upsides: class State: def __instancevars__( manager: BuildManager, order: int, # Order in which modules were encountered id: str, # Fully qualified module name path: Optional[str], # Path to module source xpath: str, # Path or '<string>' source: Optional[str], # Module source code meta: Optional[CacheMeta], data: Optional[str], tree: Optional[MypyFile], dependencies: List[str], suppressed: List[str], # Suppressed/missing dependencies priorities: Dict[str, int] ): ... The benefits of this is: * No need to add new syntax or keywords here, this works in any 3.x * This part of the change can be done right now without waiting for python 3.6 * it's guaranteed to be consistent with function annotations * The instance vars annotations end up in a standard place on runtime if someone needs them (run time checking, documentation generators, etc). * No confusing access to a name that may look like a NameError * Very obvious runtime behaviour for python users even if they don't use a typechecker. * This can even be used in python 2.x, by changing the annotation to a "# type: T" along the argument. The downsides are: * There is a bit of boilerplate (the parenthesis, the ellipsis, the extra commas) * It's unclear what the body of this function is supposed to do, but probably the typechecker can ensure it's always "..." (I'm assuming that whoever writes this wants to use a type checker anyway) * It's also unclear what a default argument would mean here, or *args, or **kwargs (again, the typechecker could enforce proper usage here) * It doesn't cover locals and class variables, so it's still necessary the new syntax for "type declaration and initialization assignment" (but not the "declaration without assignment") As a second order iterations on the idea * If this idea works well now it's easier to add nicer syntax later that maps to this (like a block statement) and specify the runtime semantics in term of this (which is what I see the traditional way in python) * It could be possible to give semantic to the body (like a post __new__ thing where locals() from this function are copied to the instance dict), and this would allow setting real instance defaults () but this is new class creation semantics and can have a performance cost on instantiation when having types declared Hope this adds to the discussion, On Thu, Aug 4, 2016 at 9:46 AM, Sven R. Kunze <srkunze@mail.de> wrote:
-- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Thu, Aug 4, 2016 at 4:03 AM, Daniel Moisset <dmoisset@machinalis.com> wrote:
That feels too hacky for a long-term solution, and some of your downsides are very real. The backwards compatible solution is to keep using type comments (with initial values set to None, or `...` if Python 2 isn't needed). But thanks for thinking creatively about the problem! -- --Guido van Rossum (python.org/~guido)
On 2016-08-01 14:31, Guido van Rossum wrote:
Let me ask a perhaps silly question. Reading a lot of subsequent messages in this thread, it seems that the main intended use for all this is external static type checkers. (Hence all the references to thinking with "runtime hats" on.) Given this, what is the benefit of making this change to Python syntax? If it's only going to be used by static checker tools, is there any difference between having those tools grab it from comments vs. from "real" syntax? This seems doubly questionable if, for local variables, the type annotations are totally unavailable at runtime (which I think is what was suggested). It seems odd to have Python syntax that not only doesn't do anything, but can't even be made to do anything by the program itself when it runs. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, Aug 4, 2016 at 2:33 PM, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
It seems odd to have Python syntax that not only doesn't do anything, but can't even be made to do anything by the program itself when it runs.
Why do you find this odd? Python already has #-comment syntax with exactly the property you complain about. Think of local variable annotation as a structured comment syntax.
On 2016-08-04 12:09, Alexander Belopolsky wrote:
Exactly. But the existing comment annotations (# type: float) already are structured comment syntax, so what does this new one add? (Note we already have at least one parallel --- although it is much more limited in scope --- namely encoding declarations, which are also done within existing comment syntax, rather than having their own "real" syntax.) -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, 4 Aug 2016 at 12:20 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
[I'm going to give an explanation one shot, then I will leave the topic alone as explaining the "why" definitely has the possibility of dragging on] You're right this could be viewed as syntactic sugar, but then again so is multiplication for integers as a short-hand for addition or the fact we even have infix operators which translate to method calls on objects. In all cases, the syntax is meant to either make people's lives easier to to promote their use. In this instance it both makes people's lives easier and promotes use. By making it syntax it becomes easier to use as the compiler can now detect when the information is improperly typed. And by making it syntax, not only does it make sure there's a strong standard but lets people know that this isn't some little feature but in fact is being promoted by the Python language itself. There's also the disconnect of having type annotations on function parameters but completely missing the OOP aspect of attributes on objects which leaves a gaping hole in terms of syntactic support for type hints. Hopefully that all makes sense as to why Guido has brought this up.
participants (25)
-
Alexander Belopolsky -
Alvaro Caceres -
Brendan Barnwell -
Brett Cannon -
Chris Angelico -
Chris Barker -
Daniel Moisset -
Emanuel Barry -
Eric Snow -
Ethan Furman -
Greg Ewing -
Gregory P. Smith -
Guido van Rossum -
Guido van Rossum -
Joao S. O. Bueno -
Kevin Ollivier -
Nick Coghlan -
Paul Moore -
Pavol Lisy -
Stephan Houben -
Stephen Hansen -
Steven D'Aprano -
Sven R. Kunze -
Terry Reedy -
tritium-list＠sdamon.com