[Python-Dev] Please reject or postpone PEP 526

[Steven D'Aprano]

Hi Mark,

I'm going to trim your post drastically, down to the bare essentials, in 
order to keep this already long post down to a manageable size.


On Fri, Sep 02, 2016 at 02:47:00PM +0100, Mark Shannon wrote:

[...]
> With type comments, this is intuitively correct and should type check:
> def eggs(cond:bool):
>     if cond:
>         x = None
>     else:
>         x = [] # type: List[int]
>     spam(x)  # Here we can infer the type of x

It isn't correct. You've declared something to be a list of ints, but 
assigned a value None to it! How is that not an error?

The error is more obvious if you swap the order of assignments:

    if cond:
        x = [] # type: List[int]
    else:
        x = None


MyPy currently requires the experimental --strict-optional flag to 
detect this error:

[steve at ando ~]$ mypy --strict-optional test.py
test.py: note: In function "eggs":
test.py:10: error: Incompatible types in assignment (expression has type 
None, variable has type List[int])


Changing that from comment syntax to (proposed) Python syntax will not 
change that. There is no semantic difference to the type checker 
between 

    x = []  #type: List[int]

and

    x: List[int] = []

and any type checker will have to treat them identically.


> With PEP 526 we loose the ability to infer types.

On re-reading the PEP, I have just noticed that nowhere does it 
explicitly state that checkers are still expected to perform type 
inference. However, this is the companion to PEP 484, which states:

    Type comments

    No first-class syntax support for explicitly marking variables
    as being of a specific type is added by this PEP. TO HELP WITH 
    TYPE INFERENCE IN COMPLEX CASES, a comment of the following
    format may be used: [...]

(Emphasis added.) So the intention is that, regardless of whether you 
use a type annotation using a comment or the proposed syntax, that is 
intended to *help* the checker perform inference, not to replace it.

Perhaps this PEP should include an explicit note to that end.



[...]
> So we need to use a more complex type
> def eggs(cond:bool):
>     x: Optional[List[int]]
>     if cond:
>         x = None # Now legal
>     else:
>         x: = []
>     spam(x)
> 
> I don't think this improves readability.

Maybe not, but it sure improves *correctness*.

A human reader might be able to logically deduce that x = None and 
x = [] are both valid, given that spam() takes either a list or None, 
but I'm not sure if that level of intelligence is currently possible in 
type inference. (Maybe it is, and MyPy simply doesn't support it yet.) 
So it may be that this is a case where you do have to apply an explicit 
type hint, using *either* a type comment or this new proposed syntax:

    x: Optional[List[int]]
    if cond:
        x = None
    else:
        x = []

should be precisely the same as:

    if cond:
        x = None #type: Optional[List[int]]
    else:
        x = []


> Quoting from the PEP:
> ```
> a: int
> a: str # Static type checker will warn about this.
> ```
> In other words, it is illegal for a checker to split up the variable, 
> even though it is straightforward to do so.

No, it is a *warning*, not an error.

Remember, the Python interpreter itself won't care. The type checker is 
optional and not part of the interpreter. You can still write code like:

a = 1
do_something(a)
a = "string"
do_another(a)

and Python will be happy. But if you do run a type checker, it should 
warn you that you're changing types, as that suggests the possibility of 
a type error. (And it also goes against a lot of people's style 
guidelines.)


> We should be free to add extra variables, whenever we choose, for 
> clarity. For example,
>     total = foo() - bar()
> should not be treated differently from:
>     revenue = foo()
>     tax = bar()
>     total = revenue - tax
> 
> If types are inferred, there is no problem.
> However, if they must be declared, then the use of meaningfully named 
> variables is discouraged.

Was there something in the PEP that lead you to believe that they "must" 
be declared? Under "Non-goals", the PEP states in bold text:

"the authors have no desire to ever make type hints mandatory"

so I'm not sure why you think that types must be declared.

Perhaps the PEP should make it more obvious that type hints on variables 
are *in addition to* and not a substitute for type inference.


> PEP 484 states:
> "If type hinting proves useful in general, a syntax for typing variables 
> may be provided in a future Python version."
> Has it proved useful in general? I don't think it has.

According to the PEP, it has proved useful in typeshed.


> It seems confused about class attributes and instance attributes
> ================================================================
> 
> The PEP also includes a section of how to define class attributes and 
> instance attributes. It seems that everything needs to be defined in the 
> class scope, even it is not an attribute of the class, but of its 
> instances.

Quoting from the PEP:

     "As a matter of convenience, instance attributes can be 
     annotated in __init__ or other methods, rather than in class"


Perhaps the PEP could do with a little more justification for why 
we would want to declare instance attributes in the class rather than in 
__init__.



> Example from PEP 526:
> 
> class Starship:
> 
>     captain: str = 'Picard'
>     damage: int
>     stats: ClassVar[Dict[str, int]] = {}
> 
>     def __init__(self, damage: int, captain: str = None):
>         self.damage = damage
>         if captain:
>             self.captain = captain  # Else keep the default

On re-reading this, I too wonder why damage is being declared in the 
class body. Can the type checker not infer that self.damage has the same 
type as damage in the __init__ method?



> Finally, in the rejected proposal section, under "Should we introduce 
> variable annotations at all?" it states that "Variable annotations have 
> already been around for almost two years in the form of type comments, 
> sanctioned by PEP 484."
> I don't think that this is entirely true.

PEP 484 itself was created almost two years ago (Sept 2014) and although 
it doesn't list prior art for type comments, I seem to recall that it 
copied the idea from MyPy. I expect that MyPy (and maybe even linters 
like PyLint, PyFlakes, etc.) have been using type comments for "almost 
two years", if not longer.



> PEP 484 was about the syntax for types, declaring parameter and return 
> types, and declaring custom types to be generic.
> PEP 484 does include a description of type comments, but they are always 
> annotations on assignment statements and were primarily intended for use 
> in stub files.

I'm not seeing what distinction you think you are making here. What 
distinction do you see between:

    x: int = func(value)

and

    x = func(value)  #type: int





-- 
Steve