[Python-ideas] PEP 484 (Type Hints) -- first draft round

Guido van Rossum guido at python.org
Mon Jan 19 04:28:35 CET 2015


On Sun, Jan 18, 2015 at 11:39 AM, Chris Barker <chris.barker at noaa.gov>
wrote:

> On Fri, Jan 16, 2015 at 12:08 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>> Yes, the PEP tries to gently push an agenda where annotations are only
>> used for Python type hints.
>>
>
> Type hints, as opposed to arbitrary other uses, or type _hints_ as
> aopposed to run-time of compile time static typing?
>

Um, what did you mean by "run-time of compile time static typing"? I'm
guessing "or" instead of "of". With that assumption, I think the answer is
both.

I want annotations to be (eventually) reserved for a standard type
notation. The PEP focuses on the notation. The (by far largest) use case I
see for the notation is running a separate off-line type checker, which
acts like a linter but with much more understanding of types. (But not
perfect!) A second use case (for the same notation!) is equally off-line,
to provide information an IDE needs so it can give accurate suggestions etc.

Some use cases that people (including myself) have often assumed but which
I now find mostly anti-ideas: efficient code generation; run-time type
checking; exact type checking by the Python interpreter when it compiles
and loads modules.


> Cython, providing both of those, for example.
>
> I don't know what to answer for Cython -- doesn't it have its own syntax
>> for declaring C types?
>>
>
> yes:
>
> cdef int i
>
> or:
>
> def func( int i ).
>
> There is also a decorator-based system, for "pure python
>
> But I think it would be nice it it could use the same syntax as other
> systems, particularly if there is an official one.
>

I think it would be nice too. But what I hear from Stefan is that Cython's
requirements are not a good fit for the proposal. And I believe it makes
more sense to let Cython continue to do its own thing rather than trying to
either expand the proposal so Cython can use it without change, or allow
for alternate type notations so Cython can do its own thing. The latter
however sounds closer to what's possible.


> If Cython uses annotations with different meanings, perhaps for Cython
>> files the Python annotations could be put in stub files.
>>
>
> I think we're good there.
>
> But I think Stephan's issues is that currently cython allows both C and
> Python type names:
>
> cdef int i
> cdef list a_list
>
> but where the names conflict, the C name is assumed (int, float, others?)
>
> Given C's typedef, I'd say we should just have Cython use unique names --
> i.e. c_int -> int, etc. But that all requires that no one decides to write
> a new python type called c_int. Though I'm not sure it's really a problem
> to reserve names for Cython.
>

Everything in the proposal has a namespace (either 'typing' or 'builtins')
and the checker is expected to understand and follow imports properly. So I
don't see a problem here.


> Anyway, I guess it's unclear to me whether the intent is to create a
> system that can be used for a static/run-time type checker like Cython.
>

That is not the intent. (Though static and run-time seem two very different
forms of type-checking. Apparently Cython does both?)


> There have been a fair number of comments alone the lines of "this is only
> for lint-like static analysis". But I'd really love to see this move toward
> full functionailty.
>

Once this proposal is in place you may try to adapt it for that purpose.
But I'm not holding my breath, and I think we're better off with an
imperfect proposal that can make it into 3.5 than with waiting until the
ideal all-encompassing proposal is written.


> In fact, after reading the PEPs, I still don't really see the point of
> static type analysis that does not lead to compile time or run-time type
> checking. Maybe someone could provide the motivation for that, oradd alink
> to the PEP where that is.
>
> My question:
>
> I see two clear advantages to static typing: type safety and performance.
>

But in the case of Python, the performance thing is entirely unproven, and
the focus on it is a hindrance to trying to get any type system at all
introduced.


> As a proponent of Python (and thus dynamic typing), I have argued a lot
> that static type checking does indeed catch a lot of bugs, but they are
> generally shallow bugs -- easy to test for, and easy to understand and fix
> -- thus I don't much need that. However, I can see how it's a "good thing",
> and maybe important for "Enterprise" systems.
>

Nobody writes enough tests though, and type checking catches a different
category of issues (shallow or not) than tests.


> On the other hand, dynamic typing does impact performance in a way that
> effects my code -- so I tun to Cython (and sometimes C, C++, Fortran) to
> address those issues. (maybe JIT compilation will one day meet this need,
> but it hasn't yet).
>
> So if there is type specifying, I want i to solve these problems.
>

How?


> But where I'm lost is the utility of static analysis that is not used to
> either provide run-time type checking OR compile optimized code (either JIT
> or ahead-of-time). It seems that your entire system would have to be fully
> type-hinted to get the type safety. But once you've done that, why not just
> use a static language?
>

This is the problem of the traditional focus on type-safety. The
traditional idea is that if you can't prove that the entire program is
type-safe you should not run any of it (because the entire program is
"broken"), and type-checking just part of the program is useless because
you still don't know whether the program is "correct". (This is also often
combined with the inability to generate code for an incorrect program.)
However, even "correct" programs often contain bugs, because the
correctness of a program depends on more than its type-safety. (Almost
every program has some constraints that cannot be expressed in the
language's type system.)

However, gradual typing, for dynamically-typed languages, takes a different
position. It claims that if you can type-check *some* of your program, the
added type safety for that part helps find bugs. It's not a black or white
situation. Just like a linter will find some bugs (after you wade through
the false positives, tuning the linter options locally and globally to get
a decent yield), a type checker of the kind proposed here will find some
bugs (after you spend some time adding type annotations to part of the
program).

This approach works much better for dynamic languages than for static
languages, because a dynamic language can still run a program that doesn't
type-check correctly. (And for a variety of reasons, the program may still
work well enough.)

On the other hand, if run-time type checking is provided, then you could
> use that at the boundaries of your code base (kind of like Cython does) and
> get both the safety and performance benifits.
>

But it sounds like it is Cython's job to do the checking at these
boundaries. Isn't this similar to the way Python extensions written in C or
C++ have to do type checking at the boundary between Python code and the
extension? The specification of these checks should be defined by Cython.


> I've read the MyPY FAQ, which does address these issues to some extent,
> but doesn't really make a compelling case to me, anyway.
>

It did to me. :-)


>
> I guess the gist here is that I think if Python is going to have an
> official type hinting system, it should leave the door open to compile time
> and run-time type checking. A good test case might be: can it be used for
> Cython? But this thread seems to be indicating that that is not the goal,
> and indeed, that it's almost an anti-goal. I hope I'm wrong about that.
>

TBH I think the question of whether it can be used for Cython is still very
much open. Why couldn't Cython take code that says a certain parameter has
type List[int] and generate code from it? (Though no matter how you express
this, I don't see how this could help much, since the input, being
generated by regular Python code, would have to be a builtins.list object
full of builtins.int objects. At that point there isn't much that Cython
can do to speed things up.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150118/ff3136d8/attachment-0001.html>


More information about the Python-ideas mailing list