On Sun, Jan 18, 2015 at 12:36 PM, Stefan Behnel <stefan_ml@behnel.de> wrote:
Guido van Rossum schrieb am 18.01.2015 um 05:11:
> On Sat, Jan 17, 2015 at 1:43 AM, Stefan Behnel wrote:
>> Guido van Rossum schrieb am 16.01.2015 um 21:08:
>>> I don't know what to answer for Cython -- doesn't it have its own syntax
>>> for declaring C types? Or is it now also using annotations?
>>
>> It can, but it's optional. When I added support for it, I took care not to
>> enable it by default, so that it wouldn't accidentally interfere with other
>> usages of annotations.
>>
>> The current proposal seems less generous in this regard, and the way I see
>> it, the only gain from Cython's point of view is its very limited support
>> for container item type declarations and function signatures of callable
>> objects. Both are restricted to Python object types and even imply slightly
>> different semantics than Cython (I didn't see a notion of "exact types" as
>> opposed to "type or subtype").
>
> Actually, while the PEP is still lacking in clarity and we're still having
> some discussions, type variables will most likely be "invariant" by
> default, which I think is the "exact types" notion you are after. There's
> lots of discussion in https://github.com/ambv/typehinting/issues/2 (you
> might want to read this from the end :-).

No, AFAICT, it's actually not what I meant. The discussion above seems to
refer only to the case where a container like "List[X]" may or may not
accept subtypes of *X*. For Cython, what matters is whether "list" (or
"List[X]") refers to exactly a "list" or allows subtypes of the "list"
container type. If the type annotation allows subtypes of builtins, it's
useless for Cython, as it does not allow it to generate better code than it
does anyway. If it means "exactly list and no subtypes of list", Cython can
use it to avoid generating generic fallback code.

Can you explain how this works? How does Cython manage to use the knowledge that the argument's __class__ is equal to builtins.list to generate more efficient code?

Also, don't you have to insert dynamic checks to ensure the caller passes exactly a list anyway?

Very few people subclass builtins.list (usually it's much simpler to subclass collections.abc.MutableSequence). So perhaps Cython could just assume that typing.List means builtins.list and (dynamically) reject calls that pass a subclass of builtins.list?
 
That's why Cython's
current type system enforces exact types when builtin Python types are
declared. (You may call it use case optimised: declare only types that
help.) And it's one of the reasons why the proposed type system
representation is of so little value for Cython. It's simply not intended
for anything but type checking.

Note that I'm not proposing Cython's semantics for a type system
representation that is designed only for type checking, but I guess it
would be nice to at least be able to explicitly express somehow that
subtypes should be disallowed.

Can you come up with a specific proposal? Maybe we're not as far apart as we think. :-)
 
>> If there was at least a notion of "unknown types" that tools like static
>> type checkers MUST ignore (couldn't find that in the PEP either), then
>> Cython users could mix this with Cython's Python level type mapping through
>> the "cython" magic module, e.g. something like "cython.struct(x=cython.int
>> , y=cython.p_char)".
>
> Perhaps you could supply a stub for the cython module that declares
> everything in it as type 'Any'? That should shut the type checker up.

Yes, I guess that would allow Cython's extensions to Python's type system
to be ignored. We could even try to be a little smarter and map some C
parts of Cython's type system down to more generic types in Python's type
system (e.g. int32->int). That would at least give us a minimal baseline
level of "interoperability" (as in "it doesn't break and isn't completely
useless").

Right. I still don't understand enough about Cython's type system to be able to make useful suggestions myself -- I am really hoping that you'll take the time to understand PEP 484 well enough so you can help guide it towards something that would benefit Cython without giving up the existing goals (which are similar to what is done e.g. in Hack and Typescript -- see PEP 482).
 
Regarding non-typish declarations (like the contrived doc() example in my
pull request), will there be a way (maybe in that "stub" mechanism) to tell
type analysis tools that whatever they are looking for, they are not going
to find it in this specific kind of annotation, as it's "not a type"? That
seems sufficiently different (semantically) from "this represents any
type". If it's really left to users to do that at a per-annotation basis,
it sounds cumbersome enough to make the real intention appear as saying
"annotations are for types only".

We'll start out with a way using magic comments (maybe # type: OFF|ON) to disable the interpretation of annotations as types by the type checker during specific sections of a module. The checker will then assume no annotations, and in that case it will assume every argument and return value has the special type 'Any'. (Please do read up in PEP 483 about the important difference between Any and object.) We'll also have a decorator to disable type checking per function or class.

--
--Guido van Rossum (python.org/~guido)