Whoops, I definitely misremembered that!

It explains that people request something for Any too (but I don’t want to propose a similar hack there, you gave the reason exactly).

But Callable is definitely the next highest on the list not yet covered, I didn’t misremember that. :-)

—Guido

On Wed, Nov 30, 2022 at 23:05 Shantanu Jain <hauntsaninja@gmail.com> wrote:
> Callable was the next most common special form after various things that were given built-in names by PEP 585. A substitute for `Any` is very low on the list 

This isn't true, Any is used significantly more than Callable, see my comment in https://github.com/python/cpython/issues/86268#issuecomment-1093887929

Let's see if I can pull more up-to-date numbers...
  • The corpus I'm using is https://github.com/hauntsaninja/mypy_primer (note that the corpus is much larger than it was two years ago^)
  • I'm searching for imports of the form `from typing import X, Y, Z` or `from collections.abc import X, Y, Z` (since that's now quite common!)
  • Search is done using the AST. I've attached code to this email, feel free to check or run on a corpus of your own. Numbers roughly line up to some grep spot checks.
  • Most of these projects use a type checker in CI — so I'd say "typically" typed, although you may still consider that "poor" ;-)
  • Full list of projects here
Out of about 10,000 files that contain an import of this form:

Any: 5686
Optional: 3340
List: 2829
Dict: 2415
Callable: 2216
Tuple: 2000
Union: 1967
TYPE_CHECKING: 1817
cast: 1296
Sequence: 964
Iterable: 939
TypeVar: 815
Mapping: 754
Iterator: 617
Type: 614
Set: 515
Generator: 420
NamedTuple: 411
Literal: 384
overload: 312
Generic: 303
Final: 284
Awaitable: 246
TypedDict: 186
MutableMapping: 151
IO: 149
Coroutine: 147
Collection: 145
Hashable: 138
ClassVar: 130
NoReturn: 116
... [longer tail of stuff] ...


On Wed, 30 Nov 2022 at 22:07, Guido van Rossum <guido@python.org> wrote:
There's not much difference between the situation for `callable` and for, say, `list` though. Both are built-ins, and both continue to be valid when called: `callable(x)` and `list(x)` keep their meaning. PEP 585 added `list[itemtype]`, which was previously invalid; a new PEP could add `callable[[arg1, arg2, ...], ret]`, with a new meaning.

The form `callable[...]` is currently *always* invalid (just like `list[...]` used to be); so it could be given a new meaning regardless of context.

The main difficulty would seem to be that `callable(x)` needs to continue returning a bool. Given that *if* we were to do this, type checkers would have to special-case `callable` anyways (it's not a simple generic type like `list` -- `Callable` is also a special form), the special-casing could include "and if it's not subscripted, it's a boolean function". I have a feeling that this would be doable -- we have some other rather special forms, e.g. `type`, which can be used as `type(x)`, `type(name, bases, namespace)`, and `type[cls]`. I don't know about Pyright, but mypy certainly has plenty of references to "builtins.type" that make me think it special-cases the snot out of it, and whatever its definition in typeshed doesn't convey the desired semantics.

I agree that there's no need to replace everything defined in typing.py with a builtin form; but PEP 677 (Callable Type Syntax) was proposed because it was the next most common special form after various things that were given built-in names by PEP 585. A substitute for `Any` is very low on the list (in part because its use is often a symptom of poorly typed code rather than of a missing typing feature).



On Wed, Nov 30, 2022 at 9:13 PM Eric Traut <eric@traut.com> wrote:
I'm -1 on both `any` and `callable`.

If the Python grammar distinguished clearly between type expressions and value expressions like most other languages, this would be feasible because the meaning of `any` and `callable` would be unambiguous from its syntactical position. However, there are many places in Python where a value expression is treated by a type checker as though it is a "type context". This includes cases like base classes in class declarations, bounds and constraints in TypeVar declarations, arguments to NewType, isinstance, cast, etc. Because of these cases, `any` and `callable` would be treated as their traditional builtin meaning by the runtime, and their meaning would be ambiguous for type checkers.

For this reason, I don't this proposal would work. Or it would require that the lowercase forms of `any` and `callable` would be usable only in certain places where type expressions are allowed, and developers would need to know which locations were OK and which were problematic. That would be really confusing.

I agree with Shantanu that it's not a goal to cover all typing use cases without imports, and I'm fine with requiring developers to import `Callable` and `Any`.

 -Eric

--
Eric Traut
Contributor to Pylance & Pyright
Microsoft
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: guido@python.org


--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: hauntsaninja@gmail.com
--
--Guido (mobile)