Mailman 3 October 2020 - Typing-sig

Update: Work on Optional Syntax PEP
by maggiemoss＠fb.com 07 Oct '20

07 Oct '20

Hi there, Recently I have been working on an implementation for PEP 604 (https://www.python.org/dev/peps/pep-0604/). While working on this, I started reviewing and exploring what new Optional syntax would look like in the Python language. I've written a rough draft of a PEP and wanted to present it to this email list before submitting a PR. I know this is a topic that has been discussed in many settings (Github issues, in person typing meetups, etc. )previously, and wanted to solicit feedback here first. Let me know if submitting a PR would be a better way to gauge sentiment and feedback on this proposal. Draft Proposal (branch compare): https://github.com/python/peps/compare/master...MaggieMoss:optional-syntax-… Gist: https://gist.github.com/MaggieMoss/c848cb3a581979f445d075c15629c950 Thanks again, Maggie

6 9

Validating type completeness for py.typed packages
by Eric Traut 03 Oct '20

03 Oct '20

PEP 561 indicates that “package maintainers who wish to support type checking of their code MUST add a marker file named py.typed…”. It doesn’t define what “support type checking” means or what expectations are implied. This has led to a situation where packages claim to support type checking but omit many type annotations. There’s currently no tooling that validates the level of “type completeness” for a package, so even well-intentioned package maintainers are unable to confirm that their packages are properly and completely annotated. This leads to situations where type checkers and language servers need to fall back on type inference, which is costly and gives inconsistent results across tools. Ideally, all py.typed packages would have their entire public interface completely annotated. I’m working on a new feature in Pyright that allows package maintainers to determine whether any of the public symbols in their package are missing type annotations. To do this, I need to clearly define what constitutes a “public symbol”. In most cases, the rules are pretty straightforward and follow the naming guidelines set forth in PEP 8 and PEP 484. For example, symbols that begin with an underscore are excluded from the list of public symbols. One area of ambiguity is related to import statements. PEP 484 indicates that within stub files, a symbol is not considered exported unless it is used within an import statement of the form `import x as y` or `from x import y as z` or `from x import *`. The problem is that this rule applies only to “.pyi” files and not to “.py” files. For packages that use inlined types, it’s ambiguous whether an import statement of the form `import x` or `from y import x` should treat `x` as a public symbol that is exported from that module. I can think of a few solutions here: 1. For py.typed packages, type checkers should always apply PEP 484 import rules for “.py” files. If a symbol `x` is imported with an `import x` or `from y import x`, it is treated as “not public”, and any attempt to import it from another package will result in an error. 2. For py.typed packages, PEP 484 rules are _not_ applied for import statements. This maintains backward compatibility. Package maintainers can opt in to PEP 484 rules using some well-defined mechanism. For example, we could define a special flag “stub_import_rules” that can be added to a “py.typed” file. Type checkers could then conditionally use PEP 484 rules for imports. Option 1 will likely break some assumptions for existing packages. Option 2 avoids that break, but it involves more complexity. Any suggestions? Thoughts? -Eric --- Eric Traut Contributor to Pyright and Pylance Microsoft

4 14

Typing dunders, wrappers and overrides
by John Snow 02 Oct '20

02 Oct '20

Hello, greetings from the QEMU project! We've been investigating adopting mypy for some of our python utilities that we use as a kind of turbocharged preprocessor for defining QEMU API types in our project ('QAPI'). So far I've had good success (And only nice things to say about the improvements made in each python version so far), but I do have a question about a few paradigms I see here and again. Is there a short-hand way to type certain dunder methods? For example, we've got an __exit__ method on one of our VM management classes we use for testing. The types for this are fairly arcane for the average C programmer with only surface-level knowledge of Python. e.g., we are left with this: def __exit__(self, # pylint: disable=duplicate-code # see https://github.com/PyCQA/pylint/issues/3619 exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.close() That's a bit of noise! In this case, I assume that for well-known dunders, mypy could be configured to *assume* the canonical types for well-known names. Maybe I am mistaken and this is a horrible assumption to make? On a similar train of thought, I was wondering if there was any discussion about the typing of things like **kwargs. I see this paradigm in python code quite often: def my_cool_wrapper(**kwargs): logging.debug(kwargs['arbitrarily-important-key']) return some_stdlib_function(**kwargs) Is there any formalization or shorthand for declaring this function as a pass-through type? i.e., being able to declare that the entirety of this functions arguments simply match precisely some other function's arguments. Usually, I have been looking into typeshed to copy-paste type signatures back into my code when I need to type something to match the standard library, but this is prone to breakage -- and not always possible, because typeshed uses lots of internal types I can't copy out. Is there a more elegant way to write strictly typed wrappers, or some existing thought/discussion on this area of design? Similar cases pop up for overriding a parent class's method: there are times where I might be attempting to extend an interface with new optional parameters, but sometimes I am merely implementing an interface, and the replication of all those types becomes a source of lots of repeated lines of code. I would be happy to be pointed to archived mailing list discussions, formal proposals, etc. Thanks, --js

3 2

Handling of (potentially mutating) method calls on NewType
by sascha.schlemmer＠me.com 02 Oct '20

02 Oct '20

`NewType` is great for making API code more type safe while not introducing runtime wrappers. Take for example ``` from typing import NewType, List NonEmptyList = NewType('NonEmptyList', List) def prove_list_is_not_empty(seq: List) -> NonEmptyList: if len(seq) > 0: return NonEmptyList(seq) else: raise ValueError('Sequence is empty') def foo(seq: NonEmptyList) -> NoReturn: pass # api function that will explode when passed an empty list ``` Test cases: ``` a = [1, 2, 3] b = prove_list_is_not_empty(a) c = b + a d = b d.pop() # when using pylint it complains that d has no member 'pop' ``` Results: ``` foo(b) # accepted foo(a) # rejected foo(c) # rejected foo(d) # accepted, though see what pylint does for the use-case above ``` In the results above `foo(c)` is rejected by mypy because `reveal_type(c)` is `List[Any]` which I think is good behavior, however `reveal_type(d)` is still `NonEmptyList`. Obviously mypy can't know that `d.pop` is dangerous here (and `d.append(1)` would be fine). Would it be possible (and desirable) for mypy to *downgrade* `d` back to `List[Any]` for all method calls?

1 0