Typing dunders, wrappers and overrides
Hello, greetings from the QEMU project! We've been investigating adopting mypy for some of our python utilities that we use as a kind of turbocharged preprocessor for defining QEMU API types in our project ('QAPI'). So far I've had good success (And only nice things to say about the improvements made in each python version so far), but I do have a question about a few paradigms I see here and again. Is there a short-hand way to type certain dunder methods? For example, we've got an __exit__ method on one of our VM management classes we use for testing. The types for this are fairly arcane for the average C programmer with only surface-level knowledge of Python. e.g., we are left with this: def __exit__(self, # pylint: disable=duplicate-code # see https://github.com/PyCQA/pylint/issues/3619 exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.close() That's a bit of noise! In this case, I assume that for well-known dunders, mypy could be configured to *assume* the canonical types for well-known names. Maybe I am mistaken and this is a horrible assumption to make? On a similar train of thought, I was wondering if there was any discussion about the typing of things like **kwargs. I see this paradigm in python code quite often: def my_cool_wrapper(**kwargs): logging.debug(kwargs['arbitrarily-important-key']) return some_stdlib_function(**kwargs) Is there any formalization or shorthand for declaring this function as a pass-through type? i.e., being able to declare that the entirety of this functions arguments simply match precisely some other function's arguments. Usually, I have been looking into typeshed to copy-paste type signatures back into my code when I need to type something to match the standard library, but this is prone to breakage -- and not always possible, because typeshed uses lots of internal types I can't copy out. Is there a more elegant way to write strictly typed wrappers, or some existing thought/discussion on this area of design? Similar cases pop up for overriding a parent class's method: there are times where I might be attempting to extend an interface with new optional parameters, but sometimes I am merely implementing an interface, and the replication of all those types becomes a source of lots of repeated lines of code. I would be happy to be pointed to archived mailing list discussions, formal proposals, etc. Thanks, --js
El jue., 1 oct. 2020 a las 13:22, John Snow (<jsnow@redhat.com>) escribió:
Hello, greetings from the QEMU project!
We've been investigating adopting mypy for some of our python utilities that we use as a kind of turbocharged preprocessor for defining QEMU API types in our project ('QAPI').
So far I've had good success (And only nice things to say about the improvements made in each python version so far), but I do have a question about a few paradigms I see here and again.
Is there a short-hand way to type certain dunder methods? For example, we've got an __exit__ method on one of our VM management classes we use for testing. The types for this are fairly arcane for the average C programmer with only surface-level knowledge of Python.
e.g., we are left with this:
def __exit__(self, # pylint: disable=duplicate-code
# see https://github.com/PyCQA/pylint/issues/3619
exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.close()
That's a bit of noise! In this case, I assume that for well-known dunders, mypy could be configured to *assume* the canonical types for well-known names. Maybe I am mistaken and this is a horrible assumption to make?
There currently isn't a way, but I definitely see the pain there. Perhaps mypy could assume the type for `__exit__`. I'm not sure there are many dunders other than `__exit__` for which this a problem, since most others either have type-specific annotations (like `__add__`) or are very simple (like `__len__`, which just returns an int).
On a similar train of thought, I was wondering if there was any discussion about the typing of things like **kwargs. I see this paradigm in python code quite often:
def my_cool_wrapper(**kwargs): logging.debug(kwargs['arbitrarily-important-key']) return some_stdlib_function(**kwargs)
Is there any formalization or shorthand for declaring this function as a pass-through type? i.e., being able to declare that the entirety of this functions arguments simply match precisely some other function's arguments.
There isn't. PEP 612 (https://www.python.org/dev/peps/pep-0612/) is close but I think mostly covers generic functions, not functions that wrap some specific other function.
Usually, I have been looking into typeshed to copy-paste type signatures back into my code when I need to type something to match the standard library, but this is prone to breakage -- and not always possible, because typeshed uses lots of internal types I can't copy out. Is there a more elegant way to write strictly typed wrappers, or some existing thought/discussion on this area of design?
Similar cases pop up for overriding a parent class's method: there are times where I might be attempting to extend an interface with new optional parameters, but sometimes I am merely implementing an interface, and the replication of all those types becomes a source of lots of repeated lines of code.
I would be happy to be pointed to archived mailing list discussions, formal proposals, etc.
Thanks, --js _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
There's also some discussion of using TypedDict to type kwargs here: https://github.com/python/mypy/issues/4441 On Fri, 2 Oct 2020 at 02:36, Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El jue., 1 oct. 2020 a las 13:22, John Snow (<jsnow@redhat.com>) escribió:
Hello, greetings from the QEMU project!
We've been investigating adopting mypy for some of our python utilities that we use as a kind of turbocharged preprocessor for defining QEMU API types in our project ('QAPI').
So far I've had good success (And only nice things to say about the improvements made in each python version so far), but I do have a question about a few paradigms I see here and again.
Is there a short-hand way to type certain dunder methods? For example, we've got an __exit__ method on one of our VM management classes we use for testing. The types for this are fairly arcane for the average C programmer with only surface-level knowledge of Python.
e.g., we are left with this:
def __exit__(self, # pylint: disable=duplicate-code
# see https://github.com/PyCQA/pylint/issues/3619
exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.close()
That's a bit of noise! In this case, I assume that for well-known dunders, mypy could be configured to *assume* the canonical types for well-known names. Maybe I am mistaken and this is a horrible assumption to make?
There currently isn't a way, but I definitely see the pain there. Perhaps mypy could assume the type for `__exit__`.
I'm not sure there are many dunders other than `__exit__` for which this a problem, since most others either have type-specific annotations (like `__add__`) or are very simple (like `__len__`, which just returns an int).
On a similar train of thought, I was wondering if there was any discussion about the typing of things like **kwargs. I see this paradigm in python code quite often:
def my_cool_wrapper(**kwargs): logging.debug(kwargs['arbitrarily-important-key']) return some_stdlib_function(**kwargs)
Is there any formalization or shorthand for declaring this function as a pass-through type? i.e., being able to declare that the entirety of this functions arguments simply match precisely some other function's arguments.
There isn't. PEP 612 (https://www.python.org/dev/peps/pep-0612/) is close but I think mostly covers generic functions, not functions that wrap some specific other function.
Usually, I have been looking into typeshed to copy-paste type signatures back into my code when I need to type something to match the standard library, but this is prone to breakage -- and not always possible, because typeshed uses lots of internal types I can't copy out. Is there a more elegant way to write strictly typed wrappers, or some existing thought/discussion on this area of design?
Similar cases pop up for overriding a parent class's method: there are times where I might be attempting to extend an interface with new optional parameters, but sometimes I am merely implementing an interface, and the replication of all those types becomes a source of lots of repeated lines of code.
I would be happy to be pointed to archived mailing list discussions, formal proposals, etc.
Thanks, --js _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: hauntsaninja@gmail.com
participants (3)
-
Jelle Zijlstra -
John Snow -
Shantanu Jain