Stub class for Generic to further improve PEP 560
As explained in PEP 560, creation of Generic subclasses is much slower (~7x on my machine). My guess is that a limited number of applications actually need Generic classes at "production" runtime (i.e. when not under analysis of mypy or IDE). I propose a special class that will alias typing.Generic whenever typing.TYPE_CHECKING == True and a no-op stub otherwise. Best Regards, Ilya Kulakov
On 30 November 2017 at 22:08, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
As explained in PEP 560, creation of Generic subclasses is much slower (~7x on my machine). My guess is that a limited number of applications actually need Generic classes at "production" runtime (i.e. when not under analysis of mypy or IDE).
I propose a special class that will alias typing.Generic whenever typing.TYPE_CHECKING == True and a no-op stub otherwise.
FWIW, with PEP 560 generic classes will stay generic at runtime. I am not sure what you mean by this, but practically all existing runtime APIs (with few exceptions, see PEP 560) for generics will stay. -- Ivan
My point is to have a class with public interface identical to typing.Generic, but without all runtime overhead. It's based on the assumption that few developers benefit with typing.Generic addition during production application execution. Those who do, can subclass typing.Generic directly.
On Nov 30, 2017, at 1:22 PM, Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
FWIW, with PEP 560 generic classes will stay generic at runtime. I am not sure what you mean by this, but practically all existing runtime APIs (with few exceptions, see PEP 560) for generics will stay.
On 30 November 2017 at 22:25, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
My point is to have a class with public interface identical to typing.Generic, but without all runtime overhead. It's based on the assumption that few developers benefit with typing.Generic addition during production application execution.
Those who do, can subclass typing.Generic directly
Could you please give some examples for these statements? They still look to abstract for me. -- Ivan
A very rough implementation: typing.py: class _GenericMetaNoop(type): def __getitem__(self, params): return self class _GenericNoop(metaclass=_GenericMetaNoop) pass GenericStub = Generic if TYPE_CHECKING else _GenericNoop elsewhere.py: import typing T = typing.TypeVar('T') class MyClass(typing.GenericStub[T]): pass Now when run under type checkers MyClass will inherit typing.Generic with all runtime complexity. However when run outside of them code will just work.
On Nov 30, 2017, at 1:35 PM, Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
Could you please give some examples for these statements? They still look to abstract for me.
On 30 November 2017 at 22:38, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
A very rough implementation:
typing.py:
class _GenericMetaNoop(type): def __getitem__(self, params): return self
class _GenericNoop(metaclass=_GenericMetaNoop) pass
GenericStub = Generic if TYPE_CHECKING else _GenericNoop
elsewhere.py:
import typing
T = typing.TypeVar('T')
class MyClass(typing.GenericStub[T]): pass
OK, I see. Note that all type checkers known to me never actually read content of typing.py (it is always heavily special cased). Therefore, one can safely write `GenericStub = _GenericNoop`. Anyway, my expectation is that going along this way (i.e. removing all runtime API apart from a necessary minimum) will give a minor speed-up as compared to PEP 560 at the cost of a breaking change (even for small number of developers). PEP 560 already gives overhead of 80% as compared to normal classes in worst case scenario (empty body with a single generic base). This is actually less than for ABCs (they can give up to 120% in worst case scenario). Moreover, performance is not a single motivation for PEP 560, there are other arguments such as metaclass conflicts which will not be solved without the machinery proposed by the PEP. -- Ivan
Anyway, my expectation is that going along this way (i.e. removing all runtime API apart from a necessary minimum) will give a minor speed-up as compared to PEP 560 at the cost of a breaking change (even for small number of developers).
I don't think the change will be breaking: usage of this class will be entirely voluntarily and does not replace typing.Generic
PEP 560 already gives overhead of 80% as compared to normal classes in worst case scenario (empty body with a single generic base). This is actually less than for ABCs (they can give up to 120% in worst case scenario).
GenericMeta inherits from ABCMeta. Do you mean that it will be removed after 560 lands?
Moreover, performance is not a single motivation for PEP 560, there are other arguments such as metaclass conflicts which will not be solved without the machinery proposed by the PEP.
Perhaps you can consider designing Generic / GenericMeta in a way that will allow end user to create GenericStub-alike class without much trouble?
On 1 December 2017 at 00:34, Ilya Kulakov <kulakov.ilya@gmail.com> wrote:
Anyway, my expectation is that going along this way (i.e. removing all runtime API apart from a necessary minimum) will give a minor speed-up as compared to PEP 560 at the cost of a breaking change (even for small number of developers).
I don't think the change will be breaking: usage of this class will be entirely voluntarily and does not replace typing.Generic
If you propose an additional class, then yes, it is non-breaking, but I still don't see much value given a minor performance improvement.
PEP 560 already gives overhead of 80% as compared to normal classes in worst case scenario (empty body with a single generic base). This is actually less than for ABCs (they can give up to 120% in worst case scenario).
GenericMeta inherits from ABCMeta. Do you mean that it will be removed after 560 lands?
Yes, GenericMeta will be removed.
Moreover, performance is not a single motivation for PEP 560, there are other arguments such as metaclass conflicts which will not be solved without the machinery proposed by the PEP.
Perhaps you can consider designing Generic / GenericMeta in a way that will allow end user to create GenericStub-alike class without much trouble?
This can be done, but the hardest part here is not to make the runtime changes, but to get the support of all static type checkers (they need to understand what GenericStub means and add all the necessary special casing). -- Ivan
participants (2)
-
Ilya Kulakov -
Ivan Levkivskyi