On Thu., 31 Oct. 2019, 11:45 pm Philippe Prados, <philippe.prados@gmail.com> wrote:
To implement a full version of PEP604, I analyze the typing module, started with _GenericAlias.
1) I must rewrite :
  • def _type_check(arg, msg, is_argument=True)
  • def _type_repr(obj)
  • def _collect_type_vars(types)
  • def _subs_tvars(tp, tvars, subs)
  • def _check_generic(cls, parameters)
  • def _remove_dups_flatten(parameters)
  • def _tp_cache(func)
  • class _Final
  • class _Immutable
  • class _SpecialForm(_Final, _Immutable, _root=True)
  • class ForwardRef(_Final, _root=True)
  • class TypeVar(_Final, _Immutable, _root=True)
  • def _is_dunder(attr)
  • class _GenericAlias(_Final, _root=True)
  • class Generic
  • class _TypingEmpty
  • class _TypingEllipsis
  • def _get_protocol_attrs(cls)
  • def _is_callable_members_only(cls)
  • def _allow_reckless_class_cheks()
  • class _ProtocolMeta(ABCMeta)
  • class Protocol(Generic, metaclass=_ProtocolMeta)
2) The function _tp_cache use functools.lru_cache()
def _tp_cache(func):
    cached = functools.lru_cache()(func)

it's not reasonable to move the lru_cache() in the core

However, there may be C level caching machinery that can be used instead (we have a lot of low level caches, lru_cache is just the flexible option that's available to Python code).

3) The method TypeVar.__init__() use:
def_mod = sys._getframe(1).f_globals['__name__']  # for pickling 

4) The method def _allow_reckless_class_cheks() use:
return sys._getframe(3).f_globals['__name__'] in ['abc', 'functools']

_getframe is already a Python level wrapper around an underlying C API. A C implementation of this code would skip the wrapper and call the underlying API directly.


5) The method Protocol.__init_subclass___proto_hook() use:
if (isinstance(annotations, collections.abc.Mapping)
it's not reasonable to move the Mapping type in the core

I believe the collections ABCs are already available to builtin code through the frozen "_collections_abc" module.

If I'm misremembering, and that module is just imported early rather than being frozen as precompiled bytecode, then freezing it as part of the interpreter build process (rather than rewriting it in C) would be the way to handle this (and a proof of concept could likely get away with importing it just before it is needed).


It's not enough to move the typing classes, I must move functools.lru_cache() and dependencies, collections.abs.Mapping and dependencies, and track the frame level.

It's too big for me.


It's certainly not an easy project to tackle.

For some of the specific points you raise though, you wouldn't translate the existing Python code directly to C.

Instead, you would want to look for existing builtin code that does something similar (e.g. by searching for references to "_collections_abc"), and then use C level code that gives the same external behavior, even though it works differently internally.

Cheers,
Nick.