Improve ABCs _dump_registry() readability
In python 2.7, ABCs's caches and registries are sets. But in python 3.6 they are WeakSet. In consequence, the output of _dump_registry() is almost useless:
from collections import abc abc.Iterator._dump_registry() Class: collections.abc.Iterator Inv.counter: 40 _abc_cache: <_weakrefset.WeakSet object at 0x7f4b58fe2668> _abc_negative_cache: <_weakrefset.WeakSet object at 0x7f4b53283780> _abc_negative_cache_version: 40 _abc_registry: <_weakrefset.WeakSet object at 0x7f4b58fe2630>
We could convert them into a regular set before printing: if isinstance(value, WeakSet): value = set(value) The result:
abc.Iterator._dump_registry() Class: collections.abc.Iterator Inv.counter: 40 _abc_cache: {<class 'dict_valueiterator'>, <class 'bytearray_iterator'>, <class 'tuple_iterator'>, <class 'dict_itemiterator'>, <class 'dict_keyiterator'>, <class 'str_iterator'>, <class 'zip'>, <class 'set_iterator'>, <class 'list_reverseiterator'>, <class 'range_iterator'>, <class 'longrange_iterator'>, <class 'list_iterator'>, <class 'bytes_iterator'>} _abc_negative_cache: set() _abc_negative_cache_version: 40 _abc_registry: set()
NB: It seems pretty weird to me that registry is empty... All the iterators in the cache should've been in the registry instead, should'nt they?
Yeah, I guess few developers have needed to use _dump_registry(), and also it's easy enough to just access e.g. Iterator._abc_registry yourself. The reason Iterator._abc_registry is empty is that no class directly registered with it -- they are all registered with e.g. Sequence. The cache includes classes registered with subclasses, but the registry itself does not. I guess a PR to fix the registry output would make sense (first file a bug on bugs.python.org for it). On Sat, Dec 30, 2017 at 11:19 PM, Yahya Abou 'Imran via Python-ideas < python-ideas@python.org> wrote:
In python 2.7, ABCs's caches and registries are sets. But in python 3.6 they are WeakSet. In consequence, the output of _dump_registry() is almost useless:
from collections import abc abc.Iterator._dump_registry() Class: collections.abc.Iterator Inv.counter: 40 _abc_cache: <_weakrefset.WeakSet object at 0x7f4b58fe2668> _abc_negative_cache: <_weakrefset.WeakSet object at 0x7f4b53283780> _abc_negative_cache_version: 40 _abc_registry: <_weakrefset.WeakSet object at 0x7f4b58fe2630>
We could convert them into a regular set before printing:
if isinstance(value, WeakSet): value = set(value)
The result:
abc.Iterator._dump_registry() Class: collections.abc.Iterator Inv.counter: 40 _abc_cache: {<class 'dict_valueiterator'>, <class 'bytearray_iterator'>, <class 'tuple_iterator'>, <class 'dict_itemiterator'>, <class 'dict_keyiterator'>, <class 'str_iterator'>, <class 'zip'>, <class 'set_iterator'>, <class 'list_reverseiterator'>, <class 'range_iterator'>, <class 'longrange_iterator'>, <class 'list_iterator'>, <class 'bytes_iterator'>} _abc_negative_cache: set() _abc_negative_cache_version: 40 _abc_registry: set()
NB: It seems pretty weird to me that registry is empty... All the iterators in the cache should've been in the registry instead, should'nt they? _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
Yeah, I guess few developers have needed to use _dump_registry(), and also it's easy enough to just access e.g. Iterator._abc_registry yourself.
Yes, I saw that it's not well-known. I was studying hard the internals of ABCs and ABCMeta, so I end up using it and modifying it.
The reason Iterator._abc_registry is empty is that no class directly registered with it -- they are all registered with e.g. Sequence. The cache includes classes registered with subclasses, but the registry itself does not.
No, in the source code they are! in _collections_abc.py, just after Iterator definition: Iterator.register(bytes_iterator) Iterator.register(bytearray_iterator) #Iterator.register(callable_iterator) Iterator.register(dict_keyiterator) Iterator.register(dict_valueiterator) Iterator.register(dict_itemiterator) Iterator.register(list_iterator) Iterator.register(list_reverseiterator) Iterator.register(range_iterator) Iterator.register(longrange_iterator) Iterator.register(set_iterator) Iterator.register(str_iterator) Iterator.register(tuple_iterator) Iterator.register(zip_iterator) For some reason, the register is being cleared at some point. I tried: Iterator.register(bytes_iterator) Iterator._dump_registry(open('iterator_registry.log', 'w')) Iterator.register(bytearray_iterator) . . . And I got: $ cat iterator_registry.log Class: collections.abc.Iterator Inv.counter: 8 _abc_cache: {<class 'bytes_iterator'>} _abc_negative_cache: set() _abc_negative_cache_version: 8 _abc_registry: set() It's going into the cache and not into the registry. Strange behaviour...
I guess a PR to fix the registry output would make sense (first file a bug on bugs.python.org for it).
Ok, I will!
On 31 December 2017 at 19:24, Yahya Abou 'Imran via Python-ideas < python-ideas@python.org> wrote:
I guess a PR to fix the registry output would make sense (first file a bug on bugs.python.org for it).
Ok, I will!
Please don't hurry with this. I am going to rewrite ABCMeta in C soon. In fact most of the work is done but I am waiting for implementation of PEP 560 to settle (need few more days for this). In the C version the caches/registry will be simpler and will not use WeakSet (instead they will be thin C wrappers around normal sets). -- Ivan
I guess a PR to fix the registry output would make sense (first file a bug onbugs.python.org for it).
Ok, I will!
Please don't hurry with this. I am going to rewrite ABCMeta in C soon. In fact most of the work is done but I am waiting for implementation of PEP 560 to settle (need few more days for this).
In the C version the caches/registry will be simpler and will not use WeakSet (instead they will be thin C wrappers around normal sets).
Ok, no problem. Found out myself why the registry's empty: every iterator passed to Iterator.register() defines __iter__ and __next__, so they satisfy Iterator.__subclasshook__ and are added to the cache beforehand.
Ah, glad the mystery's solved! And sorry for the misdirection. On Sun, Dec 31, 2017 at 11:38 AM, Yahya Abou 'Imran < yahya-abou-imran@protonmail.com> wrote:
I guess a PR to fix the registry output would make sense (first file a bug onbugs.python.org for it).
Ok, I will!
Please don't hurry with this. I am going to rewrite ABCMeta in C soon. In fact most of the work is done but I am waiting for implementation of PEP 560 to settle (need few more days for this).
In the C version the caches/registry will be simpler and will not use WeakSet (instead they will be thin C wrappers around normal sets).
Ok, no problem.
Found out myself why the registry's empty: every iterator passed to Iterator.register() defines __iter__ and __next__, so they satisfy Iterator.__subclasshook__ and are added to the cache beforehand.
-- --Guido van Rossum (python.org/~guido)
On Sun, 31 Dec 2017 19:31:06 +0100 Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
On 31 December 2017 at 19:24, Yahya Abou 'Imran via Python-ideas < python-ideas@python.org> wrote:
I guess a PR to fix the registry output would make sense (first file a bug on bugs.python.org for it).
Ok, I will!
Please don't hurry with this. I am going to rewrite ABCMeta in C soon. In fact most of the work is done but I am waiting for implementation of PEP 560 to settle (need few more days for this).
In the C version the caches/registry will be simpler and will not use WeakSet (instead they will be thin C wrappers around normal sets).
Hmm... Just because you are rewriting the thing in C doesn't mean that Yahya shouldn't submit a patch for the Python version (which I assume will be staying around anyway). Regards Antoine.
On 31 December 2017 at 20:05, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 31 Dec 2017 19:31:06 +0100 Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
On 31 December 2017 at 19:24, Yahya Abou 'Imran via Python-ideas < python-ideas@python.org> wrote:
I guess a PR to fix the registry output would make sense (first file a bug on bugs.python.org for it).
Ok, I will!
Please don't hurry with this. I am going to rewrite ABCMeta in C soon. In fact most of the work is done but I am waiting for implementation of PEP 560 to settle (need few more days for this).
In the C version the caches/registry will be simpler and will not use WeakSet (instead they will be thin C wrappers around normal sets).
Hmm... Just because you are rewriting the thing in C doesn't mean that Yahya shouldn't submit a patch for the Python version (which I assume will be staying around anyway).
Yes, good point! -- Ivan
participants (4)
-
Antoine Pitrou -
Guido van Rossum -
Ivan Levkivskyi -
Yahya Abou 'Imran