[Python-Dev] Creating dicts from dict subclasses

[Walter Dörwald]

Guido van Rossum wrote:
> On 12/13/06, Walter Dörwald <walter at livinglogic.de> wrote:
>> Guido van Rossum wrote:
>> > On 12/13/06, Walter Dörwald <walter at livinglogic.de> wrote:
>> >> I tried to reimplement weakref.WeakValueDictionary as a subclass of
>> >> dict. The test passes except for one problem: To compare results
>> >> test_weakref.py converts a weakdict to a real dict via dict(weakdict).
>> >> This no longer works because PyDict_Merge() does a PyDict_Check()
>> on the
>> >> argument and then ignores all overwritten methods. (The old version
>> >> worked because UserDict.UserDict was used).
>> >>
>> >> The simplest solution is to replace the PyDict_Check() call with
>> >> PyDict_CheckExact(), but this might slow things down too much, because
>> >> the fallback code basically does:
>> >>
>> >>    for key in iter(arg.keys()):
>> >>       self[key] = arg.__getitem__(key)
>> >>
>> >> Why can't we use:
>> >>
>> >>    for key in iter(arg):
>> >>       self[key] = arg.__getitem__(key)
>> >>
>> >> instead?
>> >
>> > The only reason I can think of is backwards compatibility: not all
>> > "mappings" created pre-2.2 would support iteration. Maybe you could
>> > check for a tp_iter slot and if non-NULL use the latter otherwise use
>> > the original fallback?
>>
>> This doesn't seem to work. It breaks test_update() in test_dict.py which
>> does this:
>>
>>     d = {}
>>     class SimpleUserDict:
>>         def __init__(self):
>>             self.d = {1:1, 2:2, 3:3}
>>         def keys(self):
>>             return self.d.keys()
>>         def __getitem__(self, i):
>>             return self.d[i]
>>     d.update(SimpleUserDict())
>>     self.assertEqual(d, {1:1, 2:2, 3:3})
>>
>> This fails with
>>
>>     KeyError: 0
>>
>> because SimpleUserDict doesn't implement __iter__, so it gets an
>> iterator implementation via __getitem__.
>>
>> So maybe this only makes sense for Python 3.0 where we can demand that
>> dict-like classes implement __iter__?
> 
> Ah, right. But I think you should still use PyDict_CheckExact, and
> slow fallbacks be damned. (I guess you could look for iterkeys first.)

OK, here's a patch that tries iterkeys() before keys():
http://bugs.python.org/1615701

Both versions seem to be faster than Python 2.5:

class dictik:
    def __init__(self, n):
        self.d = dict((i, i) for i in xrange(n))
    def iterkeys(self):
        return iter(self.d)
    def __getitem__(self, i):
        return self.d[i]

class dictk:
    def __init__(self, n):
        self.d = dict((i, i) for i in xrange(n))
    def keys(self):
        return self.d.keys()
    def __getitem__(self, i):
        return self.d[i]

$ python2.5 -mtimeit -s'from foo import dictik, dictk; d=dictk(100000)'
'dict(d)'
10 loops, best of 3: 179 msec per loop
$ ./python -mtimeit -s'from foo import dictik, dictk; d=dictk(100000)'
'dict(d)'
10 loops, best of 3: 138 msec per loop
$ ./python -mtimeit -s'from foo import dictik, dictk; d=dictik(100000)'
'dict(d)'
10 loops, best of 3: 123 msec per loop

Servus,
   Walter