defaultdict.fromkeys returns a surprising defaultdict

[Raymond Hettinger]

On Jun 3, 1:11 pm, Matthew Wilson <m... at tplus1.com> wrote:
> I used defaultdict.fromkeys to make a new defaultdict instance, but I
> was surprised by behavior:
>
>     >>> b = defaultdict.fromkeys(['x', 'y'], list)
>
>     >>> b
>     defaultdict(None, {'y': <type 'list'>, 'x': <type 'list'>})
>
>     >>> b['x']
>     <type 'list'>
>
>     >>> b['z']
>     ------------------------------------------------------------
>     Traceback (most recent call last):
>       File "<ipython console>", line 1, in <module>
>     KeyError: 'z'
>
> I think that what is really going on is that fromdict makes a regular
> dictionary, and then hands it off to the defaultdict class.

Nope.  It works like this:

def fromkeys(iterable, value):
    d = defaultdict(None)
    for k in iterable:
        d[k] = value
    return d

Use fromkeys() to set all entries to a single specific value.
Remember that setting an entry never triggers the default factory;
instead, it is triggered on the *lookup* of a missing key.  To do what
you likely intended, don't use fromkeys().  Instead write something
like:

    d = defaultdict(list)
    for elem in iterable:
        d[elem]   # factory triggered by lookup of missing key



> I find this confusing, because now I have a defaultdict that raises a
> KeyError.

It is only confusing because you had already convinced yourself that
the second argument in fromkeys() was the default factory function.
Of course, its real meaning is the single value assigned by fromkeys
for every insertion (the same way it works for regular dicts).  Once
you realize that the second argument was the single assigned value, it
is easy to understand that the default factory function is None and
that its documented behavior is to raise a KeyError when a missing key
is looked-up.

The confusion is mostly based on a misunderstanding of how
defaultdicts work (calling a factory whenever a missing key is looked-
up, not when it is inserted).  It is alo based on a misunderstanding
of what fromkeys does (assign the same value over and over for each
key in the input interable).  Given those two misconceptions,
confusion over the result was inevitable.

> Would it be better if defaultdict.fromkeys raised a
> NotImplementedException?

No.  It is a dict subclass and should provide all of those methods.
Also, it is useful in its current form.


> Or would it be better to redefine how defaultdict.fromkeys works, so
> that it first creates the defaultdict, and then goes through the keys?
>
> All comments welcome.  If I get some positive feedback, I'm going to try
> to submit a patch.

No need.  The patch would be rejected.  It would break existing code
that uses default.fromkeys() as designed and documented.


Raymond