[issue3458] dict.update() optimisation gives unexpected/invalid results when passed a subclass of DictType

[David Farrar]

New submission from David Farrar <pythonbugs at spam.decafbad.co.uk>:

Using a python module that expected me to pass a dictionary as a
parameter to a function, I found that I was getting unexpected results
when using a class which inherits from types.DictType.

The module was passing the instance I had supplied as a parameter to
update() on a newly created dictionary. I wanted my class to present its
methods as items of the dictionary it was pretending to be. Even though
I could see that __getattribute__ had been called to get 'keys', keys()
was not called and I ended up with an empty dictionary.

Trying to find the cause of the problem, I downloaded the python source
package and found that in PyDict_Merge (in Objects/dictobject.c, line
1359), there is an optimisation that is run if PyDict_Check determines
that the parameter we passed is itself a dictionary. The optimisation
takes advantage of the fact that we know how data is stored in the
dictionary and accesses whatever it needs directly.

I had overridden keys() to provide the result I wanted but this
optimisation prevented the code from ever being run, leading to code
that seems like it should work failing with no apparent cause. The only
way I could find around this was to raise AttributeError in
__getattribute__ when getting 'keys', causing dict_update_common to call
PyDict_MergeFromSeq2 in place of PyDict_Merge.

Should the call to PyDict_Check in PyDict_Merge not be replaced with
PyDict_CheckExact ? This would keep the optimisation in place in most
cases, where we know that it is safe, still allowing keys() to be
overridden as you expect.

----------
components: Interpreter Core
files: dictionary.patch
keywords: patch
messages: 70353
nosy: david
severity: normal
status: open
title: dict.update() optimisation gives unexpected/invalid results when passed a subclass of DictType
type: behavior
versions: Python 2.5, Python 2.6
Added file: http://bugs.python.org/file10997/dictionary.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3458>
_______________________________________