[issue28383] __hash__ documentation recommends naive XOR to combine but this is suboptimal
New submission from Kevin Norris: The documentation for __hash__ contains this text: "The only required property is that objects which compare equal have the same hash value; it is advised to somehow mix together (e.g. using exclusive or) the hash values for the components of the object that also play a part in comparison of objects." The recommendation of "using exclusive or" is likely to result in programmers naively doing this: def __hash__(self): return hash(self.thing1) ^ hash(self.thing2) ^ hash(self.thing3) In the event that (say) self.thing1 and self.thing2 have almost or exactly the same hash (with "almost" referring to bitwise differences rather than integral distance), this wipes out most or all of the entropy from both values and greatly increases the likelihood of hash collisions. Indeed, Python's own tuple type does not do this (while it does use XOR, it also does some other math to ensure the bits are as mixed up as is practical).[1] Because the correct algorithm is both nontrivial to implement and already exists in the tuple type's __hash__, I propose that the documentation be updated to recommend something like the following: def __hash__(self): return hash((self.thing1, self.thing2, self.thing3)) One possible wording: "The only required property is that objects which compare equal have the same hash value; it is advised to mix together the hash values of the components of the object that also play a part in comparison of objects by packing them into a tuple and hashing the tuple: [code example]" [1]: https://hg.python.org/cpython/file/fca5c4a63251/Objects/tupleobject.c#l348 ---------- assignee: docs@python components: Documentation messages: 278229 nosy: Kevin.Norris, docs@python priority: normal severity: normal status: open title: __hash__ documentation recommends naive XOR to combine but this is suboptimal type: performance versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
Changes by Éric Araujo <merwok@netwok.org>: ---------- nosy: +eric.araujo _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
Alexander Belopolsky added the comment: This makes sense. Note that this is the way hashes are implemented for the datetime objects: <https://hg.python.org/cpython/file/v3.6.0b1/Lib/datetime.py#l635>. ---------- nosy: +belopolsky _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
STINNER Victor added the comment: hash(tuple of attributes) is what I'm using in all my projects. Here is a patch for the doc. ---------- keywords: +patch nosy: +haypo Added file: http://bugs.python.org/file45133/hash_doc.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
Christian Heimes added the comment: ACK! ---------- nosy: +christian.heimes _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
STINNER Victor added the comment: Christian Heimes:
ACK!
Does it mean that my patch LGTY? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
INADA Naoki added the comment: LGTM ---------- nosy: +inada.naoki _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
Roundup Robot added the comment: New changeset cb802a78ceea by Victor Stinner in branch '3.5': doc: Suggest to hash(tuple of attr) rather than XOR https://hg.python.org/cpython/rev/cb802a78ceea ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
Roundup Robot added the comment: New changeset fac2362f248c by Victor Stinner in branch '2.7': doc: Suggest to hash(tuple of attr) rather than XOR https://hg.python.org/cpython/rev/fac2362f248c ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
STINNER Victor added the comment: I updated the doc of Python 2.7, 3.5, 3.6 and default (3.7). Thanks for the suggestion Kevin, thanks for the review Naoki. ---------- resolution: -> fixed status: open -> closed versions: -Python 3.3, Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28383> _______________________________________
participants (7)
-
Alexander Belopolsky -
Christian Heimes -
INADA Naoki -
Kevin Norris -
Roundup Robot -
STINNER Victor -
Éric Araujo