[Python-Dev] Hashes in Python3.5 for tuples and frozensets

[Anthony Flury]

Victor,
Thanks for the link, but to be honest it will just confuse people - 
neither the link or the related bpo entries state that the fix is only 
limited to strings. They simply talk about hash randomization - which in 
my opinion implies ALL hash algorithms; which is why I asked the question.

I am not sure how much should be exposed about the scope of security 
fixes but you can understand my (and other's) confusion.

I am aware that applications shouldn't make assumptions about the value 
of any given hash value - apart from some simple assumptions based hash 
value equality (i.e. if two objects have different hash values they 
can't be the same value).

/B//TW : //
//
//This question was prompted by a question on a social media platform 
about the whether hash values are transferable between across platforms. 
Everything I could find stated that after Python 3.3 ALL hash values 
were randomized - but that clearly isn't the case; and the original 
questioner identified that some hash values are randomized and other 
aren't.//
//
//I did suggest strongly to the original questioner that relying on the 
same hash value across different platforms wasn't a clever solution - 
their original plan was to store hash values in a cross system database 
to enable quick retrieval of data (!!!). I did remind the OP that a hash 
value wasn't guaranteed to be unique anyway - and they might come across 
two different values with the same hash - and no way to distinguish 
between them if all they have is the hash. Hopefully their revised 
design will store the key, not the hash./


On 17/05/18 07:38, Victor Stinner wrote:
> Hi,
>
> String hash is randomized, but not the integer hash:
>
> $ python3.5 -c 'print(hash("abc"))'
> -8844814677999896014
> $ python3.5 -c 'print(hash("abc"))'
> -7757160699952389646
>
> $ python3.5 -c 'print(hash(1))'
> 1
> $ python3.5 -c 'print(hash(1))'
> 1
>
> frozenset hash is combined from values of the set. So it's only
> randomized if values hashes are randomized.
>
> The denial of service is more likely to occur with strings as keys,
> than with integers.
>
> See the following link for more information:
> http://python-security.readthedocs.io/vuln/cve-2012-1150_hash_dos.html
>
> Victor
>
> 2018-05-16 17:48 GMT-04:00 Anthony Flury via Python-Dev <python-dev at python.org>:
>> This may be known but I wanted to ask this esteemed body first.
>>
>> I understand that from Python3.3 there was a security fix to ensure that
>> different python processes would generate different hash value for the same
>> input - to prevent denial of service based on crafted hash conflicts.
>>
>> I opened two python REPLs on my Linux 64bit PC and did the following
>>
>> Terminal 1:
>>
>>      >>> hash('Hello World')
>>     -1010252950208276719
>>
>>      >>> hash( frozenset({1,9}) )
>>       -7625378979602737914
>>      >>> hash(frozenset({300,301}))
>>     -8571255922896611313
>>
>>      >>> hash((1,9))
>>     3713081631926832981
>>      >>> hash((875,932))
>>     3712694086932196356
>>
>>
>>
>> Terminal 2:
>>
>>      >>> hash('Hello World')
>>     -8267767374510285039
>>
>>      >>> hash( frozenset({1,9}) )
>>       -7625378979602737914
>>      >>> hash(frozenset({300,301}))
>>     -8571255922896611313
>>
>>      >>> hash((1,9))
>>     3713081631926832981
>>      >>> hash((875,932))
>>     3712694086932196356
>>
>> As can be seen - taking a hash of a string does indeed create a different
>> value between the two processes (as expected).
>>
>> However the frozen set hash, the same in both cases, as is the hash of the
>> tuples - suggesting that the vulnerability resolved in Python 3.3 wasn't
>> resolved across all potentially hashable values. lI even used different
>> large numbers to ensure that the integers weren't being interned.
>>
>> I can imagine that frozensets aren't used frequently as hash keys - but I
>> would think that tuples are regularly used. Since that their hashes are not
>> salted does the vulnerability still exist in some form ?.
>>
>> --
>> --
>> Anthony Flury
>> email : *Anthony.flury at btinternet.com*
>> Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>*
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com


-- 
-- 
Anthony Flury
email : *Anthony.flury at btinternet.com*
Twitter : *@TonyFlury <https://twitter.com/TonyFlury/>*