hashing strings to integers
Adam Funk
a24061 at ducksburg.com
Tue Jun 3 06:32:44 EDT 2014
On 2014-05-27, Steven D'Aprano wrote:
> On Tue, 27 May 2014 16:13:46 +0100, Adam Funk wrote:
>> Well, here's the way it works in my mind:
>>
>> I can store a set of a zillion strings (or a dict with a zillion
>> string keys), but every time I test "if new_string in seen_strings",
>> the computer hashes the new_string using some kind of "short hash",
>> checks the set for matching buckets (I'm assuming this is how python
>> tests set membership --- is that right?),
>
> So far so good. That applies to all objects, not just strings.
>
>
>> then checks any
>> hash-matches for string equality. Testing string equality is slower
>> than integer equality, and strings (unless they are really short)
>> take up a lot more memory than long integers.
>
> But presumably you have to keep the string around anyway. It's going to
> be somewhere, you can't just throw the string away and garbage collect
> it. The dict doesn't store a copy of the string, it stores a reference to
> it, and extra references don't cost much.
In the case where I did something like that, I wasn't keeping copies
of the strings in memory after hashing (& otherwise processing them).
I know that putting the strings' pointers in the set is a light memory
load.
[snipping the rest because...]
You've convinced me. Thanks.
--
I heard that Hans Christian Andersen lifted the title for "The Little
Mermaid" off a Red Lobster Menu. [Bucky Katt]
More information about the Python-list
mailing list