Populating a dictionary, fast [SOLVED SOLVED]

[Steven D'Aprano]

On Wed, 14 Nov 2007 18:16:25 +0100, Hrvoje Niksic wrote:

> Aaron Watters <aaron.watters at gmail.com> writes:
> 
>> On Nov 12, 12:46 pm, "Michael Bacarella" <m... at gpshopper.com> wrote:
>>>
>>> > It takes about 20 seconds for me. It's possible it's related to
>>> > int/long
>>> > unification - try using Python 2.5. If you can't switch to 2.5, try
>>> > using string keys instead of longs.
>>>
>>> Yes, this was it.  It ran *very* fast on Python v2.5.
>>
>> Um.  Is this the take away from this thread?  Longs as dictionary keys
>> are bad?  Only for older versions of Python?
> 
> It sounds like Python 2.4 (and previous versions) had a bug when
> populating large dicts on 64-bit architectures.

No, I found very similar behaviour with Python 2.5.


>> Someone please summarize.
> 
> Yes, that would be good.


On systems with multiple CPUs or 64-bit systems, or both, creating and/or 
deleting a multi-megabyte dictionary in recent versions of Python (2.3, 
2.4, 2.5 at least) takes a LONG time, of the order of 30+ minutes, 
compared to seconds if the system only has a single CPU. Turning garbage 
collection off doesn't help.


-- 
Steven.