Populating a dictionary, fast [SOLVED SOLVED]
steve at REMOVE-THIS-cybersource.com.au
Thu Nov 15 00:26:17 CET 2007
On Wed, 14 Nov 2007 18:16:25 +0100, Hrvoje Niksic wrote:
> Aaron Watters <aaron.watters at gmail.com> writes:
>> On Nov 12, 12:46 pm, "Michael Bacarella" <m... at gpshopper.com> wrote:
>>> > It takes about 20 seconds for me. It's possible it's related to
>>> > int/long
>>> > unification - try using Python 2.5. If you can't switch to 2.5, try
>>> > using string keys instead of longs.
>>> Yes, this was it. It ran *very* fast on Python v2.5.
>> Um. Is this the take away from this thread? Longs as dictionary keys
>> are bad? Only for older versions of Python?
> It sounds like Python 2.4 (and previous versions) had a bug when
> populating large dicts on 64-bit architectures.
No, I found very similar behaviour with Python 2.5.
>> Someone please summarize.
> Yes, that would be good.
On systems with multiple CPUs or 64-bit systems, or both, creating and/or
deleting a multi-megabyte dictionary in recent versions of Python (2.3,
2.4, 2.5 at least) takes a LONG time, of the order of 30+ minutes,
compared to seconds if the system only has a single CPU. Turning garbage
collection off doesn't help.
More information about the Python-list