Newbie: Large dictionaries

[Mikkel Rasmussen]

I found the error, that is: my error.

I tried building a dictionary of 65.531 words held in a list. That worked
fine. Then I wanted to test Python by building a *large* dictionary of four
million words. I obtained my test data by concatenating the list with
itself. I poured the four million words *into* the dictionary, but only got
65.531 words out - because there was only as many different words and each
word was its own key.

Always beware of your test data :-)

Now I tested again with better data and got a fine working dictionary. It
only took 90 seconds to delete it after the test :-)

Thanks for your answers!


Mikkel Rasmussen


Mikkel Rasmussen <footech at get2net.dk> wrote in message
news:vpsl6.134$Gv2.4218 at news.get2net.dk...
> Hi!
>
> I can also put more than 4 million entries *into* the dictionary. But I
can
> only get less than 65.000 back out!
>
> What do you get with: len(dict.keys()) ?
>
> And with a loop like
>
> count = 0
> for elem in dict:
>     count = count + 1
> print count
>
> The keys are ordinary words like an ordinary dictionary :-) with an
average
> length of about 6 characters.
>
>
> Mikkel Rasmussen
>
>
> Steve Purcell <stephen_purcell at yahoo.com> wrote in message
> news:mailman.982924567.27809.python-list at python.org...
> > Mikkel Rasmussen wrote:
> > > Why can't there be more than about 65.000 entries in a dictionary, or
> more
> > > importantly: how can I make a large dictionary with room for at least
> > > 500.000
> > > elements?
> >
> > It's probably a memory issue: how big are the keys and/or values?
> >
> > On my machine, I can easily get a million entries into a dictionary:-
> >
> > >>> dict = {}
> > >>> try:
> > ...   for i in xrange(1000000):
> > ...     dict[i] = None
> > ... except:
> > ...   print "stopped at", i
> > ...   raise
> > ...
> > >>>
> >
> > -Steve
> >
> > --
> > Steve Purcell, Pythangelist
> > Get testing at http://pyunit.sourceforge.net/
> > Get servlets at http://pyserv.sourceforge.net/
> > "Even snakes are afraid of snakes." -- Steven Wright
> >
>
>