very large dictionaries
M.Waack at gmx.de
Wed Jun 16 22:07:47 CEST 2004
> I need to do a search through about 50 million records, each of
> which are less than 100 bytes wide. A database is actually too slow
> for this, so I thought of optimising the data and putting it all in
You'll hardly find something faster than a real database.
> There is a single key field, so a dictionary is an obvious choice
> for a structure, since Python optimises these nicely.
> But is there a better choice? Is it worth building some sort of
It depends on the structure of the data. What kind of data do you
have, what are you looking for and on what machine (at least how
much memory is available) will the whole thing run?
More information about the Python-list