[sapug] Large dictionaries

Darryl Ross spam at afoyi.com
Thu May 11 10:29:53 CEST 2006


Chris Foote wrote:

> Does anyone know of a fast hash module which is more optimal for
> large datasets ?
> 
> p.s. Disk-based DBs are out of the question because most key lookups
> will result in a miss, and lookup time is critical for this application.


In memory sqlite database (use ':memory:' as the filename) with
appropriate indexes?

Make sure you put any inserts into a transaction. Using a disk based
database, inserting 10,000 rows went from being a multi-minute operation
to a split-second operation, just by putting them inside a BEGIN/COMMIT.

Cheers
D

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 187 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/sapug/attachments/20060511/da0f91ef/attachment.pgp 


More information about the sapug mailing list