Guido van Rossum [guido@CNRI.Reston.VA.US] wrote:
Someone has asked me for a dbm clone that can store 16M keys of 350 bytes each, and runs on Linux, HPUX, and NT. That's 5.6 Gigabyte in keys alone! I presume most classic approaches won't cut it since total file size is typicall limited by the seek system call, internal data structures and/or file index format to 2Gb (signed longs) or 4Gb (unsigned longs).
Does anyone have an idea where to start looking? Would a Python extension already exist?
Assuming you mean an interface to a ddbm-style situation, you could easily use berkeley DB, I belive it is limited in the 4TB range...