handeling very large dictionaries
aahz at pythoncraft.com
Mon Jun 29 04:39:00 CEST 2009
In article <9efff087-bd6e-49fb-ad30-a955a64b85f9 at j32g2000yqh.googlegroups.com>,
mclovin <hanooter at gmail.com> wrote:
>I need to have a dictionary of about 8 gigs (well the data it is
>processing is around 4gb). so naturally i am running into memory
>So i looked around and found bsddb which acts like a dictionary object
>only offloads the data from the RAM to the HDD, however that only
Look at the pickle module. Personally, I'd use SQLite instead of bsddb.
You might also look into a full-blown ORM such as SQLAlchemy.
Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/
"as long as we like the same operating system, things are cool." --piranha
More information about the Python-list