optimizing memory utilization

Frithiof Andreas Jensen frithiof.jensen at die_spammer_die.ericsson.com
Wed Sep 15 14:37:50 CEST 2004


"Wes Crucius" <wcrucius at sandc.com> wrote in message
news:1db49385.0409141355.5d204a03 at posting.google.com...

> Thanks for the specific suggestion, but it's an approach I'd hoped to
> avoid.  I'd rather get 2Gig of RAM if I was confident I could make it
> work that way...  The problem with a file-based database (versus an
> in-memory data-structure-based database as I was meaning) is
> performance.

I am not convinced ;-)

The people designing databases have run into exactly that kind of problem
and they have spent a long time studying algorithms, eficcient
datastructures and writing confererence papers on how to cleverly map and
search through said data structures that may not fit in memory - to the
point where I think one will have a hard time doing it better by hand.

PySQLite will try to keep everything in memory, so if you have enough, it
will only hit the disk on opening the database and on comitting a change.
Commit can take a long time though.





More information about the Python-list mailing list