Populating huge data structures from disk

Paul Rubin http
Tue Nov 6 23:14:04 CET 2007


"Michael Bacarella" <mbac at gpshopper.com> writes:
> Very sure.  If we hit the disk at all performance drops
> unacceptably.  The application has low locality of reference so
> on-demand caching isn't an option.  We get the behavior we want when
> we pre-cache; the issue is simply that it takes so long to build
> this cache.

The way I do it is run a separate process that mmaps the file and
reads one byte from each page every half hour or so.  You are right
that it makes a huge difference.



More information about the Python-list mailing list