Populating huge data structures from disk
Tue Nov 6 23:14:04 CET 2007
"Michael Bacarella" <mbac at gpshopper.com> writes:
> Very sure. If we hit the disk at all performance drops
> unacceptably. The application has low locality of reference so
> on-demand caching isn't an option. We get the behavior we want when
> we pre-cache; the issue is simply that it takes so long to build
> this cache.
The way I do it is run a separate process that mmaps the file and
reads one byte from each page every half hour or so. You are right
that it makes a huge difference.
More information about the Python-list