The 'arrayfrombuffer' package features support for Numerical Python arrays whose contents are stored in buffer objects, including memory-mapped files. This has the following advantages: - loading your array from a file is easy --- a module import and a single function call --- and doesn't use excessive amounts of memory. - loading your array is quick; it doesn't need to be copied from one part of memory to another in order to be loaded. - your array gets demand-loaded; parts you aren't using don't need to be in memory or in swap. - under memory-pressure conditions, your array doesn't use up swap, and parts of it you haven't modified can be evicted from RAM without the need for a disk write - your arrays can be bigger than your physical memory - when you modify your array, only the parts you modify get written back out to disk This is something that's been requested on the Numpy list a few times a year since 1999. arrayfrombuffer lives at http://pobox.com/~kragen/sw/arrayfrombuffer/ The current version is version 2; it is released under the X11 license (the BSD license without the advertising clause). <kragen@pobox.com> <P><A HREF="http://pobox.com/~kragen/sw/arrayfrombuffer/">arrayfrombuffer 2</A> - creates Numeric arrays from memory-mapped files. (23-Jan-02)