[Numpy-discussion] Possible roadmap addendum: building better text file readers

[Nathaniel Smith]

On Mon, Feb 27, 2012 at 2:44 PM, Erin Sheldon <erin.sheldon at gmail.com> wrote:
> What I've got is a solution for writing and reading structured arrays to
> and from files, both in text files and binary files.  It is written in C
> and python.  It allows reading arbitrary subsets of the data efficiently
> without reading in the whole file.  It defines a class Recfile that
> exposes an array like interface for reading, e.g. x=rf[columns][rows].

What format do you use for binary data? Something tiled? I don't
understand how you can read in a single column of a standard text or
mmap-style binary file any more efficiently than by reading the whole
file.

-- Nathaniel