[PYTHON MATRIX-SIG] Re: Saving HUGE arrays

[Guido van Rossum]

> How about a stream-like protocol, i.e. implementing read() and write()?
> That could be done for non-contiguous arrays, and still be efficient.

This makes some sense, except that it wouldn't (really) help for the
situation where you have a HUGE array (HUGE == too big to copy in
virtual memory) and you need to write it to disk.  If the array is
contiguous, a single write() system call could write the entire thing
to disk.

Next try (though this is getting complicated :-( ): how about a
"sequence-of-buffers" interface.  E.g.:

long PyBuffer_Get(object *o, int i, const void **addres);

would return the length of the i'th contiguous chunk, and store its
address via *address; when there are no more chunks, it can return 0,
and for non-buffering objects, it can return -1 and set an exception.

--Guido van Rossum (home page: http://www.python.org/~guido/)

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================