[PYTHON MATRIX-SIG] More on Less Copying
Tue, 11 Mar 1997 13:34:33 -0500
Alan Watson writes:
> > > (a) One of my needs is to read large arrays of arbitrary rank from
> > > binary files. The size of the array is known in advance but need not
> > > be equal to the remaining size of the file.
> > You are adressing an important topic that has largely been neglected
> > so far: file I/O, both in binary and in text form, suitable for
> > interchanging data with non-Python programs
This problem can be easily solved with an extension module rather than by
adding file io methods to array objects. For many reasons I think this is
the right solution. In only the most extreme cases will the temporary
double memory currently required for an array on file input be a problem.
Notice that this memory is used only for a very short time. In general,
if your memory is so tight that you can't afford this brief hit on a single
array, then you're really pushing the limits of where NumPy is applicable
(vs. hand-crafted C).
I really want to keep the basic array objects as simple as possible with
the hopes that one day they can be added to the core python distribution.
This would make extension writing involving large binary arrays much
easier for the community at large.
So, I encourage you to code up a file/io module for NumPy arrays. If I
like it a lot I might add it to the standard distribution, if not you can
still easily make it available to all users of NumPy.
> > I doubt this will happen. The assumption that numbers are immutable is
> > pretty basic in Python, and a lot of code would break if it were
> > violated. At best, operators like += would be added for mutable types
> > (e.g. lists) and classes. And even that doesn't look like a
> > high-priority item for future versions.
> Operators like += need not break the imutable types; they could act
> as a shorthand and might save looking up the same name twice.
I don't know if his opinions have changed lately, but at the Livermore
workshop, Guido expressed a definite willingness to consider += operators.
The best way to get him to take such a proposal seriously is to code up a
nice set of patches that implement the new operators and release them for
comment (hint, hint).
MATRIX-SIG - SIG on Matrix Math for Python
send messages to: firstname.lastname@example.org
administrivia to: email@example.com