[Python-Dev] Pickling w/ low overhead

[David Ascher]

On Tue, 3 Aug 1999, Skip Montanaro wrote:

> Using __getstate__/__setstate__, could you create a compressed
> representation using zlib or some other scheme?  I don't know how well
> numeric data compresses in general, but that might help.  Also, I trust you
> use cPickle when it's available, yes?

I *really* hate to admit it, but I've found the source of the most massive
problem in the pickling process that I was using.  I didn't use binary
mode, which meant that the huge strings were written & read
one-character-at-a-time.

I think I'll put a big fat note in the NumPy doc to that effect.

(note that luckily this just affected my usage, not all NumPy users).

<embarassed sheepish grin>

--da