Strange array.array performance

[John Machin]

On Feb 20, 6:53 am, Maxim Khitrov <mkhit... at gmail.com> wrote:
> On Thu, Feb 19, 2009 at 2:35 PM, Robert Kern <robert.k... at gmail.com> wrote:
> > On 2009-02-19 12:52, Maxim Khitrov wrote:
>
> >> Hello all,
>
> >> I'm currently writing a Python<->  MATLAB interface with ctypes and
> >> array.array class, using which I'll need to push large amounts of data
> >> to MATLAB.
>
> > Have you taken a look at mlabwrap?
>
> >  http://mlabwrap.sourceforge.net/
>
> > At the very least, you will probably want to use numpy arrays instead of
> > array.array.
>
> >  http://numpy.scipy.org/
>
> I have, but numpy is not currently available for python 2.6, which is
> what I need for some other features, and I'm trying to keep the
> dependencies down in any case. Mlabwrap description doesn't mention if
> it is thread-safe, and that's another one of my requirements.
>
> The only feature that I'm missing with array.array is the ability to
> quickly pre-allocate large chunks of memory. To do that right now I'm
> using array('d', (0,) * size).

It would go somewhat faster if you gave it a float instead of an int.

> It would be nice if array accepted an
> int as the second argument indicating how much memory to allocate and
> initialize to 0.

While you're waiting for that to happen, you'll have to use the
fromstring trick, or another gimmick that is faster and is likely not
to use an extra temp 8Mb for a 1M-element array, as I presume the
fromstring does.

[Python 2.6.1 on Windows XP SP3]
[Processor: x86 Family 15 Model 36 Stepping 2 AuthenticAMD ~1994 Mhz]

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d',(0,)*
1000000)"
10 loops, best of 3: 199 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d',(0.,)*1000000)"
10 loops, best of 3: 158 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d');x.fromstring('\0'*8*1000000)"
10 loops, best of 3: 36 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "x=array
('d','\0'*8*1000000)"
10 loops, best of 3: 35.7 msec per loop

C:\junk>\python26\python -mtimeit -s"from array import array" "array
('d',(0.,))*1000000"
10 loops, best of 3: 19.5 msec per loop

HTH,
John