[PYTHON MATRIX-SIG] How time-consuming is array()?

[Kyle Schalm]

janko hauser writes: 
> Now I have some functions in my analysis-pack and encounter, that most
> time is spent in reading the data, although I use the netcdf-package
> of Bill Noon.
> 
> With this package I can read binary NetCDF-files, but the fields are
> returned as nested lists.
> 
> X=array((f_head.var('xp')),Float32)
> 
> Is the translation from a list to an array very timeconsuming or is
> the reading in the NetCDF-package the costly operation?
> 
> __Janko
> 

maybe these profile results are useful:

>>> import profile
>>> import nc
>>> f=nc.open('sub1.n14.95228.1706.ch4.nc', nc.NOWRITE)
>>> v=f.var('ch4')
>>> profile.run('list = v[:]')
         2 function calls in 19.180 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   16.520   16.520   19.180   19.180 profile:0(list = v[:])
        0    0.000             0.000          profile:0(profiler)
        1    2.660    2.660    2.660    2.660 python:0(2051.C.1)

>>> profile.run('b=array(list)')
         2 function calls in 6.870 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    3.670    3.670    6.870    6.870 profile:0(b=array(list))
        0    0.000             0.000          profile:0(profiler)
        1    3.200    3.200    3.200    3.200 python:0(2051.C.5)

>>> profile.run('a = v.get()')
         2 function calls in 1.890 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    1.560    1.560    1.890    1.890 profile:0(a = v.get())
        0    0.000             0.000          profile:0(profiler)
        1    0.330    0.330    0.330    0.330 python:0(2051.C.3)


so in this example, using the list -> array method takes 2.660 + 3.200
= 5.860 seconds, whereas the mysterious "v.get()" takes 0.330 seconds,
under 6% of the time for the other method (although i wouldn't be
surprised if the profiler overhead or variable system load affected
the results a little).

the "v.get()" comes from a modification i made to bill noon's module.
you can grab it temporarily at http://www.geog.ubc.ca/~kschalm/
if you think it might be useful. it is not very well tested but
you are welcome to use it/play around with it. there are four files
available there:

    ncmodule-demo.py       29-Oct-96 05:02     1K  
    ncmodule.c             29-Oct-96 05:01    48K  
    netcdf-demo.py         29-Oct-96 05:23     1K  
    netcdf.py              29-Oct-96 05:11     1K  

ncmodule-demo.py - illustrates the additional functions i have added
	to the original module.
ncmodule.c - is the modified module, basically it adds a get and put
	method for getting/writing arrays, but there's some frills too.
netcdf-demo.py - is a "demo prototype" of the netcdf.py module
netcdf.py - adds some convenience functions on top of ncmodule

-Kyle

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================