[Numpy-discussion] copying ctypes arrays to numarray?
rays
rays at san.rr.com
Thu Dec 16 13:28:01 EST 2004
I'm re-posting this here to see if anyone can shed light on numarray.info()
behavior, and why assignment from a (ctypes.c_long * 2000)() to a numarray
is so much slower than a memmove().
I was surprised that assignment wasn't faster and that numarray assignment
was consistently ~.5% faster than Numeric.
The N vs. n memmove() flip-flopped for fastest, with array.array always slower.
After some discussion, I made a better direct comparison of ctypes and
memmove, map etc., using all the methods suggested so far. I think it's
fairly valid.(?) Run on a 2.4GHz WinXP, Py2.3, a number of times.
Parsing numarray.info() was a royal pain, it writes directly to stdout!
I.e.:
>>> inf = src.info()
class: <class 'numarray.numarraycore.NumArray'>
shape: (32,)
strides: (8,)
byteoffset: 0
bytestride: 8
itemsize: 8
aligned: 1
contiguous: 1
data: <memory at 00815950 with size:256 held by object 00815930 aliasing object
00000000>
byteorder: little
byteswap: 0
type: Float64
>>> inf
>>> type(inf)
<type 'NoneType'>
Since the ctype does support slicing, I was considering leaving the data in
the device driver buffer (~1MB circular) and poking into it, but memmove is
so much faster than slicing the ctype, I'm doing memmove()s to numarray.
I presume that I should check for numarray.iscontiguous( ) or is_c_array(
) first to be safe...
Results and code below.
Thanks to all for the help,
Ray
>python test.py
Array address 11443208
n address 17039424
N address 21102656
for loop 2027.0 us ea
map 1781.0 us ea
slice 1704.0 us ea
assign N 1244.0 us ea
assign n 1242.0 us ea
memmove 4.3831 us ea
memmove N 3.4773 us ea
memmove n 3.4803 us ea
_____________________________________________________
## test.py
import array
import numarray
import ctypes
import time
import string
import StringIO
import sys
buf = (ctypes.c_long * 2000)()
Array = array.array("l", [0]*10000)
n = numarray.zeros((1000000), numarray.Int32)
N = numarray.zeros((1000000), numarray.Int32)
#!!!!!!!!!!!!!!! arrrrgggg! !!!!!!!!!!!!!!!!
# n.info() writes directly to stdout!
stdout = sys.stdout
fileo = StringIO.StringIO()
sys.stdout = fileo
n.info()
ninfo = fileo.getvalue( )
fileo.close()
sys.stdout = stdout
stdout = sys.stdout
fileo = StringIO.StringIO()
sys.stdout = fileo
N.info()
Ninfo = fileo.getvalue( )
fileo.close()
sys.stdout = stdout
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
print 'Array address', Array.buffer_info()[0]
ninfo = string.split(ninfo)
nAddress = int(ninfo[20], 16)
print 'n address', nAddress
Ninfo = string.split(Ninfo)
NAddress = int(Ninfo[20], 16)
print 'N address', NAddress
t0 = time.clock()
for loop in range(1000):
for i in range(2000):
n[loop+i] = buf[i]
print 'for loop ', round((time.clock()-t0)*1000), 'us ea'
t0 = time.clock()
for loop in range(1000):
n[loop:loop+2000] = map(None, buf)
print 'map ', round((time.clock()-t0)*1000), 'us ea'
t0 = time.clock()
for loop in range(1000):
n[loop:loop+2000] = buf[0:2000]
print 'slice ', round((time.clock()-t0)*1000), 'us ea'
t0 = time.clock()
for loop in range(10000):
N[loop:loop+2000] = buf
print 'assign N', round((time.clock()-t0)*100), 'us ea'
t0 = time.clock()
for loop in range(10000):
n[loop:loop+2000] = buf
print 'assign n', round((time.clock()-t0)*100), 'us ea'
t0 = time.clock()
for loop in range(10000):
ctypes.memmove(10+Array.buffer_info()[0],
buf,
2000)
print 'memmove ', round((time.clock()-t0)*1, 4), 'us ea'
t0 = time.clock()
for loop in range(10000):
ctypes.memmove(10+NAddress,
buf,
2000)
print 'memmove N', round((time.clock()-t0)*1, 4), 'us ea'
t0 = time.clock()
for loop in range(10000):
ctypes.memmove(10+nAddress,
buf,
2000)
print 'memmove n', round((time.clock()-t0)*1, 4), 'us ea'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20041216/c17af6b4/attachment.html>
More information about the NumPy-Discussion
mailing list