Hi,
i am having troubles freezing some python code with call the Numeric 17.1.1 module. Here is an example of what happens under Linux Red Hat 6.2 and Python 2.0:
$ cat t.py import Numeric a = Numeric.ones([10,10]) b = a * 10 print b
$ python ~/Python-2.0/Tools/freeze/freeze.py -o tt t.py ... $ cd tt $ ./t Traceback (most recent call last): File "t.py", line 1, in ? File "/scisoft/python/lib/python2.0/site-packages/Numeric/Numeric.py", line 79, in ? import multiarray ImportError: /scisoft/python/lib/python2.0/site-packages/Numeric/multiarray.so: undefined symbol: _Py_NoneStruct
The very same code, under Solaris 2.6 and Python 2.0 works just fine. Code which does not use the Numeric package freeze just fine under Linux, so I think that this point to some problem/incompatibility of Numeric with freeze.py. Does anybody have a suggestion, or a work-around??
Nor
When I initialize an array, I use a = ones(shape)*initial_val
But, I am wondering if Numpy has more efficient way. For example, a = array(initial_value, shape)
Peter
Daehyok Shin wrote:
When I initialize an array, I use a = ones(shape)*initial_val
But, I am wondering if Numpy has more efficient way. For example, a = array(initial_value, shape)
I don't know if it's any more efficient (what you have is pretty fast already), but another option is to use resize:
shape = (3,4)
initial_val = 5.0
resize(initial_val,shape)
array([[ 5., 5., 5., 5.], [ 5., 5., 5., 5.], [ 5., 5., 5., 5.]])
-Chris
Comparing the performance, resize() seems to be more effient than ones().
Daehyok Shin
----- Original Message ----- From: "Chris Barker" cbarker@jps.net Cc: numpy-discussion@lists.sourceforge.net Sent: Wednesday, November 29, 2000 11:13 AM Subject: Re: [Numpy-discussion] Initialization of array?
Daehyok Shin wrote:
When I initialize an array, I use a = ones(shape)*initial_val
But, I am wondering if Numpy has more efficient way. For example, a = array(initial_value, shape)
I don't know if it's any more efficient (what you have is pretty fast already), but another option is to use resize:
shape = (3,4)
initial_val = 5.0
resize(initial_val,shape)
array([[ 5., 5., 5., 5.], [ 5., 5., 5., 5.], [ 5., 5., 5., 5.]])
-Chris
-- Christopher Barker, Ph.D. cbarker@jps.net --- --- --- http://www.jps.net/cbarker -----@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Water Resources Engineering ------ @ ------ @ ------ @ Coastal and Fluvial Hydrodynamics ------- --------- --------
Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net http://lists.sourceforge.net/mailman/listinfo/numpy-discussion
"DS" == Daehyok Shin sdhyok@email.unc.edu writes:
DS> When I initialize an array, I use a = ones(shape)*initial_val
DS> But, I am wondering if Numpy has more efficient way. For example, DS> a = array(initial_value, shape)
Looking at the definition of "ones":
def ones(shape, typecode='l', savespace=0): """ones(shape, typecode=Int, savespace=0) returns an array of the given dimensions which is initialized to all ones. """ return zeros(shape, typecode, savespace)+array(1, typecode)
It looks like you could try a=zeros(shape)+initial_val instead.
Hm.. I might do some experimenting.
Rob
Initialization on huge arrays is frequent operations in scientific programming. It must be efficient as much as possible. So, I was surprisized to see the inner codes of ones() in Numpy. It maybe use calloc() rather than malloc() in C level, then for(..) for addition. Why not use malloc() and for(...) simultaneously in C level with a command such as:
a = arrray(1,shape=(10000,10000))
Daehyok Shin
----- Original Message ----- From: "Rob W. W. Hooft" rob@hooft.net To: "Daehyok Shin" sdhyok@email.unc.edu Cc: numpy-discussion@lists.sourceforge.net Sent: Wednesday, November 29, 2000 1:00 PM Subject: Re: [Numpy-discussion] Initialization of array?
"DS" == Daehyok Shin sdhyok@email.unc.edu writes:
DS> When I initialize an array, I use a = ones(shape)*initial_val
DS> But, I am wondering if Numpy has more efficient way. For example, DS> a = array(initial_value, shape)
Looking at the definition of "ones":
def ones(shape, typecode='l', savespace=0): """ones(shape, typecode=Int, savespace=0) returns an array of the
given
dimensions which is initialized to all ones. """ return zeros(shape, typecode, savespace)+array(1, typecode)It looks like you could try a=zeros(shape)+initial_val instead.
Hm.. I might do some experimenting.
Rob
-- ===== rob@hooft.net http://www.hooft.net/people/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! =========
I had another look at the definition of "ones" and of another routine I frequently use: arange. It appears that even without rewriting them in C, some speedup can be achieved:
- in ones(), the + 1 should be done "in place", saving about 15%, more if you run out of processor cache:
amigo[167]~%3% /usr/local/bin/python test_ones.py Numeric.ones 10 -> 0.098ms Numeric.ones 100 -> 0.103ms Numeric.ones 1000 -> 0.147ms Numeric.ones 10000 -> 0.830ms Numeric.ones 100000 -> 11.900ms Numeric.zeros 10 -> 0.021ms Numeric.zeros 100 -> 0.022ms Numeric.zeros 1000 -> 0.026ms Numeric.zeros 10000 -> 0.290ms Numeric.zeros 100000 -> 4.000ms Add inplace 10 -> 0.091ms Add inplace 100 -> 0.094ms Add inplace 1000 -> 0.127ms Add inplace 10000 -> 0.690ms Add inplace 100000 -> 8.100ms Reshape 1 10 -> 0.320ms Reshape 1 100 -> 0.436ms Reshape 1 1000 -> 1.553ms Reshape 1 10000 -> 12.910ms Reshape 1 100000 -> 141.200ms
Also notice that zeros() is 4-5 times faster than ones(), so it may pay to reimplement ones in C as well (it is used in indices() and arange()).
The "resize 1" alternative is much slower.
- in arange, additional 10% can be saved by adding brackets around (start+(stop-stop)) (in addition to the gain by the faster "ones"):
amigo[168]~%3% /usr/local/bin/python test_arange.py Numeric.arange 10 -> 0.390ms Numeric.arange 100 -> 0.410ms Numeric.arange 1000 -> 0.670ms Numeric.arange 10000 -> 4.100ms Numeric.arange 100000 -> 59.000ms Optimized 10 -> 0.340ms Optimized 100 -> 0.360ms Optimized 1000 -> 0.580ms Optimized 10000 -> 3.500ms Optimized 100000 -> 48.000ms
Regards,
Rob Hooft
I had another look at the definition of "ones" and of another routine I frequently use: arange. It appears that even without rewriting them in C, some speedup can be achieved:
- in ones(), the + 1 should be done "in place", saving about 15%, more if you run out of processor cache:
I'd also try assignment in place:
def ones(shape, typecode='l', savespace=0): a = zeros(shape, typecode, savespace) a[len(shape)*[slice(0, None)]] = 1 return a
Konrad.
KH> I had another look at the definition of "ones" and of another routine KH> I frequently use: arange. It appears that even without rewriting them KH> in C, some speedup can be achieved: KH> KH> - in ones(), the + 1 should be done "in place", saving about 15%, more KH> if you run out of processor cache:
KH> I'd also try assignment in place:
KH> def ones(shape, typecode='l', savespace=0): KH> a = zeros(shape, typecode, savespace) KH> a[len(shape)*[slice(0, None)]] = 1 KH> return a
This is even faster, but it is better to write "a[...] = 1", because your manual calculation of "..." gives a large overhead for small arrays.
On another machine this time: Numeric.ones 10 -> 0.254ms Numeric.ones 100 -> 0.268ms Numeric.ones 1000 -> 0.340ms Numeric.ones 10000 -> 1.960ms Numeric.ones 100000 -> 29.300ms Numeric.zeros 10 -> 0.055ms Numeric.zeros 100 -> 0.059ms Numeric.zeros 1000 -> 0.068ms Numeric.zeros 10000 -> 0.430ms Numeric.zeros 100000 -> 9.800ms Add inplace 10 -> 0.246ms Add inplace 100 -> 0.255ms Add inplace 1000 -> 0.312ms Add inplace 10000 -> 1.270ms Add inplace 100000 -> 18.100ms Assign inplace 10 -> 0.192ms Assign inplace 100 -> 0.201ms Assign inplace 1000 -> 0.242ms Assign inplace 10000 -> 1.010ms Assign inplace 100000 -> 16.300ms Reshape 1 10 -> 0.842ms Reshape 1 100 -> 1.175ms Reshape 1 1000 -> 4.100ms Reshape 1 10000 -> 35.100ms Reshape 1 100000 -> 368.600ms
Rob
-
Chris Barker -
Daehyok Shin -
Konrad Hinsen -
Nor Pirzkal -
rob＠hooft.net