Mailman 3 December 2007 - NumPy-Discussion

Towards using scons in numpy: 1st step
by David Cournapeau 04 Jan '08

04 Jan '08

Hi, as discussed with some other numpy developers, in particular Travis, I started to prepare my work related to scons for step-by-step merging into the trunk. The first step is done, and is in cleanconfig_rtm branch (rtm for ready to merge). This branch basically: - avoid introducing new SIZEOF_* like symbols, which are usually used by autotools. Python itself defines some of those, but should not. - avoid #ifdef/#endif in config.h files: normally, configuration headers should only contains define. This also makes their generation more straightforward, and more compatible with standard build tools (autotools/scons). This is the main goal of the branch - split configuration defines into two files: one private (config.h), which is not visible by any other packages compiled against numpy headers, and one public (numpyconfig.h), which replaces the config.h. If there is no problem, I would like to merge this one as quickly as possible: since it changes numpy.core public header, it may potentially break things, and I would prefer detecting those breakages now. cheers, David

2 3

[OT] Which version of svn does svn.scipy.org run ?
by David Cournapeau 04 Jan '08

04 Jan '08

Hi, Is it possible to know which version of subversion is used by the svn.scipy.org server ? I am trying to use svnsync on it, without any success, and wanted to know if this was coming from svn.scipy.org or from something else ? cheers, David

2 2

random access iterators
by Neal Becker 30 Dec '07

30 Dec '07

I'm looking at writing some c++ code to interoperate with numpy. A c++ random access iterator must include a 'distance' function. distance (i1, i2) must return the number of times i1 has to be incremented to reach i2. Is there a way to get this from PyArrayIterObject?

2 1

how do I list all combinations
by Mathew Yeates 30 Dec '07

30 Dec '07

Hi I've been looking at "fromfunction" and itertools but I'm flummoxed. I have an arbitrary number of lists. I want to form all possible combinations from all lists. So if r1=["dog","cat"] r2=[1,2] I want to return [["dog",1],["dog",2],["cat",1],["cat",2]] It's obvious when the number of lists is not arbitrary. But what if thats not known until runtime? Mathew

8 12

iterator speed
by Neal Becker 30 Dec '07

30 Dec '07

In numpybook, in discussing iterators, it says that if you know your array is contiguous, it may be faster to just use C-style pointers instead of iterators. Looking at the code a little, it appears that PyArray_ITER_NEXT(it) attempts to optimize for certain cases. I wonder if anyone has done any benchmarks on modern compilers? In particular, I'm hoping that PyArray_ITER_NEXT on a modern compiler will optimize the contiguous case to give equivalent performance to a c-style pointer. In that case, I could happily use the generic iterator without a speed penalty.

1 0

Doc-day
by Travis E. Oliphant 30 Dec '07

30 Dec '07

Doc-day will start tomorrow (in about 12 hours). It will be Friday for much of America and be moving into Saturday for Europe and Asia. Join in on the irc.freenode.net (channel scipy) to coordinate effort. I imaging people will be in an out. I plan on being available in IRC from about 9:30 am CST to 6:00 pm CST and then possibly later. If you are available at different times in different parts of the world, jump in and pick something to work on. Jarrod and/or I will put out a list of priorities in the next few hours. -Travis O.

8 16

Passing a Null pointer after ndarray prototype definition
by Nicholas 30 Dec '07

30 Dec '07

I have a problem in that I have a DLL function which accepts several numpy double arrays, but you need to pass a NULL pointer for the ones from which you do not require results. The prototype is as follows: T_ndDp = numpy.ctypeslib.ndpointer(dtype=float, ndim=1, flags='C_CONTIGUOUS') Prototype_example = CFUNCTYPE(None, c_long, T_ndDp, T_ndDp, T_ndDp) In ctypes None is equivelent to NULL, though in order to get past the prototype argchecks you need to cast the object to None. If for example the pointer was to a simple double one could write something like: cast(None, POINTER(c_double)) the problem is in casting to None for the above (T_ndDp) type Any help would be greatly appreciated Nicholas

1 0

numpy installed but can' use
by dikshie 29 Dec '07

29 Dec '07

Hi, successfully installed numpy but i cant use it (numpy is not defined). for example: Python 2.5.1 (r251:54863, Nov 25 2007, 02:18:29) [GCC 4.2.1 20070719 [FreeBSD]] on freebsd7 Type "help", "copyright", "credits" or "license" for more information. >>> from numpy import * >>> numpy.__version__ Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'numpy' is not defined >>> any hints ? regards, -- -dikshie-

5 6

numpy and math sqrt timings
by Bruce Sherwood 29 Dec '07

29 Dec '07

On the VPython list Scott Daniels suggested using try/except to deal with the problem of sqrt(5.5) being numpy.float64 and thereby making sqrt(5.5)*(VPython vector) not a (VPython vector), which ends up as a big performance hit on existing programs. I tried his suggestion and did some timing using the program shown below. Using "from numpy import *", the numpy sqrt(5.5) gives 5.7 microsec per sqrt, whereas using "from math import *" a sqrt is only 0.8 microsec. Why is numpy so much slower than math on this simple case? For completeness I also timed the old Numeric sqrt, which was 14 microsec, so numpy is a big improvement, but still very slow compared to math. Using Daniels's suggestion of first trying the math sqrt, falling through to the numpy sqrt only if the argument isn't a simple scalar, gives 1.3 microsec per sqrt on the simple case of a scalar argument. Shouldn't/couldn't numpy do something like this internally? Bruce Sherwood ---------------------------- from math import * mathsqrt = sqrt from numpy import * numpysqrt = sqrt from time import clock # 0.8 microsec for "raw" math sqrt # 5.7 microsec for "raw" numpy sqrt # 1.3 microsec if we try math sqrt first def sqrt(x): try: return mathsqrt(x) except TypeError: return numpysqrt(x) # Check that numpy sqrt is invoked on an array: nums = array([1,2,3]) print sqrt(nums) x = 5.5 N = 500000 t1 = clock() for n in range(N): y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) y = sqrt(x) t2 = clock() for n in range(N): pass t3 = clock() # t3-t2 is the loop overhead (turns out negligible) print "%i loops over 10 sqrt's takes %.1f seconds" % (N,t2-t1) print "Total loop overhead = %.2f seconds (negligible)" % (t3-t2) print "One sqrt takes %.1f microseconds" % (1e6*((t2-t1)-(t3-t2))/(10*N))

2 1

Re: [Numpy-discussion] [C++-sig] Overloading sqrt(5.5)*myvector
by Bruce Sherwood 29 Dec '07

29 Dec '07

Thanks for the suggestion. It hadn't occurred to me to try to override numpy as you suggest. However, when I try the code shown below as the start of a test of this scheme, I get the following error: Traceback (most recent call last): File "C:\Documents and Settings\Bruce\My Documents\0VPythonWork\vectors.py", line 24, in <module> numpy.float64.__mul__ = new_mul TypeError: can't set attributes of built-in/extension type 'numpy.float64' I'm copying this to the numpy discussion list, as maybe someone there will see where to go starting from your suggestion. Bruce Sherwood --------------- import numpy class vector(object): def __init__(self,x,y,z): self.data = [x,y,z] def __mul__(self,other): return vector(other*self.data[0], other*self.data[1], other*self.data[2]) __rmul__ = __mul__ def show(self): print self.data old_mul = numpy.float64.__mul__ def new_mul( self, other ): if isinstance( other, vector ): return other*self else: return old_mul( self, other ) numpy.float64.__mul__ = new_mul a = vector(1,2,3) a.show() b = 5*a b.show() c = a*7 c.show() Roman Yakovenko wrote: > On Dec 26, 2007 8:11 AM, Bruce Sherwood <Bruce_Sherwood(a)ncsu.edu> wrote: > >> Sorry to repeat myself and be insistent, but could someone please at >> least comment on whether I'm doing anything obviously wrong, even if you >> don't immediately have a solution to my serious problem? There was no >> response to my question (see copy below) which I sent to both the numpy >> and Boost mailing lists. >> >> To the Boost experts: Is there something wrong, or something I >> could/should change in how I'm trying to define to Boost the overloaded >> multiplication of a numpy square root (or other numpy function) times my >> own "vector" object? I'm seeing a huge performance hit in going from >> Numeric to numpy because Numeric sqrt returned float whereas numpy sqrt >> returns numpy.float64, so that the result is not one of my vector >> objects. I don't have a problem with myvector*sqrt(5.5). Here is what I >> currently am doing: >> >> py::class_<vector>("vector", py::init< py::optional<double, double, >> double> >()) >> .def( self * double()) >> .def( double() * self) >> >> Desperately, >> > > If I understand you right, than you can do something like this: > > replace __mul__ method of numpy.float64 class: > > old_mul = numpy.float64.__mul__ > > def new_mul( self, other ): > if other isinstance( vector ): > return other*self > else: > return old_mul( self, other ) > > numpy.float64.__mul__ = new_mul > > HTH > >

3 5

NumPy-Discussion December 2007