Mailman 3 February 2010 - NumPy-Discussion

Unpleasant behavior with poly1d and numpy scalar multiplication
by Fernando Perez Feb. 15, 2010

Feb. 15, 2010

Hi all, consider this little script: from numpy import poly1d, float, float32 p=poly1d([1.,2.]) three=float(3) three32=float32(3) print 'three*p:',three*p print 'three32*p:',three32*p print 'p*three32:',p*three32 which produces when run: In [3]: run pol1d.py three*p: 3 x + 6 three32*p: [ 3. 6.] p*three32: 3 x + 6 The fact that multiplication between poly1d objects and numbers is: - non-commutative when the numbers are numpy scalars - different for the same number if it is a python float vs a numpy scalar is rather unpleasant, and I can see this causing hard to find bugs, depending on whether your code gets a parameter that came as a python float or a numpy one. This was found today by a colleague on numpy 1.0.4.dev3937. It feels like a bug to me, do others agree? Or is it consistent with a part of the zen of numpy I've missed thus far? Thanks, f

6 17

[ANN] pyfftw-0.2 released
by Jochen Schroeder Feb. 15, 2010

Feb. 15, 2010

Hi all, I'm pleased to announce version 0.2 of pyfftw, a python module providing access to the FFTW3 library. New features: - pyfftw can now create advanced plans (needs testing) - provide location of fftw libraries at runtime with an environment variable - better detection of fftw location at install time Note: Pyfftw moved to launchpad, new url is http://launchpad.net/pyfftw. Cheers Jochen

1 0

Multithreading support
by Wolfgang Kerzendorf Feb. 15, 2010

Feb. 15, 2010

Dear all, I don't know much about parallel programming so I don't know how easy it is to do that: When doing simple arrray operations like adding two arrays or adding a number to the array, is numpy able to put this on multiple cores? I have tried it but it doesnt seem to do that. Is there a special multithread implementation of numpy. IDL has this feature where it checks how many cores available and uses them. This feature in numpy would make an already amazing package even better. Is this feature coming in numpy? Is there some sort of ETA on that? Thanks in advance Wolfgang

4 3

Remaining buildbot errors.
by Charles R Harris Feb. 14, 2010

Feb. 14, 2010

Python 2.4 ====================================================================== ERROR: test_view_to_flexible_dtype (test_core.TestMaskedView) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/numpybb/Buildbot/numpy/b13/numpy-install/lib/python2.4/site-packages/numpy/ma/tests/test_core.py", line 3333, in test_view_to_flexible_dtype test = a[0].view([('A', float), ('B', float)]) File "../numpy-install/lib/python2.4/site-packages/numpy/ma/core.py", line 2877, in view TypeError: attribute 'shape' of 'numpy.generic' objects is not writable BSD ====================================================================== FAIL: test_umath.TestComplexFunctions.test_loss_of_precision(<type 'numpy.complex64'>,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/tmp/numpy-buildbot/b12/numpy-install/lib/python2.5/site-packages/numpy/core/tests/test_umath.py", line 721, in check_loss_of_precision check(x_basic, 2*eps/1e-3) File "/tmp/numpy-buildbot/b12/numpy-install/lib/python2.5/site-packages/numpy/core/tests/test_umath.py", line 691, in check 'arcsinh') AssertionError: (0, 0.0010023052, 0.99711633, 'arcsinh') ====================================================================== FAIL: test_umath.TestComplexFunctions.test_precisions_consistent ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.5/site-packages/nose-0.10.4-py2.5.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/tmp/numpy-buildbot/b12/numpy-install/lib/python2.5/site-packages/numpy/core/tests/test_umath.py", line 602, in test_precisions_consistent assert_almost_equal(fcf, fcd, decimal=6, err_msg='fch-fcd %s'%f) File "../numpy-install/lib/python2.5/site-packages/numpy/testing/utils.py", line 435, in assert_almost_equal AssertionError: Arrays are not almost equal fch-fcd <ufunc 'arcsin'> ACTUAL: (0.66623944+0.95530742j) DESIRED: (0.66623943249251527+1.0612750619050355j)

2 1

Scalar-ndarray arguments passed to not_equal
by Friedrich Romstedt Feb. 14, 2010

Feb. 14, 2010

Hi, I'm just coding a package for uncertain arrays using the accelerated numpy functionality intensively. I'm sorry, but I have to give some background information first. The package provides a class upy.undarray, which holds the nominal value and the uncertainty information. It has methods __add__(other), __radd__(other), ..., __eq__(other), __ne__(other), which accept both upy.undarrays and all other values suitable for coercion, thus also native numpy.ndarrays. But because numpy treats in the statement: result = numpyarray * upyarray upyarray as a scalar, because it's not an numpy.ndarray, I have to overload the numpy arithmetics by own objects by using numpy.set_numeric_ops(add = ..., ..., equal = equal, not_equal = not_equal). The arguments are defined by the module (it will be clearifiied below). Because numpy.add etc. are ufuncs exhibiting attributes, I wrote a class to wrap them: class ufuncWrap: """Wraps numpy ufuncs. Behaves like the original, with the exception that __call__() will be overloaded.""" def __init__(self, ufunc, overload): """UFUNC is the ufunc to be wrapped. OVERLOAD is the name (string) of the undarray method to be used in overloading __call__().""" self.ufunc = ufunc self.overload = overload def __call__(self, a, b, *args, **kwargs): """When B is an undarray, call B.overload(a), else .ufunc(a, b).""" if isinstance(b, undarray): return getattr(b, self.overload)(a) else: return self.ufunc(a, b, *args, **kwargs) def __getattr__(self, attr): """Return getattr(.ufunc, ATTR).""" return getattr(self.ufunc, attr) I only have to wrap binary operators. Then, e.g.: class Equal(ufuncWrap): def __init__(self): ufuncWrap.__init__(self, numpy.equal, '__eq__') equal = Equal() This works as expected. But this approach fails (in first iteration) for a similar class NotEqual. I have let the module output the arguments passed to ufuncWrap.__call__(), and I found that the statement: result = (numpyarray != upyarray) with: numpyarray = numpy.asarray([1.0]) upyarray = upy.ndarray([2.0], error = [0.1]) is passed on to NotEqual.__call__() as the arguments: a = a numpy-array array([1.0]) b = a numpy-array array(shape = (), dtype = numpy.object), which is a scalar array holding the upy.ndarray instance passed to !=. I can work around the exhbited behaviour by: class NotEqual(ufuncWrap): def __init__(self): ufuncWrap.__init__(self, numpy.not_equal, '__ne__') def __call__(self, a, b, *args, **kwargs): # numpy's calling mechanism of not_equal() seems to have a bug, # such that b is always a numpy.ndarray. When b should be an undarray, # it is a numpy.ndarray(dtype = numpy.object, shape = ()) ... # Make the call also compatible with future, bug-fixed versions. if isinstance(b, numpy.ndarray): if b.ndim == 0: # Implement some conversion from scalar array to stored object. b = b.sum() return ufuncWrap.__call__(self, a, b, *args, **kwargs) What is the reason for the behaviour observed? I'm using numpy 1.4.0 with Python 2.5. Friedrich

7 26

dtype=None as default for np.genfromtxt ?
by Pierre GM Feb. 14, 2010

Feb. 14, 2010

It has been suggested (ticket #1262) to change the default dtype=float to dtype=None in np.genfromtxt. Any thoughts ?

5 8

Buildbots in red meltdown.
by Charles R Harris Feb. 14, 2010

Feb. 14, 2010

*All* the buildbots are showing errors. Here are some: ====================================================================== ERROR: test_view_to_flexible_dtype (test_core.TestMaskedView) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/numpybb/Buildbot/numpy/b13/numpy-install/lib/python2.4/site-packages/numpy/ma/tests/test_core.py", line 3333, in test_view_to_flexible_dtype test = a[0].view([('A', float), ('B', float)]) File "../numpy-install/lib/python2.4/site-packages/numpy/ma/core.py", line 2866, in view File "../numpy-install/lib/python2.4/site-packages/numpy/ma/core.py", line 2786, in __array_finalize__ TypeError: attribute 'shape' of 'numpy.generic' objects is not writable ====================================================================== ERROR: test_view_to_subdtype (test_core.TestMaskedView) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/numpybb/Buildbot/numpy/b13/numpy-install/lib/python2.4/site-packages/numpy/ma/tests/test_core.py", line 3354, in test_view_to_subdtype test = a[0].view((float, 2)) File "../numpy-install/lib/python2.4/site-packages/numpy/ma/core.py", line 2866, in view File "../numpy-install/lib/python2.4/site-packages/numpy/ma/core.py", line 2786, in __array_finalize__ TypeError: attribute 'shape' of 'numpy.generic' objects is not writable ====================================================================== FAIL: test_buffer_hashlib (test_regression.TestRegression) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/numpybb/Buildbot/numpy/b13/numpy-install/lib/python2.4/site-packages/numpy/core/tests/test_regression.py", line 1255, in test_buffer_hashlib assert_equal(md5(x).hexdigest(), '2a1dd1e1e59d0a384c26951e316cd7e6') File "../numpy-install/lib/python2.4/site-packages/numpy/testing/utils.py", line 305, in assert_equal AssertionError: Items are not equal: ACTUAL: '1264d4a9f74dc462700fd163e3ff09a6' DESIRED: '2a1dd1e1e59d0a384c26951e316cd7e6' Chuck

3 6

docstring suggestions
by Hans Meine Feb. 14, 2010

Feb. 14, 2010

Hi, I was just looking for numpy.ma.compressed, but forgot its name. I suggest to add a pointer/"see also" to numpy.ma.filled at least: http://docs.scipy.org/numpy/docs/numpy.ma.core.filled/ Unfortunately, I forgot the PW of my account (hans_meine), otherwise I'd have given it a shot. Ciao, Hans

6 21

Re: [Numpy-discussion] numpy 2.0, what else to do?
by Perry Greenfield Feb. 13, 2010

Feb. 13, 2010

On Feb 13, 2010, at 11:24 AM, Charles R Harris wrote: > Hi All, > > Since there has been talk of deprecating the numarray and numeric > compatibility Can someone be explicit about what is mean by this deprecation? > parts of numpy for the upcoming 2.0 release I thought maybe we could > consider a

1 0

A New Coercion Model
by Friedrich Romstedt Feb. 13, 2010

Feb. 13, 2010

Hi, there recently were some problems occuring when coercing numpy.ndarrays with something else. I would like to try to review the examples I've seen so far (it are two examples in only ~ one week), and also would try to summarise the approach taken up to now as far as I understand. Furthermore, I would like to start discussion about a more robust coercion model. I'm not shure whether I'm in the correct position for starting such a deep thread, but I simply hope you appreciate and excuse me for trying to do so. EXAMPLES 1. numpy.poly1d. The problem arises when coercing a numpy.poly1d instance as second operand with a numpy.ndarray (e.g., a scalar array) or a numpy scalar (like a numpy.int32 instance). It seems that numpy tries to create an array out of the poly1d instance, maybe via asarray(), and this results indeed in an ndarray. But it isn't clear to me, why. poly1d supports __len__() and __getitem__(), but one can easily check that a class supporting this is _not_ converted into an ndarray by numpy.asarray(). Also int32(2) * X([1,2]) where X is the resp. class does yield TypeError: unsupported operand type(s) for *: 'int' and 'instance'. Same applies for a class supporting __iter__(). Also same for a class supporting __iter__, __len__, __getitem__ simultaneously. instance(numpy.poly1d([1]), numpy.ndarray) returns False. 2. upy.undarray. As I reported myself, I encountered the same problem with my own code with also non-numpy.ndarray objects. The class supports also __len__() and __getitem__(), but I found that it is treated as a scalar. (Nevertheless, changing this behaviour wouldn't solve the problem.) numpy.asarray(upy.undarray(...)) returns a scalar array, opposed to the numpy.asarray(poly1d(...)) case, where it returns a corresponding ndarray. It seems to do not matter, that I never tried with numpy.int32 instances on the left hand side of, e.g., multiplication. 3. numpy.polynomial.Polynomial. I have no idea how this was solved for Polynomial. Even numpy.int32(2).__mul__(p) with p being a Polynomial instance works fine. Maybe it's a static route? Polynomial([1, 2]).__array_priority__ does not exist. CURRENT APPROACH So, this seems easy: There is __array_priority__, there may be static routes hardcoded, and there is numpy.set_numeric_ops(). Are there more ways implemented to treat the problem I'm not aware of? DISCUSSION __array_priority__ induces a linear order on all classes in the namespace. I think, I guess, this is not appropriate. At least my intuition tells me that this easily breaks down. Consider A and B having precedence over ndarray, for supporting an expression with A() as right operand. Now, C is introduced, which wants to have precedence over A, but B leaving more precedent than the new C. Now everything depends on the definition of A and B. The aim is unreachable if the __array_priority__s of A and B are by coincidence not compatible with this new elements of the relation. (It's maybe an a bit silly example, but I have no better at hand.) In fact, no one knows the precedence of something over anything else as long as there is no definition for that. There may occur even rings in the relation. Calling the relation >, although it's no longer assumed to be linear, there may hold A > B > C > A. What about implementing a simple class "Relation" in numpy. Users may register relations in the numpy.relations instance they are shure to be existent. For instance, I would say numpy.relations.set(lower = numpy.ndarray, higher = upy.undarray). Same for the numpy.poly1d thing, and it could also be used for numpy.polynomial.Polynomial. Though there are subtleties with inheritance. E.g. when class X(numpy.ndarray): [...], it should be treated as numpy.ndarray as long as there is no definition involving X directly. One could simply say, the last defined relation rules first, "LIFO". I could code a Python module for this, but it would slow numpy down. Maybe a C implementation would be helpful. I have much experience with C++ and Python both, but no experience with building numpy ... But now, I would appreciate any response in discussion. Friedrich P.S. To me it's clear that this would apply to 3.0 (or whatever) ...

1 0