Mailman 3 October 2010 - NumPy-Discussion

known failure test decorator not treated as expected by nose
by Jonathan March Oct. 10, 2010

Oct. 10, 2010

It appears that the numpy testing decorators for skipping and for known failure should behave similarly to each other, at least from their descriptions here: http://projects.scipy.org/numpy/wiki/TestingGuidelines#known-failures-skipp… Yet in the following example, run under nose, they behave quite differently: ==== from numpy.testing import dec @dec.knownfailureif( True, "known failure test message") def test_will_fail(): pass @dec.skipif( True, "skip test message") def test_to_skip(): pass ==== The skipped test is marked as "S" as expected, but when the KnownFailureTest exception is raised, nose handles it as an error (marked "E" rather than "K" as expected, and printing the stack trace.) It's also interesting that nose has a --no-skip option but no corresponding option for ignoring known failure decorators. Why the discrepancy? Who is out of step with whom? This is with numpy 1.4.0 and nosetests 0.11.3 Thanks, Jonathan March

3 4

datarray repositories have diverged
by Rob Speer Oct. 9, 2010

Oct. 9, 2010

There's some DataArray code that I've had for a while, but I just finished it up and tested it today. Most notably, it includes a new __str__ for DataArrays that does some nice layout and includes ticks: In [9]: print d_arr country year --------- ------------------------------------------------- 1994 1998 2002 2006 2010 Netherlan -0.505758 0.096597 1.083148 -0.450156 0.172754 Uruguay 1.772182 -0.113394 -0.781307 1.002416 -0.64925 Germany -2.013874 0.283947 1.170848 -0.504823 0.448497 Spain -0.725844 0.909713 -1.191371 -0.465167 -1.518764 (The layout functions in datarray.print_grid actually work with any ndarray, so you can use it as an alternative to the __str__ in NumPy.) However, I notice that all the new development on datarray is happening on Fernando Perez's branch, which mine diverged from long ago. I forked from Lluis (jesusabdullah)'s branch, which was the most active at the time, and I got all but the most recent changes merged back in. But that branch in turn was never merged back into fperez's. The divergence point is even before I added the syntax for named axes and ticks by name (such as arr.named['Spain', 1994:2010] or arr.year.named[2010]). Does it make sense to merge my branch into the main one? http://github.com/rspeer/datarray -- Rob

4 11

Schedule for 1.5.1?
by Pauli Virtanen Oct. 9, 2010

Oct. 9, 2010

Hi, Should we set a date for a bugfix 1.5.1 release? There are some bugs that would be nice to sort out in the 1.5.x series: Any Python versions: - #1605 (Cython vs. PEP-3118 issue: raising exceptions with active cython buffers caused undefined behavior. Breaks Sage.) - #1617 (Ensure complex(np.longcomplex(...)) doesn't drop the imag part) - Fix doc build Python 3 specific: - #1604 (//-issue in distutils; infinite loop in some cases :) - #1609 (dotblas was never used on Python 3) - #1610 (fromfile/tofile did not stay in sync with Python 3 file handle position -- breaks e.g. scipy.io pretty badly) - f2py startup script didn't run properly on Py3 Not so many fixes so far, but I'd like to see a Numpy release with #1610 fixed before releasing Scipy with Python 3 support. Since 2.0.0 breaks binary compatibility and merging the refactoring back may take some time, it would be nice to have another 1.5.x release. Ralf & c, opinions? I'd maybe suggest somewhere on the Oct/Nov axis. Pauli

8 24

Meshgrid with Huge Arrays
by sicre Oct. 8, 2010

Oct. 8, 2010

I do not have good programming skills, I am trying to create a 2048x2048 3D pixel array where each pixel has 100 cells. I am using meshgrid in order to create a 3D array, then adding a gaussian function to the entire array centered in (1024,1024). I am having three types of errors: #FIRST ERROR: File "/home/sicre/PHASES/Examples/test.py", line 18, in <module> XArray, YArray = np.meshgrid(XArray, YArray) File "/usr/lib/python2.6/dist-packages/numpy/lib/function_base.py", line 2931, in meshgrid X = x.repeat(numRows, axis=0) ValueError: dimensions too large. #SECOND ERROR (choosing instead NrCellsPerPixel=36) Traceback (most recent call last): File "/home/sicre/PHASES/Examples/test.py", line 19, in <module> Z = np.zeros([len(XArray),len(YArray)]) MemoryError "from pylab import * import numpy as np #VARIABLES NrHorPixels=2048 NrVerPixels=2048 NrCellsPerPixel=100 GaussianCenterX=1024 GaussianCenterY=1024 SigmaX=1 SigmaY=1 Amplitude = 150 #3D ARRAY XArray = np.arange(0, NrHorPixels, 1./sqrt(NrCellsPerPixel)) YArray = np.arange(0, NrVerPixels, 1./sqrt(NrCellsPerPixel)) XArray, YArray = np.meshgrid(XArray, YArray) Z = np.zeros([len(XArray),len(YArray)]) #Z = np.zeros((len(XArray),len(YArray)),dtype=int16) #Add Gaussian to Array Z = Z+Amplitude*e**-(((XArray-GaussianCenterX)**2/(2*SigmaX**2))+((YArray-GaussianCenterY)**/(2*SigmaY**2))) #Z = rand(len(XArray),len(YArray)) #Plot #pcolormesh(Z) #colorbar()" For sure there are better solutions for what i am trying to do. Can anyone figure it out? I would appreciate it very much, i've been searching a solution/answer for days. -- View this message in context: http://old.nabble.com/Meshgrid-with-Huge-Arrays-tp29902859p29902859.html Sent from the Numpy-discussion mailing list archive at Nabble.com.

11 27

Help building numpy
by Ian Goodfellow Oct. 8, 2010

Oct. 8, 2010

Can anyone explain how to get numpy to recognize atlas? I have atlas built and installed, and I put what I thought should be sufficient, based on numpy's INSTALL.TXT, to make numpy use atlas in my site.cfg then ran python setup.py install. The resulting build is incredibly slow. Multiplying a 1,000 x 1,000 matrix takes 8 seconds. The default packages take .8 . On a computer at work with a similar but slower cpu to mine, it only takes .2, so I know I should be able to get a lot more speed out of numpy. This is my site.cfg: [atlas] atlas_libs = lapack, f77blas, cblas, atlas [DEFAULT] library_dirs = /usr/local/atlas/lib include_dir = /usr/local/atlas/include Thanks in advance, Ian

3 8

DEV_README.txt update, SVN -> git
by Peter Oct. 8, 2010

Oct. 8, 2010

Just FYI: The DEV_README.txt file needs a trivial update to talk about git not SVN http://github.com/numpy/numpy/blob/master/DEV_README.txt Regards, Peter

2 1

Re: [Numpy-discussion] Schedule for 1.5.1?
by Nils Becker Oct. 8, 2010

Oct. 8, 2010

Hi, what about the normed=True bug in numpy.histogram? It was discussed here a while ago, and fixed (although i did not find it on the tracker), but the message <AANLkTikWBSRxq0YnF3u3JO3ekriKszmWQy30pnSFG8Dr(a)mail.gmail.com> suggests it just missed 1.5.0? I don't have 1.5 installed, so I can't check right now. sorry to mention my personal pet bug, but maybe it's also a candidate? Nils

2 1

ufunc.accumulate question
by Chris Withers Oct. 8, 2010

Oct. 8, 2010

Hi All, I can't find any docs on this behavior. So, I have a python function. To keep it simple, lets just do addition: def add(x,y): print x,y retun x+y So, I can turn this into a ufunc as follows: uadd = np.frompyfunc(add,2,1) Now, I can apply it to an array: >>> uadd.accumulate(np.arange(3,10)) 3 4 7 5 12 6 18 7 25 8 33 9 array([3, 7, 12, 18, 25, 33, 42], dtype=object) Okay, but where did the initial 3 come from? http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… suggests that: r = np.empty(len(A)) t = op.identity # op = the ufunc being applied to A's elements for i in xrange(len(A)): t = op(t, A[i]) r[i] = t return r ...but: >>> print uadd.identity None ...and: >>> add(None,3) None 3 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in add TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' So, where is the reason that the 3 ends up in the output array documented? Also, what if I want to specify the identity of my newly created ufunc? I have a case where I want to specify it as zero: >>> uadd.identity = 0 Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: attribute 'identity' of 'numpy.ufunc' objects is not writable Any help gratefully received! Chris

2 2

Re: [Numpy-discussion] ndarray of object dtype
by Ioan Ferencik Oct. 8, 2010

Oct. 8, 2010

thanks, it was actually obvious and I figured it out minutes later I sent the message. Thanks anyway. Quoting Robert Kern <robert.kern(a)gmail.com>: > On Thu, Oct 7, 2010 at 10:09, Ioan Ferencik <ioan.ferencik(a)tkk.fi> wrote: >> Hello Robert, >> >> I dare to bother you again with some questions. >> >> this time I have a numpy array with fields >> ar = array([(1.0, 2.0, 3, 4), (2.0, 3.0, 4, 5)], dtype={'names': >> ['q','wl','cssid','br'], 'formats':['f4', 'f4', 'i4', 'i4'], >> 'offsets': [0, 4, 8, 12]}, order='F') >> >> on C API I want to match this struct >> typedef struct val1{ >> float q; >> float wl; >> int cssid; >> int br; >> }hm1dval1; >> >> to an element of this array >> I get the array object, i can see the fields. The type of the array >> elements is Pyarray_VOID >> >> however following code gives me a segfault >> >> hm1dval1 **s = PyArray_DATA(ao); > > Since this is *not* an object array but an array of hm1dval1 structs, > you declare it like so: > > hm1dval1 *s = PyArray_DATA(ao); > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma that is made terrible by our own mad attempt to interpret it as > though it had an underlying truth." > -- Umberto Eco > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > Ioan Ferencik PhD student Aalto University School of Science and Technology Faculty Of Civil and Env. Engineering Lahti Center Tel: +358505122707

1 0

test failure
by John Mitchell Oct. 8, 2010

Oct. 8, 2010

After spending a lot of time building 'numpy' (1.5.0) and 'scipy' (0.8.0) I ran the following tests -- per what I read somewhere: numpy.test() Out of several thousand small tests I found this one error -- perhaps its not really an error? This doesn't look serious but it can be bothersome when one is trying to be sure that the installation is as good as possible. Not sure if it matters, but my compiler is gcc-4.4.1 and I'm using gfortran. Both were wrapped with mpich2. Regards, John FAIL: test_doctests (test_polynomial.TestDocs) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 90, in test_doctests return rundocs() File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/testing/utils.py", line 962, in rundocs raise AssertionError("Some doctests failed:\n%s" % "\n".join(msg)) AssertionError: Some doctests failed: ********************************************************************** File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 38, in test_polynomial Failed example: p / q Expected: (poly1d([ 0.33333333]), poly1d([ 1.33333333, 2.66666667])) Got: (poly1d([ 0.333]), poly1d([ 1.333, 2.667])) ********************************************************************** File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 60, in test_polynomial Failed example: p.integ() Expected: poly1d([ 0.33333333, 1. , 3. , 0. ]) Got: poly1d([ 0.333, 1. , 3. , 0. ]) ********************************************************************** File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 62, in test_polynomial Failed example: p.integ(1) Expected: poly1d([ 0.33333333, 1. , 3. , 0. ]) Got: poly1d([ 0.333, 1. , 3. , 0. ]) ********************************************************************** File "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 64, in test_polynomial Failed example: p.integ(5) Expected: poly1d([ 0.00039683, 0.00277778, 0.025 , 0. , 0. , 0. , 0. , 0. ]) Got: poly1d([ 0. , 0.003, 0.025, 0. , 0. , 0. , 0. , 0. ]) >> raise AssertionError("Some doctests failed:\n%s" % "\n".join(['**********************************************************************\nFile "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 38, in test_polynomial\nFailed example:\n p / q\nExpected:\n (poly1d([ 0.33333333]), poly1d([ 1.33333333, 2.66666667]))\nGot:\n (poly1d([ 0.333]), poly1d([ 1.333, 2.667]))\n', '**********************************************************************\nFile "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 60, in test_polynomial\nFailed example:\n p.integ()\nExpected:\n poly1d([ 0.33333333, 1. , 3. , 0. ])\nGot:\n poly1d([ 0.333, 1. , 3. , 0. ])\n', '**********************************************************************\nFile "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 62, in test_polynomial\nFailed example:\n p.integ(1)\nExpected:\n poly1d([ 0.33333333, 1. , 3. , 0. ])\nGot:\n poly1d([ 0.333, 1. , 3. , 0. ])\n', '**********************************************************************\nFile "/usr/local/python2.6.6/lib/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 64, in test_polynomial\nFailed example:\n p.integ(5)\nExpected:\n poly1d([ 0.00039683, 0.00277778, 0.025 , 0. , 0. ,\n 0. , 0. , 0. ])\nGot:\n poly1d([ 0. , 0.003, 0.025, 0. , 0. , 0. , 0. , 0. ])\n'])) ---------------------------------------------------------------------- Ran 2966 tests in 9.896s FAILED (KNOWNFAIL=4, failures=1) <nose.result.TextTestResult run=2966 errors=0 failures=1>

2 2