Mailman 3 February 2011 - NumPy-Discussion

[ANN] Invitation for Cython workshop
by Dag Sverre Seljebotn 07 Feb '11

07 Feb '11

Like the Cython project? Want to get your hands dirty with it? A great way to do so is to participate in the Cython workshop, which takes place in Munich, March 30 -- April 3. We are renting a house to live and work in, and will also give some talks at a local university, LMU Munich: http://wiki.cython.org/workshop1/OpenCythonDay Funding for travel and lodging can be arranged. Even if you don't feel up to writing any code, please consider whether you want to come and help out with our documentation sprint. It would mean a lot to the Cython project, and also teach you all the ins and outs of Cython. Also, if you are a student who plans to apply for a Google Summer of Code project with Cython, this is a great place to be. To apply for consideration, please reply with your CV and a short email telling us why you want to participate. -- Dag Sverre Seljebotn

1 0

histogram function return type
by josef.pktd＠gmail.com 06 Feb '11

06 Feb '11

there is some inconsistency in the return type of the histogram function, 1d returns int, others return float >>> np.histogramdd([y, x], bins=3)[0] array([[ 3., 15., 3.], [ 10., 2., 9.], [ 4., 0., 5.]]) >>> np.histogram2d(y, x, bins=3)[0] array([[ 3., 15., 3.], [ 10., 2., 9.], [ 4., 0., 5.]]) >>> np.histogram(y, bins=3)[0] array([21, 21, 9]) Josef

1 0

core library structure
by Mark Wiebe 04 Feb '11

04 Feb '11

Moving this to a new thread. On Thu, Feb 3, 2011 at 10:50 AM, Charles R Harris <charlesr.harris(a)gmail.com > wrote: > > On Thu, Feb 3, 2011 at 11:07 AM, Mark Wiebe <mwwiebe(a)gmail.com> wrote: > >> On Tue, Feb 1, 2011 at 10:09 PM, David <david(a)silveregg.co.jp> wrote: >> >>> On 02/02/2011 02:57 PM, Mark Wiebe wrote: >>> [snip] >>> >> >>> > I think the error handling policy >>> > in the ufuncs could also be useful for computations in the core. >>> >>> At the moment, ufunc and multiarray are separate extensions. If each >>> depends on the other API-wise, it is an issue. If there are some >>> commonalities, then they can be put in a separate extensions. They are >>> already way too big as they are (historical reasons), we should really >>> fight this for maintanability. >>> >>> I realize the code organization and the build stuff is a bit of a mess >>> at the moment, so if you have questions on that aspect, feel free to ask >>> clarification or help, >> >> >> Yeah, I understand it's the result of organic growth and merging from many >> different sources. The core library should probably become layered in a >> manner roughly as follows, with each layer depending only on the previous >> APIs. This is what Travis was getting at, I believe, with the generator >> array idea affecting mainly the Storage and Iteration APIs, generalizing >> them so that new storage and iteration methods can be plugged in. >> >> Data Type API: data type numbers, casting, byte-swapping, etc. >> Array Storage API: array creation/manipulation/deletion. >> Array Iteration API: array iterators, copying routines. >> Array Calculation API: typedefs for various types of calculation >> functions, common calculation idioms, ufunc creation API, etc. >> >> Then, the ufunc module would use the Array Calculation API to implement >> all the ufuncs and other routines like inner, dot, trace, diag, tensordot, >> einsum, etc. >> >> > I like the lower two levels if, as I assume, they are basically aimed at > allocating, deallocating blocks of memory (or equivalent) and doing basic > manipulations such as dealing with endianess and casting. Where do you see > array methods making an appearance? > That's correct. Currently, for example, the cast functions take array objects as parameters, something that would no longer be the case. The array methods vs functions only shows up in the Python exposure, I believe. The above structure only affects the C library, and how its exposed to Python could remain as it is now. The original Numeric only had three (IIRC) rather basic methods and > everything else was function based, an approach which is probably easier to > maintain. The extensive use of methods came from numarray and might be > something that could be added at a higher level so that the current ndarrays > would be objects combining ow level arrays and ufuncs. > Sounds reasonable to me. -Mark

5 5

Complex matrix seg fault
by Jonathan Tu 04 Feb '11

04 Feb '11

Hi, When I run the following code, I get a seg fault on certain cluster nodes, but not others: >>> import numpy as N >>> A = N.random.rand(262144,10)+1j*N.random.rand(262144,10) >>> x = N.random.rand(262144,) >>> d, res, rank, s = N.linalg.lstsq( A, x ) My numpy is configured as such: lapack_opt_info: libraries = ['mkl_lapack', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_def', 'mkl_mc', 'iomp5', 'pthread'] library_dirs = ['/home/jhtu/local/lib/mkl_links'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/Compiler/11.1/072/mkl/include'] blas_opt_info: libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_def', 'mkl_mc', 'iomp5', 'pthread'] library_dirs = ['/home/jhtu/local/lib/mkl_links'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/Compiler/11.1/072/mkl/include'] lapack_mkl_info: libraries = ['mkl_lapack', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_def', 'mkl_mc', 'iomp5', 'pthread'] library_dirs = ['/home/jhtu/local/lib/mkl_links'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/Compiler/11.1/072/mkl/include'] blas_mkl_info: libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_def', 'mkl_mc', 'iomp5', 'pthread'] library_dirs = ['/home/jhtu/local/lib/mkl_links'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/Compiler/11.1/072/mkl/include'] mkl_info: libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_def', 'mkl_mc', 'iomp5', 'pthread'] library_dirs = ['/home/jhtu/local/lib/mkl_links'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/Compiler/11.1/072/mkl/include'] All unit tests pass except the following, but I was told (by this mailing list) that this shouldn't be an issue: ====================================================================== FAIL: test_special_values (test_umath_complex.TestClog) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jhtu/local/lib/python2.7/site-packages/numpy/testing/decorators.py", line 215, in knownfailer return f(*args, **kwargs) File "/home/jhtu/local/lib/python2.7/site-packages/numpy/core/tests/test_umath_complex.py", line 312, in test_special_values assert_almost_equal(np.log(np.conj(xa[i])), np.conj(np.log(xa[i]))) File "/home/jhtu/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 443, in assert_almost_equal raise AssertionError(msg) AssertionError: Arrays are not almost equal ACTUAL: array([-inf+3.14159265j]) DESIRED: array([-inf-3.14159265j]) ---------------------------------------------------------------------- Ran 2648 tests in 15.852s FAILED (KNOWNFAIL=4, SKIP=1, failures=1) <nose.result.TextTestResult run=2648 errors=0 failures=1> Anyone have ideas why this would fail on some nodes but not others? (I mean the code snippet fails on some nodes. The above unit test always fails.) Thanks, Jonathan Tu

2 1

Please keep Numpy master working on Py3
by Pauli Virtanen 03 Feb '11

03 Feb '11

Hi, The master branch did not build today on Python 3. Please make sure that your code works correctly also on Python 3, before pushing it. *** I mostly fixed the stuff for now, mostly just the usual bytes vs unicode. On Python 3, the tests however give two non-obvious failures -- I'm not sure if it's just a matter of the raised exception having a different type on Py2 vs Py3, or if it reflects something going wrong. Mark, do you have ideas? Pauli ====================================================================== ERROR: test_new_iterator.test_iter_buffering_delayed_alloc ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.1/dist-packages/nose-1.0.0.dev-py3.1.egg/ nose/case.py", line 188, in runTest self.test(*self.arg) File "/home/pauli/prj/scipy/numpy2/dist/linux/lib/python3.1/site- packages/numpy/core/tests/test_new_iterator.py", line 1471, in test_iter_buffering_delayed_alloc assert_raises(ValueError, lambda i:i[0:2], i) File "/home/pauli/prj/scipy/numpy2/dist/linux/lib/python3.1/site- packages/numpy/testing/utils.py", line 983, in assert_raises return nose.tools.assert_raises(*args,**kwargs) File "/usr/lib/python3.1/unittest.py", line 589, in assertRaises callableObj(*args, **kwargs) File "/home/pauli/prj/scipy/numpy2/dist/linux/lib/python3.1/site- packages/numpy/core/tests/test_new_iterator.py", line 1471, in <lambda> assert_raises(ValueError, lambda i:i[0:2], i) TypeError: sequence index must be integer, not 'slice' ====================================================================== ERROR: test_object_array_self_reference (test_regression.TestRegression) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pauli/prj/scipy/numpy2/dist/linux/lib/python3.1/site- packages/numpy/core/tests/test_regression.py", line 1499, in test_object_array_self_reference assert_raises(ValueError, oct, a) File "/home/pauli/prj/scipy/numpy2/dist/linux/lib/python3.1/site- packages/numpy/testing/utils.py", line 983, in assert_raises return nose.tools.assert_raises(*args,**kwargs) File "/usr/lib/python3.1/unittest.py", line 589, in assertRaises callableObj(*args, **kwargs) TypeError: only integer arrays with one element can be converted to an index

4 9

Re: [Numpy-discussion] create a numpy array of images
by Asmi Shah 03 Feb '11

03 Feb '11

Thanks a lot Friedrich and Chris.. It came in handy to use PIL and numpy.. :) @Zach, m aware of the poor handling of 16bit images in PIL, for that I am using imagemagick to convert it into 8 bit first and then PIL for rest of the processing.. I have one more question: how to avoid the limitation of memoryerror in numpy. as I have like 200 images to stack in the numpy array of say 1024x1344 resolution.. have any idea apart from downsampling?

6 11

relational join
by Ilya Shlyakhter 02 Feb '11

02 Feb '11

Does numpy have a relational join operation for joining recordarrays? thanks, ilya

3 2

BUG: add.accumulate on an array of shape (0, ) causes divide-by-zero
by Michael Droettboom 02 Feb '11

02 Feb '11

Just wanted to make the developers aware of this bug that causes one of our pyfits unit tests to fail: http://projects.scipy.org/numpy/ticket/1733 Mike

1 0

Non-deterministic floating point behavior in numpy 1.5.1 ?
by Grégory Guyomarc'h 02 Feb '11

02 Feb '11

Hello, I have noticed strange non-deterministic behaviours with numpy 1.5.1 when using floating point arrays. The following script were run on 6 different machines, all Intel Core i7: - 3 of them running numpy 1.5.1 with either Python 2.7.1 (x86) or 2.5.2 (x86) and, - 3 of them running numpy 1.3 and Python 2.5.2.(x86). import numpy x = numpy.array([[0.00010876945607980702], [0.22568137594619658], [5.6435218858623557]]) for i in range(10): m = numpy.array([[36.0 * 36.0, 36.0, 1.0] for j in range(6)]) y = (numpy.dot(m, x) - 13.90901663) * 1000.0 print y[0] The output on each machine running 1.5.1 are similar to this one: [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] I cannot make sense of the changes of the least significant digits across different iterations of the for loop since its body is actually constant. Note that this behavior is hard to reproduce: on some machines I had to insert dummy print statements here and there to reproduce the bug or increase the number of iterations or the length of the arrays inside the loop. Also, I could not reproduce it with older versions of numpy such as 1.3. Is this behavior expected ? Is there a way to make sure the results of a numpy floating point computations remains the same for multiple runs ? Thanks, Gregory.

3 2

Postdoctoral Positions at Univ of Alabama
by Eric Carlson 01 Feb '11

01 Feb '11

Hello All, I apologize for using this venue, and flame me if you must, but I wish to announce two Postdoctoral positions in the Department of Chemical and Biological Engineering at the University of Alabama, in Tuscaloosa, AL, USA. The funding concerns modeling of geological sequestration of CO2. The actual activities for the project involve development of some open source software for petroleum reservoir simulation, development of vast amounts of educational documentation and software (again open source), theoretical developments - really, these positions can be almost anything. For example: 1. Efficient parallel coding 2. Theoretical formulations for reactive transport in porous media 3. Algorithm development 4. Scientific Python educational documentation 5. GUI development 6. Visualization 7. Video production 8. Give me some suggestions If interested, you probably should contact me via email (ecarlson .at. eng. ua .edu )with a very clear subject line of "Sequestration Postdoctoral Position Inquiry" and if I like what you present, I will provide the additional instructions to formally apply through out HR dept. The positions are available now, and can last until at least 12/8/2012, with a good chance for extension for a few months after that. Regards, Eric Carlson, Associate Professor University of Alabama

1 0