Mailman 3 August 2012 - NumPy-Discussion

Re: [Numpy-discussion] building numpy 1.6.2 on OSX 10.6 / Python2.7.3
by Andrew Nelson Aug. 8, 2012

Aug. 8, 2012

Dear Pierre, as indicated yesterday OSX system python is in: /System/Library/Frameworks/Python.framework/ I am installing into: /Library/Frameworks/Python.framework/Versions/Current/lib/python2.7/site-packages This should not present a problem and does not explain why numpy does not build/import correctly on my setup. regards, Andrew. > Date: Tue, 7 Aug 2012 14:00:58 +0200 > From: Pierre GM <pgmdevlist(a)gmail.com> > Subject: Re: [Numpy-discussion] building numpy 1.6.2 on OSX 10.6 / > Python2.7.3 > > Andrew, > > I'm afraid you did. > > It's generally considered a "very bad idea"(?) to install NumPy on a recent > OSX system without specifying a destination. By default, the process will > try to install on /Library/Frameworks/Python, overwriting the pre-installed > version of NumPy that comes with your machine. You probably don't want to > do that. > > However, using either the --user flag or a virtual environment ( > http://www.virtualenv.org/ <http://www.virtualenv.org/en/latest/>) works > pretty well. EG > > `python setup.py install --user` should install bumpy in a ~/.local > directory, you'll just have to update your PYTHONPATH > > Good luck > > -- > > Pierre GM > > On Tuesday, August 7, 2012 at 08:15 , Andrew Nelson wrote: > > Dear list, > > I am trying to build numpy 1.6.2 from source but am running up against a > few problems. > > Platform: OSX10.6.8 > > Python: 2.7.3 (compiled using gcc 4.2.1) > > gcc: 4.2.1 > > gfortran: 4.2.1 > > I try the normal build sequence: > > python setup.py build > > sudo python setup.py install > > However, when I try to import numpy I get: > > >>> import numpy > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > File > > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/__init__.py", > line 137, in <module> > > import add_newdocs > > File > > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/add_newdocs.py", > line 9, in <module> > > from numpy.lib import add_newdoc > > File > > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/__init__.py", > line 4, in <module> > > from type_check import * > > File > > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/type_check.py", > line 8, in <module> > > import numpy.core.numeric as _nx > > File > > "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/__init__.py", > line 5, in <module> > > import multiarray > > ImportError: > > dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so, > 2): Symbol not found: _npy_ceil > > Referenced from: > > /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so > > Expected in: flat namespace > > in > > /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so > > The numpy source was from the Sourceforge official page. > > When I run nm on the multiarray module I get: > > %nm > /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/multiarray.so > > .......<snip a lot of stuff> > > U _npy_ceil > > U _npy_double_to_half > > U _npy_doublebits_to_halfbits > > U _npy_float_to_half > > U _npy_floatbits_to_halfbits > > U _npy_half_isnan > > U _npy_half_iszero > > U _npy_half_le > > U _npy_half_lt_nonan > > U _npy_half_to_double > > U _npy_half_to_float > > U _npy_halfbits_to_doublebits > > U _npy_halfbits_to_floatbits > > So it seems that the _npy_ceil symbol is undefined. I looked at > /build/src.macosx-10.6-intel-2.7/numpy/core/include/numpy/config.h and it > contains: > > #define HAVE_CEIL > > Am I doing something wrong? > > regards, > > Andrew > > -- > _____________________________________ > Dr. Andrew Nelson > > _____________________________________ > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

2 1

Indexing API
by Nathaniel Smith Aug. 7, 2012

Aug. 7, 2012

So the underlying problem with the controversial inplace_increment PR[1] is that currently, there's actually nothing in the public numpy API that exposes the workings of numpy indexing. The only thing you can do with a numpy index is call ndarray.__getattr__ or __setattr__. This is a pretty obvious gap, given how fundamental an operation indexing is in numpy (and how difficult to emulate). So how can we expose something that fixes it? Make PyArrayMapIterObject part of the public API? Something else? -n [1] https://github.com/numpy/numpy/pull/326

2 2

question about scipy superpack
by Tom Krauss Aug. 7, 2012

Aug. 7, 2012

Hi, I got a new job, and a new mac book pro on which I just installed Mac OS X 10.8. I need to run SWIG to generate a shared object from C++ source that works with numpy.i. I'm considering installing the Scipy Superpack, but I have a question. If I install the Scipy Superpack, which has most of the packages I need, plus some others, will it be able to find "numpy/arrayobject.h" or do I need to install numpy source and build it myself? In other words, does "numpy-1.8.0.dev_f2f0ac0_20120725-py2.7-macosx-10.8-x86_64.egg" have the source files needed by gcc to compile the swig-generated C++ wrapper? Cheers, Tom

4 5

Moving away from using accelerate framework on mac os x ?
by David Cournapeau Aug. 5, 2012

Aug. 5, 2012

Hi, During last PyCon, Olivier Grisel (from scikits-learn fame) and myself looked into a nasty bug on mac os x: https://gist.github.com/2027412. The short story is that I believe this means numpy cannot be used with multiprocessing if linked against accelerate framework, and as such we should think about giving up on accelerate, and use e.g. ATLAS on mac for our official binaries. Long story: we recently received a answer where the engineers mention that using blas on each 'side' of a fork is not supported. The meat of the email is attached below thoughts ? David ---------- Forwarded message ---------- From: <devbugs(a)apple.com> Date: 2012/8/2 Subject: Bug ID 11036478: Segfault when calling dgemm with Accelerate / GCD after in a forked process To: olivier.grisel(a)gmail.com Hi Olivier, Thank you for contacting us regarding Bug ID# 11036478. Thank you for filing this bug report. This usage of fork() is not supported on our platform. For API outside of POSIX, including GCD and technologies like Accelerate, we do not support usage on both sides of a fork(). For this reason among others, use of fork() without exec is discouraged in general in processes that use layers above POSIX. We recommend that you either restrict usage of blas to the parent or the child process but not both, or that you switch to using GCD or pthreads rather than forking to create parallelism.

4 5

Status of NumPy and Python 3.3
by Travis Oliphant Aug. 4, 2012

Aug. 4, 2012

Hey all, I'm wondering who has tried to make NumPy work with Python 3.3. The Unicode handling was significantly improved in Python 3.3 and the array-scalar code (which assumed a certain structure for UnicodeObjects) is not working now. It would be nice to get 1.7.0 working with Python 3.3 if possible before the release. Anyone interested in tackling that little challenge? If someone has already tried it would be nice to hear your experience. -Travis

8 37

Unicode revisited
by Travis Oliphant Aug. 4, 2012

Aug. 4, 2012

Hey all, Ondrej has been working hard with feedback from many others on improving Unicode support in NumPy (especially for Python 3.3). Looking at what Python has done in Python 3.3 (PEP 393) and chatting on the Python issue tracker with the author of that PEP has made me wonder if we aren't "doing the wrong thing" in NumPy quite often. Basically, NumPy only supports UTF-32 in it's Unicode representation. All bytes in NumPy arrays should be either UTF-32LE or UTF-32BE. This is all pretty easy to understand as long as you stick with NumPy arrays only. The difficulty starts when you start to interact with the unicode array scalar (which is the same data-structure exactly as a Python unicode object with a different type-name --- numpy.unicode_). However, I overlooked the "encoding" argument to the standard "unicode" constructor which might have simplified what we are doing. If I understand things correctly, now, all we need to do is to "decode" the UTF-32LE or UTF-32BE raw bytes in the array (depending on the dtype) into a unicode object. This is easily accomplished with numpy.unicode_(<bytes object>, 'utf_32_be' or 'utf_32_le'). There is also an "encoding" equivalent to go from the Python unicode object to the bytes representation in the NumPy array. I think this is what we should be doing in most of the places and it should considerably simplify the Unicode code in NumPy --- eliminating possibly the ucsnarrow.c file. Am I missing something? Thanks, -Travis

6 6

2 greatest values, in a 3-d array, along one axis
by Jim Vickroy Aug. 3, 2012

Aug. 3, 2012

Hello everyone, I'm trying to determine the 2 greatest values, in a 3-d array, along one axis. Here is an approach: # ------------------------------------------------------ # procedure to determine greatest 2 values for 3rd dimension of 3-d array ... import numpy, numpy.ma xcnt, ycnt, zcnt = 2,3,4 # actual case is (1024, 1024, 8) p0 = numpy.empty ((xcnt,ycnt,zcnt)) for z in range (zcnt) : p0[:,:,z] = z*z zaxis = 2 # max values to be determined for 3rd axis p0max = numpy.max (p0, axis=zaxis) # max values for zaxis maxindices = numpy.argmax (p0, axis=zaxis) # indices of max values p1 = p0.copy() # work array to scan for 2nd highest values j, i = numpy.meshgrid (numpy.arange (ycnt), numpy.arange (xcnt)) p1[i,j,maxindices] = numpy.NaN # flag all max values p1 = numpy.ma.masked_where (numpy.isnan (p1), p1) # hide all max values p1max = numpy.max (p1, axis=zaxis) # 2nd highest values for zaxis # additional code to analyze p0max and p1max goes here # ------------------------------------------------------ I would appreciate feedback on a simpler approach -- e.g., one that does not require masked arrays and or use of magic values like NaN. Thanks, -- jv

4 5

[ANN] SIAM Conference on Computational Science & Engineering Submission Deadlines Approaching!
by Fernando Perez Aug. 2, 2012

Aug. 2, 2012

Dear Colleagues, the SIAM CSE13 conference will be held next year in Boston, and this is a conference that is well suited for much of the type of work that goes on in the open source scientific Python development community (and Julia). The conference is co-chaired by Hans-Petter Langtangen, well known around these parts for his several books on scientific computing with Python and for having led a campus-wide adoption of Python as the core computational foundation across the University of Oslo. I am also on the program committee, as well as Randy LeVeque and other Python-friendly folks. An excellent way to participate is to organize a one- or two-part minisymposium on a specific topic with a group of related speakers (instructions at http://www.siam.org/meetings/cse13/submissions.php). Please note that the MS deadline is fast approaching: August 13, 2012. If you have any further questions, don't hesitate to contact me or one of the other organizers if you feel they can address your concerns more directly: "Fernando Perez" <Fernando.Perez(a)berkeley.edu> "Randy LeVeque" <rjl(a)amath.washington.edu> (Reproducible research track) "Hans Petter Langtangen" <hpl(a)simula.no> (Conference co-chair) "Karen Willcox" <kwillcox(a)mit.edu> (conference chair) ---------- Forwarded message ---------- From: Karen Willcox <kwillcox(a)mit.edu> Date: Tue, Jul 24, 2012 at 6:08 AM Subject: [SIAM-CSE] SIAM Conference on Computational Science & Engineering Submission Deadlines Approaching! To: SIAM-CSE(a)siam.org *SIAM Conference on Computational Science & Engineering (CSE13)* February 25-March 1, 2013 The Westin Boston Waterfront, Boston, Massachusetts, USA**** ** ** SUBMISSION DEADLINES ARE APPROACHING!**** August 13, 2012: Minisymposium proposals September 10, 2012: Abstracts for contributed and minisymposium speakers**** Visit http://www.siam.org/meetings/cse13/submissions.php to submit.**** ** ** Twitter hashtag: #SIAMcse13**** ** ** For more information about the conference, visit * http://www.siam.org/meetings/cse13/* or contact SIAM Conference Department at meetings(a)siam.org.**** -- Karen Willcox Professor and Associate Department Head Department of Aeronautics and Astronautics, MIT http://acdl.mit.edu/willcox.html _______________________________________________ SIAM-CSE mailing list To post messages to the list please send them to: SIAM-CSE(a)siam.org http://lists.siam.org/mailman/listinfo/siam-cse

1 0

load of custom .npy file fails with numpy 2.0.0
by Geoffrey Irving Aug. 2, 2012

Aug. 2, 2012

Hello, The attached .npy file was written from custom C++ code. It loads fine in Numpy 1.6.2 with Python 2.6 installed through MacPorts, but fails on a different machine with Numpy 2.0.0 installed via Superpack: box:array% which python /usr/bin/python box:array% which python box:array% python Python 2.6.1 (r261:67515, Aug 2 2010, 20:10:18) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.load('blah.npy') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Python/2.6/site-packages/numpy-2.0.0.dev_b5cdaee_20110710-py2.6-macosx-10.6-universal.egg/numpy/lib/npyio.py", line 351, in load return format.read_array(fid) File "/Library/Python/2.6/site-packages/numpy-2.0.0.dev_b5cdaee_20110710-py2.6-macosx-10.6-universal.egg/numpy/lib/format.py", line 440, in read_array shape, fortran_order, dtype = read_array_header_1_0(fp) File "/Library/Python/2.6/site-packages/numpy-2.0.0.dev_b5cdaee_20110710-py2.6-macosx-10.6-universal.egg/numpy/lib/format.py", line 361, in read_array_header_1_0 raise ValueError(msg % (d['descr'],)) ValueError: descr is not a valid dtype descriptor: 'd8' >>> numpy.__version__ '2.0.0.dev-b5cdaee' >>> numpy.__file__ '/Library/Python/2.6/site-packages/numpy-2.0.0.dev_b5cdaee_20110710-py2.6-macosx-10.6-universal.egg/numpy/__init__.pyc' It seems Numpy 2.0.0 no longer accepts dtype('d8'): >>> dtype('d8') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type "d8" not understood Was that intentional? An API change isn't too much of a problem, but it's unfortunate if old data files are no longer easily readable. Thanks, Geoffrey

2 4

Reordering 2 dimensional array by column
by Nicole Stoffels Aug. 2, 2012

Aug. 2, 2012

Dear all, I have a two-dimensional array: a = array([[1,2,3],[0,2,1],[5,7,8]]) I want to reorder it by the last column in descending order, so that I get: b =array([[5, 7, 8],[1, 2, 3],[0, 2, 1]]) What I did first is the following, which reorders the array in ascending order (I found that method in the internet): b = array(sorted(a, key=lambda new_entry: new_entry[2])) b = array([[0, 2, 1],[1, 2, 3],[5, 7, 8]]) But I want it just the other way arround. So I did the following afterwards which results in an array only containing zeros: b_indices = b.argsort() b_matrix = b[b_indices[::-1]] new_b = b_matrix[len(b_matrix)-1] Is there an easy way to reorder it? Or is there at least a complicated way which produces the right output? I hope you can help me! Thanks! Best regards, Nicole

2 2