Mailman 3 April 2012 - NumPy-Discussion

problem importing NumPy from C/C++
by Jim Bosch 13 Apr '12

13 Apr '12

One of my colleagues is having some difficulty with some of my code that uses the NumPy C-API; I think we're seeing the C import_array() function claim that it can't find NumPy, even though "import numpy" works just fine in Python. Here's a minimal embedded-Python program that is enough to cause problems: --------- #include "Python.h" #include "numpy/arrayobject.h" void doImport() { import_array(); } int main() { int result = 0; Py_Initialize(); doImport(); if (PyErr_Occurred()) { result = 1; } else { npy_intp dims = 2; PyObject * a = PyArray_SimpleNew(1, &dims, NPY_INT); if (!a) result = 1; Py_DECREF(a); } Py_Finalize(); return result; } --------- That compiles and links fine, but when running it we see: --------- ImportError: numpy.core.multiarray failed to import --------- and the exit code is 1. Note that we can run python and successfully import numpy there, in what's otherwise the same environment, so it doesn't appear to be PYTHONPATH issue (or, I think, an environment issue, though there might be a difference between embedding and extending). All of the above does work on a number of other systems, so I suspect something went wrong with this particular NumPy install, but it claims to have been successful. This is with gcc 4.1.2, RHEL 5. Any ideas? Thanks! Jim Bosch

1 0

[Reminder] Summer School "Advanced Scientific Programming in Python" in Kiel, Germany
by Tiziano Zito 13 Apr '12

13 Apr '12

Advanced Scientific Programming in Python ========================================= a Summer School by the G-Node and the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python is assumed. Participants without any prior experience with Python should work through the proposed introductory materials before the course. Date and Location ================= September 2—7, 2012. Kiel, Germany. Preliminary Program =================== Day 0 (Sun Sept 2) — Best Programming Practices - Best Practices, Development Methodologies and the Zen of Python - Version control with git - Object-oriented programming & design patterns Day 1 (Mon Sept 3) — Software Carpentry - Test-driven development, unit testing & quality assurance - Debugging, profiling and benchmarking techniques - Best practices in data visualization - Programming in teams Day 2 (Tue Sept 4) — Scientific Tools for Python - Advanced NumPy - The Quest for Speed (intro): Interfacing to C with Cython - Advanced Python I: idioms, useful built-in data structures, generators Day 3 (Wed Sept 5) — The Quest for Speed - Writing parallel applications in Python - Programming project Day 4 (Thu Sept 6) — Efficient Memory Management - When parallelization does not help: the starving CPUs problem - Advanced Python II: decorators and context managers - Programming project Day 5 (Fri Sept 7) — Practical Software Development - Programming project - The Pelita Tournament Every evening we will have the tutors' consultation hour: Tutors will answer your questions and give suggestions for your own projects. Applications ============ You can apply on-line at http://python.g-node.org Applications must be submitted before 23:59 UTC, May 1, 2012. Notifications of acceptance will be sent by June 1, 2012. No fee is charged but participants should take care of travel, living, and accommodation expenses. Candidates will be selected on the basis of their profile. Places are limited: acceptance rate last time was around 20%. Prerequisites: You are supposed to know the basics of Python to participate in the lectures. You are encouraged to go through the introductory material available on the website. Faculty ======= - Francesc Alted, Continuum Analytics Inc., USA - Pietro Berkes, Enthought Inc., UK - Valentin Haenel, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Zbigniew Jędrzejewski-Szmek, Faculty of Physics, University of Warsaw, Poland - Eilif Muller, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Emanuele Olivetti, NeuroInformatics Laboratory, Fondazione Bruno Kessler and University of Trento, Italy - Rike-Benjamin Schuppner, Technologit GbR, Germany - Bartosz Teleńczuk, Unité de Neurosciences Information et Complexité, Centre National de la Recherche Scientifique, France - Stéfan van der Walt, Helen Wills Neuroscience Institute, University of California Berkeley, USA - Bastian Venthur, Berlin Institute of Technology and Bernstein Focus Neurotechnology, Germany - Niko Wilbert, TNG Technology Consulting GmbH, Germany - Tiziano Zito, Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany Organized by Christian T. Steigies and Christian Drews of the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel , and by Zbigniew Jędrzejewski-Szmek and Tiziano Zito for the German Neuroinformatics Node of the INCF. Website: http://python.g-node.org Contact: python-info(a)g-node.org

1 0

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 67, Issue 43
by Peter Plantinga 13 Apr '12

13 Apr '12

> > Really you have to have this setup in order to run a fortran executable, > but the only thing that comes to mind is the LD_LIBRARY_PATH environment > variable. LD_LIBRARY_PATH is a colon separated list of paths that is > searched for dynamic libraries by both regular programs and by Python. Thanks for the suggestion! I added the library to LD_LIBRARY_PATH and that fixed it. On Thu, Apr 12, 2012 at 4:25 PM, <numpy-discussion-request(a)scipy.org> wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. physics simulation (Peter Plantinga) > 2. Re: physics simulation (Tim Cera) > 3. Re: YouTrack testbed (Travis Oliphant) > 4. Re: YouTrack testbed (william ratcliff) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 12 Apr 2012 15:24:43 -0400 > From: Peter Plantinga <pwp2(a)students.calvin.edu> > Subject: [Numpy-discussion] physics simulation > To: numpy-discussion(a)scipy.org > Message-ID: > <CAEy-jdL+mYCQGsuoh2fjLTOPqNotXKjA8y4ch2k1jBzJByU2KQ(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > I'm trying to run a physics simulation on a cluster. The original program > is written in fortran, and highly parallelizable. Unfortunately, I've had a > bit of trouble getting f2py to work with the compiler I'm using (absoft > v.10.1). The cluster is running Linux v. 12. > > When I type just "f2py" I get the following error: > > > f2py > Traceback (most recent call last): > File "/opt/absoft10.1/bin/f2py", line 20, in <module> > from numpy.f2py import main > File "/usr/local/lib/python2.6/dist-packages/numpy/__init__.py", line > 137, in <module> > import add_newdocs > File "/usr/local/lib/python2.6/dist-packages/numpy/add_newdocs.py", line > 9, in <module> > from numpy.lib import add_newdoc > File "/usr/local/lib/python2.6/dist-packages/numpy/lib/__init__.py", line > 13, in <module> > from polynomial import * > File "/usr/local/lib/python2.6/dist-packages/numpy/lib/polynomial.py", > line 17, in <module> > from numpy.linalg import eigvals, lstsq > File "/usr/local/lib/python2.6/dist-packages/numpy/linalg/__init__.py", > line 48, in <module> > from linalg import * > File "/usr/local/lib/python2.6/dist-packages/numpy/linalg/linalg.py", > line 23, in <module> > from numpy.linalg import lapack_lite > ImportError: libaf90math.so: cannot open shared object file: No such file > or directory > > It looks like f2py cannot find libaf90math.so, located in > /opt/absoft10.1/shlib. How can I tell f2py where af90math is? > > Thanks for the help! > Peter Plantinga >

1 0

physics simulation
by Peter Plantinga 12 Apr '12

12 Apr '12

I'm trying to run a physics simulation on a cluster. The original program is written in fortran, and highly parallelizable. Unfortunately, I've had a bit of trouble getting f2py to work with the compiler I'm using (absoft v.10.1). The cluster is running Linux v. 12. When I type just "f2py" I get the following error: > f2py Traceback (most recent call last): File "/opt/absoft10.1/bin/f2py", line 20, in <module> from numpy.f2py import main File "/usr/local/lib/python2.6/dist-packages/numpy/__init__.py", line 137, in <module> import add_newdocs File "/usr/local/lib/python2.6/dist-packages/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/usr/local/lib/python2.6/dist-packages/numpy/lib/__init__.py", line 13, in <module> from polynomial import * File "/usr/local/lib/python2.6/dist-packages/numpy/lib/polynomial.py", line 17, in <module> from numpy.linalg import eigvals, lstsq File "/usr/local/lib/python2.6/dist-packages/numpy/linalg/__init__.py", line 48, in <module> from linalg import * File "/usr/local/lib/python2.6/dist-packages/numpy/linalg/linalg.py", line 23, in <module> from numpy.linalg import lapack_lite ImportError: libaf90math.so: cannot open shared object file: No such file or directory It looks like f2py cannot find libaf90math.so, located in /opt/absoft10.1/shlib. How can I tell f2py where af90math is? Thanks for the help! Peter Plantinga

2 1

Re: [Numpy-discussion] YouTrack testbed
by Hans-Martin v. Gaudecker 12 Apr '12

12 Apr '12

On 12.04.2012, at 18:38, numpy-discussion-request(a)scipy.org wrote: >>> Redmine does look good from a quick browse (better view, does display >>> diffs). It would be good to get the opinions of a few more people on this >>> topic. >>> >> >> Redmine is "trac on RoR", but it solves two significant issues over trac: >> - mass edit (e.g. moving things to a new mileston is simple and doable >> from the UI) >> - REST API by default, so that we can build simple command line tools on >> top of it (this changed since I made the wiki page) >> >> It is a PITA to install, though, at least if you are not familiar with >> ruby, and I heard it is hard to manage as well. >> > > Thanks, that's a clear description of pros and cons. It's also easy to play > with Redmine at demo.redmine.org. That site allows you to set up a new > project and try the admin interface. > > My current list of preferences is: > > 1. Redmine (if admin overhead is not unreasonable) > 2. Trac with performance issues solved > 3. Github > 4. YouTrack > 5. Trac with current performance I've been running a Redmine server for a couple of years now, managing lots of small scientific projects. Setup was not trivial back then but if you can manage to run it under Ubuntu it should install smoothly, there seems to be a package for it nowadays. Maintenance has been practically zero over the period. Except for one issue during setup that doesn't apply to NumPy (automatic creation of a Subversion repository when a project is created), I never noticed the fact that it is written in Ruby. The overall experience has been much nicer than with Trac, but I can't comment much on the advanced issue tracker functionality, which seems to be the crucial point here. Best, Hans-Martin

1 0

creating/working NumPy-ndarrays in C++
by Holger Herrlich 12 Apr '12

12 Apr '12

Hi, I plan to migrate core classes of an application from Python to C++ using SWIG, while still the user interface being Python. I also plan to further use NumPy's ndarrays. The application's core classes will create the ndarrays and make calculations. The user interface (Python) finally receives it. C++ OOP features will be deployed. What general ways to work with NumPy ndarrays in C++ are here? I know of boost.python so far. Regards Holger

12 24

change masked array member values with conditional selection
by Chao YUE 12 Apr '12

12 Apr '12

Dear all numpy users, I am using numpy 1.6.1 I find that if you want to change some member values in a masked array according to some conditional selection. suppose a is a masked array, you want to change all value below zero to zero. you must always use a[np.nonzero(a<0)]=0 rather than a[a<0]=0. the latter will lose all masked elements. an example: In [24]: a=np.arange(10.) In [25]: a=np.ma.masked_array(a,mask=a<3) In [28]: a[a<5]=2. In [29]: a Out[29]: masked_array(data = [2.0 2.0 2.0 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [False False False False False False False False False False], fill_value = 1e+20) In [30]: b=np.arange(10.) In [31]: b=np.ma.masked_array(b,mask=b<3) In [34]: b[np.nonzero(b<5)]=2. In [35]: b Out[35]: masked_array(data = [-- -- -- 2.0 2.0 5.0 6.0 7.0 8.0 9.0], mask = [ True True True False False False False False False False], fill_value = 1e+20) cheers, Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************

2 2

partial computations
by santhu kumar 12 Apr '12

12 Apr '12

Hello all, I am trying to optimise a code and want your suggestions. A : - NX3 matrix (coordinates of N points) After performing pairwise distance computations(called pdist) between these points, depending upon a condition that the distance is in, I would perform further computations. Most of the computations require schur products (element by element) of NXN matrices with each other and then computing either the coloumn sum or row sum. As N goes to be large, these computations are taking some time (0.7 secs) which is not much generally but since this is being called many times, it acts as a bottleneck. I want to leverage on the fact that many of the NXN computations are not going to be used, or would be set to zero (if the pdist is greater than some minimum distance). How do i achieve it ?? Is masked array the elegant solution? Would it save me time? Thanks Santhosh

2 1

RuntimeWarning: numpy.ndarray size changed
by Nils Wagner 11 Apr '12

11 Apr '12

Hi all, Can someone reproduce the following message ? Python 2.7.2 (default, Aug 19 2011, 20:41:43) [GCC] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy /home/nwagner/local/lib64/python2.7/site-packages/numpy/random/__init__.py:91: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility from mtrand import * >> numpy.__version__ '1.7.0.dev-9aac543' >>> Nils

2 2

Slices from an index list
by Mads Ipsen 11 Apr '12

11 Apr '12

Hi, Suppose a have an array of indices, say indices = [0,1,2,3,5,7,8,9,10,12,13,14] Then the following slices a = slice(0,4) b = slice(4,5) c = slice(5,9) d = slice(9,12) provide information about all the consecutive parts of the index list. Given the list of indices, is there some nifty numpy function that can generate the above slices for me (or their start and stop values)? Best regards, Mads -- +-----------------------------------------------------+ | Mads Ipsen | +----------------------+------------------------------+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby | phone: +45-29716388 | | Denmark | email: mads.ipsen(a)gmail.com | +----------------------+------------------------------+

3 4