NumPy-Discussion September 2010

numpy-discussion@python.org

121 participants
125 discussions

Re: [Numpy-discussion] A proposed change to rollaxis(), behavior for negative 'start' values
by Ken Basye 23 Sep '10

23 Sep '10

Anne says: This is a tricky one. The current behaviour of rollaxis is to remove the requested axis from the list of axes and then insert it before the axis specified. This is exactly how python's list insertion works: In [1]: a = range(10) In [3]: a.insert(-1,'a') In [4]: a Out[4]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 'a', 9] And indeed, there's no clean way to add something to the end of a list using insert (apart from the obvious a.insert(len(a),'b') ). For this you have .append(). Unfortunately numpy's rollaxis, while it agrees with insert in its behaviour, doesn't have a move_axis_to_end. The situation is also somewhat muddied by the fact that rollaxis also removes the axis from the original list of axes, so that the interpretation of index numbers is a little more subtle. But I think your suggested behaviour would be confusing because of the conflict with python's insert. How about allowing the string "end" as an argument to rollaxis to specify that the axis should go at the end? Anne Ralf says: Allowing "end" is an easy solution, but note that moving an axis to the end > is already possible: > > >>> > >>> a = np.ones((3,4,5,6)) >>> > >>> np.rollaxis(a, 2, len(a)+1).shape # roll axis to to last position >>> > (3, 4, 6, 5) > > Not consistent with insert though, there you would use len(a) instead of > len(a)+1. It's a little ugly, but perhaps just documenting this is no worse > than allowing a string or adding yet another function. > > Chuck says: It is a common enough operation that it would be nice to have a less cumbersome way to specify it. Ralf, that's not going to work generally, because len(a) is a.shape[0], right? It works in the example above but if I had done >>> a = np.ones((1,2,3,4)) it wouldn't work. In my original posting, I suggested >>> np.rollaxis(a, 2, len(a.shape)) which I think will work fine, but it still seems cumbersome, as Chuck points out. I'm going to take one more tilt at the windmill. To me, the connection with Python's insert() is tenuous: insert() is a destructive operation whereas rollaxis() returns a new array with the requested change, also rollaxis is doing the delete and insert in one step. I know the implementation uses insert() and delete(), but I think the natural way to think about rollaxis() is in terms of moving an axis from one position to another and there is no Python analog to that. Anne's comment that "the situation is somehat muddied..." is precisely why I think the change is worth making - if the meaning of the 'start' argument were changed from "The axis is rolled until it lies before this position." to "The axis is rolled until it lies *at* this position." then it seems to me there's no confusion. The current description "The axis is rolled until it lies before this position." is ambiguous because "this position" seems like it means "position in the shape of the array" but then what's the position before 0? So how about "The axis is rolled until it lies before the axis currently at this position." That's much closer, but now it's hard to make sense of using len(a.shape) as an argument, since there is no axis currently at len(a.shape). To really capture the muddiness, we need to say something like "The axis is rolled until it lies before the axis currently at this position, unless the value is len(a.shape), in which case the axis is rolled to the end" - yuck. Again, note that the proposed change applies only to positions specified with negative indices; positive indices will all work as before. Best, Ken

1 0

Indexing and lookup issues
by Ross Williamson 22 Sep '10

22 Sep '10

Hi everyone I suspect this is easy but I'm stuck say I have a 1D array: t = [10,11,12] and a 2D array: id = [[0,1,0] [0,2,0] [2,0,2]] In could in IDL do y = t[id] which would produce: y = [[10,11,10] [10,12,10] [12,10,12]] i.e. use the indexes in id on the lookup array t. Is there an easy way to do this in numpy? Cheers Ross -- Ross Williamson University of Chicago Department of Astronomy & Astrophysics 773-834-9785 (office) 312-504-3051 (Cell)

2 2

Removing numpy numpy-1.1.0 in Python 2.5
by Wayne Watson 22 Sep '10

22 Sep '10

I may have bounced a similar problem around here a few months ago, but this one is a bit more important to get an answer for. I'm about to distribute some report programs to three people who work together and should be using the same version of Python and libraries that I use, and a common application to us. However, to create two new report programs they do not have, I need to make sure we do have the same versions, and in particular need to use the numpy 1.2.0 I used for the report programs. To replace the old numpy and make sure we are all on on the same wave length, I wrote the description below. I want to know if what I say about removing 1.1.0 and installing 1.2.0 is correct. I think deleting the folder as below is the right way to go. numpy is not in Control Panel Add/Remove. Comments? ================================= Required Libraries for Sentinel Report Programs for Win XP ... You will need to replace your numpy-1.1.0 library. This is accomplished by first removing numpy-1.1.0 from the Python library located at C:\Python2.5\Lib\site-packages. Delete the numpy folder. Next execute the numpy-1.2.0-win32-superpack-python2.5.exe to install numpy 1.2.0. Try sentuser.py to make sure it runs properly. If not, contact me. ================================== -- Wayne Watson (Watson Adventures, Prop., Nevada City, CA) (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time) Obz Site: 39° 15' 7" N, 121° 2' 32" W, 2700 feet Want to lose weight? Move to the equator. You'll weigh about 0.1% less than at the poles. Centrifugal force is directed opposite gravity there--zero at the poles. -- WTW Web Page:<www.speckledwithstars.net/>

4 7

printable random seed ?
by josef.pktd＠gmail.com 22 Sep '10

22 Sep '10

I would like to generate random numbers based on a random seed, for example what numpy.random does if the seed is not specified. But I would also like to print out the initial state, so I can replicate the random numbers. Can I get a human readable or printable version of the initial state? Alternatively, what's a good way to randomly generate an initial state? I could draw an integer with randint and use it as seed. Is this the best way? Josef

3 6

searching binary data
by Neal Becker 22 Sep '10

22 Sep '10

A colleague of mine posed the following problem. He wants to search large files of binary data for sequences. I thought of using mmap (to avoid reading all data into memory at once) and then turning this into a numpy array (using buffer=). But, how to then efficiently find a sequence? Note that I'm not committed to using numpy or mmap as part of the solution.

3 4

read ascii file from complex fortran format() -- genfromtxt
by Andrew Jaffe 22 Sep '10

22 Sep '10

Hi all, I've got an ascii file with a relatively complicated structure, originally written by fortran with the format: 135 format(a12,1x,2(f10.5,1x),i3,1x,4(f9.3,1x),4(i2,1x),3x, 1 16(f7.2,1x),i3,3x,f13.5,1x,f10.5,1x,f10.6,1x,i3,1x, 2 4(f10.6,1x), 2 i2,1x,f5.2,1x,f10.3,1x,i3,1x,f7.2,1x,f7.2,3x,4(f7.4,1x), 3 4(f7.2,1x),3x,f7.2,1x,i4,3x,f10.3,1x,14(f6.2,1x),i3,1x, 1 3x,2f10.5,8f11.2,2f10.5,f12.3,3x, 4 2(a6,1x),a23,1x,a22,1x,a22) Note, in particular, that many of the strings contain white space. Is there a relatively straightforward way to translate this into dtype (and delimiter?) arguments for use with genfromtxt or do I just have to do it by hand? Andrew

3 3

real and imag functions should produce errors for object arrays
by Michael Gilbert 22 Sep '10

22 Sep '10

Hi, The following example demonstrates a rather unexpected result: >>> import numpy >>> x = numpy.array( complex( 1.0 , 1.0 ) , numpy.object ) >>> print x.real (1+1j) >>> print x.imag 0 Shouldn't real and imag return an error in such a situation? Thanks, Mike

5 10

unique 2d arrays
by Peter Schmidtke 21 Sep '10

21 Sep '10

Dear all, I'd like to know if there is a pythonic / numpy way of retrieving unique lines of a 2d numpy array. In a way I have this : [[409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [409 152] [426 193] [431 129]] And I'd like to get this : [[409 152] [426 193] [431 129]] How can I do this without workarounds like string concatenation or such things? Numpy.unique flattens the whole array so it's not really of use here. Cheers. -- Peter Schmidtke PhD Student Dept. Physical Chemistry Faculty of Pharmacy University of Barcelona

4 6

[ANN] SciPy India 2010 Call for Presentations
by Jarrod Millman 21 Sep '10

21 Sep '10

========================== SciPy 2010 Call for Papers ========================== The second `SciPy India Conference <http://scipy.in>`_ will be held from December 13th to 18th, 2010 at `IIIT-Hyderabad <http://www.iiit.ac.in/>`_. At this conference, novel applications and breakthroughs made in the pursuit of science using Python are presented. Attended by leading figures from both academia and industry, it is an excellent opportunity to experience the cutting edge of scientific software development. The conference is followed by two days of tutorials and a code sprint, during which community experts provide training on several scientific Python packages. We invite you to take part by submitting a talk abstract on the conference website at: http://scipy.in Talk/Paper Submission ========================== We solicit talks and accompanying papers (either formal academic or magazine-style articles) that discuss topics regarding scientific computing using Python, including applications, teaching, development and research. Papers are included in the peer-reviewed conference proceedings, published online. Please note that submissions primarily aimed at the promotion of a commercial product or service will not be considered. Important Dates ========================== Monday, Oct. 11: Abstracts Due Saturday, Oct. 30: Schedule announced Tuesday, Nov. 30: Proceedings paper submission due Monday-Tuesday, Dec. 13-14: Conference Wednesday-Friday, Dec. 15-17: Tutorials/Sprints Saturday, Dec. 18: Sprints Organizers ========================== * Jarrod Millman, Neuroscience Institute, UC Berkeley, USA (Conference Co-Chair) * Prabhu Ramachandran, Department of Aerospace Engineering, IIT Bombay, India (Conference Co-Chair) * FOSSEE Team

1 0

compile process of numpy-1.5.0 fails in Itanium cluster with icc
by Miguel Pan Fidalgo 21 Sep '10

21 Sep '10

hi, I'm trying to install numpy-1.5.0 in our Itanium (IA-64) supercomputer [http://www.cesga.es/content/view/917/115/lang,en/ ]. This is my site.cfg (I have to install umfpack but I think this is not the problem): $ cat site.cfg [DEFAULT] library_dirs = /opt/cesga/fftw-3.2.2/lib:/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64 include_dirs = /opt/cesga/fftw-3.2.2/include:/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include #[umfpack] #umfpack_libs = umfpack [fftw] libraries = fftw3 [mkl] library_dirs = /opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64 lapack_libs = mkl_lapack mkl_libs = mkl_core The only hacks that I made: $ diff numpy/distutils/system_info.py ../numpy- scipy/numpy-1.5.0/numpy/distutils/system_info.py 818c818 < _lib_mkl = ['mkl','mkl_lapack','guide'] --- > _lib_mkl = ['mkl','vml','guide'] $ diff numpy/distutils/intelccompiler.py ../numpy- scipy/numpy-1.5.0/numpy/distutils/intelccompiler.py 11c11 < cc_exe = 'icc -O3 -openmp -lpthread -fPIC' --- > cc_exe = 'icc' And I have used this script to launch the compilation process: $ cat compilar.sh #!/bin/bash module load nose/0.10.4 fftw/3.2.2 ifort/11.1.056 icc/11.1.056 mkl/10.2.2 python setup.py config --compiler=intele --fcompiler=intele build install -- prefix=/home/cesga/mpan/numpy-1.5.0 exit 0 blas and lapack are detected properly: $ head -n 40 compilar.log Running from numpy source directory.F2PY Version 1 blas_opt_info: blas_mkl_info: FOUND: libraries = ['mkl_core', 'pthread'] library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/cesga/fftw-3.2.2/include', '/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include'] FOUND: libraries = ['mkl_core', 'pthread'] library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/cesga/fftw-3.2.2/include', '/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include'] lapack_opt_info: lapack_mkl_info: mkl_info: FOUND: libraries = ['mkl_core', 'pthread'] library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/cesga/fftw-3.2.2/include', '/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include'] FOUND: libraries = ['mkl_lapack', 'mkl_core', 'pthread'] library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/cesga/fftw-3.2.2/include', '/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include'] FOUND: libraries = ['mkl_lapack', 'mkl_core', 'pthread'] library_dirs = ['/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/lib/64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/cesga/fftw-3.2.2/include', '/opt/cesga/intel/intel11.0/Compiler/11.1/056/mkl/include'] running config running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options but, unfortunately, the compilation process ends with an unexpected realocation problem: $ tail -n 11 compilar.log /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -shared build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o build/lib.linux-ia64-2.4/numpy/core/umath_tests.so ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol inner1d_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol innerwt_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol matrix_multiply_signature ld: final link failed: Nonrepresentable section on output ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol inner1d_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol innerwt_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol matrix_multiply_signature ld: final link failed: Nonrepresentable section on output error: Command "/opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc - shared build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o build/lib.linux-ia64-2.4/numpy/core/umath_tests.so" failed with exit status 1 I have tried to link this files manually but I haven't any success: $ /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -shared build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o build/lib.linux-ia64-2.4/numpy/core/umath_tests.so ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol inner1d_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol innerwt_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol matrix_multiply_signature ld: final link failed: Nonrepresentable section on output $ /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -fPIC -shared build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o build/lib.linux-ia64-2.4/numpy/core/umath_tests.so ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol inner1d_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol innerwt_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol matrix_multiply_signature ld: final link failed: Nonrepresentable section on output $ /opt/cesga/intel/intel11.0/Compiler/11.1/056/bin/ia64/icc -i-dynamic -shared build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-ia64-2.4 -o build/lib.linux-ia64-2.4/numpy/core/umath_tests.so ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol inner1d_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol innerwt_signature ld: build/temp.linux-ia64-2.4/build/src.linux- ia64-2.4/numpy/core/src/umath/umath_tests.o: @gprel relocation against dynamic symbol matrix_multiply_signature ld: final link failed: Nonrepresentable section on output with differents combinations (MKL 9.1, no optimatazions flags, no hacks.....) I have the same problem. any idea? -- Atentamente, Miguel Pan Fidalgo (Applications Technician) mail: mpan at cesga.es web: http://www.cesga.es Avda. de Vigo s/n 15705, Santiago de Compostela Telf.: +34 981 569810 - Fax: 981 594616

1 0

← Newer
1
2
3
4
5
6
7
8
...
13
Older →

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion September 2010