Mailman 3 August 2010 - NumPy-Discussion

Replacing values based on index
by Pinner, Luke Aug. 12, 2010

Aug. 12, 2010

Apologies if this has been asked many times before, but I'm having trouble even coming up with the right terms to search on. I have an x,y,date stack of satellite images from which I need to know on what date the max and min values occur. I can calculate the index at which the min/max values occur, but how do I join that back to the date? A simple example case: import numpy as np #Date when image was captured, #images are in the stack in this order days=[10, 20, 31, 41, 51, 59, 69, 79, 90, 100] #dummy array representing something like #"stackofimages.argmin(axis=2)" indexmin=np.array([[0, 2, 1], [6, 7, 3], [5, 5, 8]]) #I want to convert indexmin to "daymin" i.e #[[10, 31, 20], # [69, 79, 41], # [59, 59, 90]] #Where the each element of indexmin represent the value #at the corresponding index in the days list/array. #What I am currently doing is below, #which is _rather_ slow on large arrays #as I'm calling np.where len(days) times daymin=np.zeros(indexmin.shape).astype(np.int) for i,d in enumerate(days): daymin[np.where(indexmin==i)]=d Any better suggestions? Regards Luke Pinner ------ If you have received this transmission in error please notify us immediately by return e-mail and delete all copies. If this e-mail or any attachments have been sent to you in error, that error does not constitute waiver of any confidentiality, privilege or copyright in respect of information in the e-mail or attachments. Please consider the environment before printing this email. ------

2 2

how to fit a given pdf
by Renato Fabbri Aug. 12, 2010

Aug. 12, 2010

Dear All, help appreciated, thanks in advance. how do you fit a pdf you have with a given pdf (say gamma). with the file attached, you can go like: a=open("AC-010_ED-1m37F100P0.txt","rb") aa=a.read() aaa=aa[1:-1].split(",") data=[int(i) for i in aaa] if you do pylab.plot(data); pylab.show() The data is something like: ___|\___ It is my pdf (probability density function). how can i find the right parameters to make that fit with a gamma? if i was looking for a normal pdf, for example, i would just find mean and std and ask for the pdf. i've been playing with scipy.stats.distributions.gamma but i have not reached anything. we can extend the discussion further, but this is a good starting point. any idea? thanks! best of luck, rf -- GNU/Linux User #479299 skype: fabbri.renato

3 2

numpy.ma.flatnotmasked_contiguous error in slice.stop values?
by Geordie McBain Aug. 11, 2010

Aug. 11, 2010

The function numpy.ma.flatnotmasked_contiguous returns slices which miss the last element; for example, based on the example at http://docs.scipy.org/doc/numpy/reference/generated/numpy.ma.flatnotmasked_…, import numpy as np a = np.arange (10) am = np.ma.array (a, mask = (a < 3) | (a > 8) | (a == 5)) segments = np.ma.flatnotmasked_contiguous (am) print 'the masked array:', am print 'flatnotmasked_contiguous: ', segments print am[segments[0]], am[segments[1]] prints the masked array: [-- -- -- 3 4 -- 6 7 8 --] flatnotmasked_contiguous: [slice(3, 4, None), slice(6, 8, None)] [3] [6 7] which isn't really what we're after; I would have expected [3 4] and [6 7 8]. Compare the similarly named matplotlib.mlab.contiguous_regions which does have the expected behaviour (although its output format is different) import matplotlib.mlab segments1 = matplotlib.mlab.contiguous_regions (am) print 'contiguous_regions:', segments1 print am[slice (*segments1[0])], am[slice (*segments1[1])] which prints contiguous_regions: [(3, 5), (6, 9)] [3 4] [6 7 8] (I'm running Python 2.6.4 on 64-bit Linux Mint 8 `Helena', with NumPy from Ubuntu Karmic, which I think means I have NumPy version 1.3.0?)

3 4

ANN fwrap v0.1.0
by Kurt Smith Aug. 8, 2010

Aug. 8, 2010

Fwrap v0.1.0 ============ I am pleased to announce the first release of Fwrap v0.1.0, a utility for wrapping Fortran code in C, Cython and Python. Fwrap focuses on supporting the features of Fortran 90/95, but will work with most well-behaved Fortran 77 code, too. Fwrap is BSD licensed. Fwrap is beta-level software; all public APIs and commandline options are subject to change. Features in the v0.1.0 release: * Fortran source parsing and automatic wrapper generation; * Top-level (non-module) Fortran subroutines and functions; * Supports all intrinsic types (integer, real, complex, logical & character); * Default and non-default intrinsic types properly wrapped; * Scalar and array arguments supported; * Supports assumed-shape, assumed-size, and explicit-shape array arguments; * Intent 'in', 'inout', 'out' and no intent arguments supported; * Automatic docstring generation for extension module and functions; * Wrappers are portable w.r.t. both Fortran and C compilers. Upcoming features: * "Use" statement support; * Python & Cython callback support; * Kind-parameters in type specification; * Module-level data and parameters; * User-derived types. See the README and documentation for requirements, compiler support, etc. Download -------- You can get fwrap from pypi or from its sourceforge download page: https://sourceforge.net/projects/fwrap/files/ More Info --------- Project homepage, including links to wiki & bug tracker: http://fwrap.sourceforge.net/ Mailing list: http://groups.google.com/group/fwrap-users Development blog: http://fortrancython.wordpress.com/

2 1

absoft and f2py
by Sturla Molden Aug. 7, 2010

Aug. 7, 2010

I asked before, but did not get an answer. What is the problem with absoft and f2py? Sturla C:\Absoft11.0>f2py -c --help-fcompiler Forcing DISTUTILS_USE_SDK=1 Unexpected ValueError in C:\Python26\lib\site-packages\numpy\distutils\fcompiler \compaq.py Traceback (most recent call last): File "C:\Python26\Scripts\f2py-script.py", line 24, in <module> main() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 557, in main run_compile() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 543, in run_co mpile setup(ext_modules = [ext]) File "C:\Python26\lib\site-packages\numpy\distutils\core.py", line 186, in set up return old_setup(**new_attr) File "C:\Python26\lib\distutils\core.py", line 138, in setup ok = dist.parse_command_line() File "C:\Python26\lib\distutils\dist.py", line 460, in parse_command_line args = self._parse_command_opts(parser, args) File "C:\Python26\lib\distutils\dist.py", line 574, in _parse_command_opts func() File "C:\Python26\lib\site-packages\numpy\distutils\command\config_compiler.py ", line 13, in show_fortran_compilers show_fcompilers(dist) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py", li ne 848, in show_fcompilers load_all_fcompiler_classes() File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py", li ne 715, in load_all_fcompiler_classes __import__ (module_name) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 55, in <module> class CompaqVisualFCompiler(FCompiler): File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 95, in CompaqVisualFCompiler raise e ValueError: [u'path', u'lib']

1 1

dtype.type for structured arrays
by Thomas Robitaille Aug. 7, 2010

Aug. 7, 2010

Hi, If I create a structured array with vector columns: >>> array = np.array(zip([[1,2],[1,2],[1,3]]),dtype=[('a',float,2)]) then examine the type of the column, I get: >>> array.dtype[0] dtype(('float64',(2,))) Then, if I try and view the numerical type, I see: >>> array.dtype[0].type <type 'numpy.void'> I have to basically do >>> array.dtype[0].subdtype[0] dtype('float64') to get what I need. I seem to remember that this used not to be the case, and that even for vector columns, one could access array.dtype[0].type to get the numerical type. Is this a bug, or deliberate? Thanks, Thomas

2 2

use index array of len n to select columns of n x m array
by Martin Spacek Aug. 6, 2010

Aug. 6, 2010

I want to take an n x m array "a" and index into it using an integer index array "i" of length n that will pull out the value at the designated column from each corresponding row of "a". >>> a = np.arange(10) >>> a.shape = 5, 2 >>> a array([[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]) >>> i = np.array([0, 1, 1, 0, 1]) I want: >>> b = a.foo(i) >>> b array([0, 3, 5, 6, 9]) What's foo? I can't get take() to do what I want. I went and wrote my own little Cython function to do this, but that seems silly (and is also array dtype dependent). I've tried reading through the numpy book, and I'm sure this is somewhere on the list, but I can't find it. I think it has something to do with fancy indexing. I should know how to do this by know... Cheers, Martin

5 14

{OT} Mailing trends
by Gökhan Sever Aug. 6, 2010

Aug. 6, 2010

Hello, There is a nice e-mailing trend tool for Gmail users at http://code.google.com/p/mail-trends/ It is a command line tool producing an html output showing your e-mailing statistics. In my inbox, the following threads are highly ranked in the top threads section. [Numpy-discussion] Announcing toydist, improving distribution and packaging situation<http://www.mail-archive.com/numpy-discussion@scipy.org/msg22760.html> [SciPy-Dev] scipy.stats<http://comments.gmane.org/gmane.comp.python.scientific.devel/13384> [Numpy-discussion] curious about how people would feel about moving to github<http://old.nabble.com/curious-about-how-people-would-feel-about-moving-to-g…> Just out of curiosity, are there any mailing trends (top threads, top posters, etc...) provided for the Python related mailing archives? Share your comments please. -- Gökhan

3 4

numpy histogram normed=True (bug / confusing behavior)
by Nils Becker Aug. 6, 2010

Aug. 6, 2010

Hi, I found what looks like a bug in histogram, when the option normed=True is used together with non-uniform bins. Consider this example: import numpy as np data = np.array([1, 2, 3, 4]) bins = np.array([.5, 1.5, 4.5]) bin_widths = np.diff(bins) (counts, dummy) = np.histogram(data, bins) (densities, dummy) = np.histogram(data, bins, normed=True) What this gives is: bin_widths array([ 1., 3.]) counts array([1, 3]) densities array([ 0.1, 0.3]) The documentation claims that histogram with normed=True gives a density, which integrates to 1. In this example, it is true that (densities * bin_widths).sum() is 1. However, clearly the data are equally spaced, so their density should be uniform and equal to 0.25. Note that (0.25 * bin_widths).sum() is also 1. I believe np.histogram(data, bins, normed=True) effectively does : np.histogram(data, bins, normed=False) / (bins[-1] - bins[0]). However, it _should_ do np.histogram(data, bins, normed=False) / bins_widths to get a true density over the data coordinate as a result. It's easy to fix by hand, but I think the documentation is at least misleading?! sorry if this has been discussed before; I did not find it anyway (numpy 1.3)

3 2

Installing numpy with MKL
by Søren Gammelmark Aug. 6, 2010

Aug. 6, 2010

Hi everyone I realize that this e-mail has been written a lot of times before. I have, however, not been able to make it work even though I have looked through a lot of e-mails on the mailinglist. I apologize if the solution is written neatly somewhere. As many others I would like to install numpy with MKL support on a cluster. The cluster contains dual core AMD Opteron machines. The reason for doing this is to enable me to perform multi-threaded matrix-operations (matrix-multiplications and singular value decompositions primarily). Additionally I need to integrate some Fortran code also using Intel MKL into numpy using f2py. I think I need to use MKL for this, but correct me if I am wrong. I have tried to follow the guide at http://www.scipy.org/Installing_SciPy/Linux#head-7ce43956a69ec51c6f2cedd894…. I have compiled my own python and installed it locally using the LD_RUN_PATH suggested there using gcc (I only have access to the intel 9.1 compilers which seems to have a bug preventing me from compiling python with it (see http://bugs.python.org/issue1689617)). This works without any problems. I have access to Intel MKL 10.2.5.035 and have the following site.cfg for numpy 1.4.1: [DEFAULT] library_dirs = /home/sgamark/usr/lib # for python is installed with prefix $HOME/usr include_dirs = /home/sgamark/usr/include [mkl] library_dirs = /com/intel/mkl/10.2.5.035/lib/em64t include_dirs = /com/intel/mkl/10.2.5.035/include lapack_libs = mkl_lapack mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5, pthread, mkl_def and have changes numpy/distutils/intelcompiler.py according to http://www.scipy.org/Installing_SciPy/Linux#head-7ce43956a69ec51c6f2cedd894… but with cc_exe = 'icc -fPIC -openmp -fomit-frame-pointer -O2 -g'. I have include openmp since I believe it is necessary for using the threading features of MKL (correct me if I am wrong). I have also tried modifiying numpy/distutils/system_info.py as described in http://article.gmane.org/gmane.comp.python.numeric.general/18877/match=inte… but without any effect. However, executing "python setup.py config" shows that the setup-script is unable to find any of the libraries I have (see attached file). If I try to compile anyway MKL is not used and setting OMP_NUM_THREADS have no effect. At this point I am a bit frustrated, since it seems that it should not be too difficult to get to work since it appears that other people have no problems. Indeed, several people I work with have no problem doing this with intel 10.1 compilers, MKL 11.1 on Intel Itanium machines. I am not particularly experienced with this sort of thing but I usually have few problems compiling a stand-alone Fortran or C/C++ program that uses MKL. I would greatly appreciate it if anyone could help with this. Regards Søren Gammelmark P.s: The operating system is Linux/CentOS 4.4, x86_64

6 10