Mailman 3 September 2008 - NumPy-Discussion

python 2.4 incompatibility in numpy.lookfor in numpy 1.2.0rc2
by joep 24 Sep '08

24 Sep '08

Hi, I was trying to use lookfor on python 2.4 and got an exception with np.lookfor because of a missing method in pkgutil that was added in python 2.5. see the following session >>> np.lookfor('range') Traceback (most recent call last): File "<pyshell#55>", line 1, in ? np.lookfor('range') File "C:\Programs\Python24\Lib\site-packages\numpy\lib\utils.py", line 556, in lookfor cache = _lookfor_generate_cache(module, import_modules, regenerate) File "C:\Programs\Python24\Lib\site-packages\numpy\lib\utils.py", line 687, in _lookfor_generate_cache for m in pkgutil.iter_modules(item.__path__): AttributeError: 'module' object has no attribute 'iter_modules' >>> import pkgutil >>> dir(pkgutil) ['__builtins__', '__doc__', '__file__', '__name__', 'extend_path', 'os', 'sys'] >>> pkgutil.__file__ 'C:\\Programs\\Python24\\lib\\pkgutil.pyc' >>> np.version.version '1.2.0rc2' Josef

4 6

Re: [Numpy-discussion] New patch for new mgrid / ogrid functionality
by David Kaplan 24 Sep '08

24 Sep '08

Hi, On Tue, 2008-09-23 at 20:54 -0500, numpy-discussion-request(a)scipy.org wrote: > On Tue, Sep 23, 2008 at 09:39:53AM +0200, David Kaplan wrote: > > I would note that there is nothing in the API breakage that prevents > > doing what Gael mentions. The only change is that: > > > > > X = mgrid[0:4, 0:4, 0:4] > > > would be replaced with: > > > X = asarray( mgrid[ 0:4, 0:4, 0:4 ] ) > > Doesn't this force a data copy? > I am not sure if this forces a copy (asarray shouldn't for arrays, unlike array, but I am not sure what it does for lists that can trivially become arrays), but if it does, you are getting it one way or another - in the non-API-breakage patch I sent, the asarray is incorporated into the class method itself. For Robert, the use cases I was talking about included ogrid - ogrid with or without non-slice arguments always returns a list, mgrid with non-slices returns a list, but mgrid with all slices returns an array. I find this funky, but I guess I am going upwind. I personally find adding another object that does stuff like mgrid and ogrid, but not exactly the same, hard to swallow. mgrid and ogrid are already pretty mysterious, adding another is going to make things worse. I was initially trying to make things easier for the new user by providing one function that does all of what you want in a transparent way; this solution seems to be going in the opposite direction. If you do end up adding one, I would call it ndgrid as this is what the matlab equivalent is called. The idea of making it a function eliminates the use of slices, which is where the real power of nd_grid comes in, so I would not go that route. If you want to go with a new object, I would add a "force_array" option to nd_grid that would be checked before executing the asarray code - this option would be true for mgrid, but false for ogrid and "ndgrid". Cheers, David > Ga?l > > -- ********************************** David M. Kaplan Charge de Recherche 1 Institut de Recherche pour le Developpement Centre de Recherche Halieutique Mediterraneenne et Tropicale av. Jean Monnet B.P. 171 34203 Sete cedex France Phone: +33 (0)4 99 57 32 27 Fax: +33 (0)4 99 57 32 95 http://www.ur097.ird.fr/team/dkaplan/index.html **********************************

3 2

np.min(np.inf,1000) == np.inf
by joep 23 Sep '08

23 Sep '08

I just fell over this: >>> np.min(np.inf,1000) 1.#INF >>> min(np.inf,1000) 1000 >>> np.max(-np.inf,1000) -1.#INF >>> max(-np.inf,1000) 1000 Is this known behavior? I finally realized, that I have to do this (the doc string says array like) >>> np.min([np.inf,1000]) 1000.0 >>> np.max([-np.inf,1000]) 1000.0 >>> np.version.version '1.2.0rc2' Josef

2 1

Re: [Numpy-discussion] New patch for new mgrid / ogrid functionality
by David Kaplan 23 Sep '08

23 Sep '08

Hi, I would note that there is nothing in the API breakage that prevents doing what Gael mentions. The only change is that: > > X = mgrid[0:4, 0:4, 0:4] would be replaced with: X = asarray( mgrid[ 0:4, 0:4, 0:4 ] ) On the other hand, not making this change makes for a strange and ugly implementation. > Agreed. Please don't change this behavior even through a deprecation. > Long term it is bound to cause problems to have nd_grid return a list in 3 out of 4 use cases, but an array in other cases. This function is already quite confusing for users because it uses indexing instead of a function call. Adding this nuance might be great for the power user, but will confuse most. I am convinced that eventually the API should be broken, the question is just when. If you want to put it off till "2.0", that is fine, but it puts more onus on the release manager to do the cleanup necessary. Cheers, David -- ********************************** David M. Kaplan Charge de Recherche 1 Institut de Recherche pour le Developpement Centre de Recherche Halieutique Mediterraneenne et Tropicale av. Jean Monnet B.P. 171 34203 Sete cedex France Phone: +33 (0)4 99 57 32 27 Fax: +33 (0)4 99 57 32 95 http://www.ur097.ird.fr/team/dkaplan/index.html **********************************

4 4

complex roots() segfault on Solaris 10/x86 with numpy 1.2.rc1 using python 2.5.2
by Michael Abshoff 23 Sep '08

23 Sep '08

Hi, I have been having trouble with the following computation with numpy 1.2.rc1: Python 2.5.2 (r252:60911, Jul 14 2008, 15:20:38) [GCC 4.2.4] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> from numpy import * >>> a = array([1,0,0,0,0,-1],dtype=complex) array([ 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, -1.+0.j]) >>> roots(a) Segmentation Fault (core dumped) python "$@" This is python 2.5.2 build with gcc 4.2.4, numpy itself is build with "-O0", i.e. this is unlikely to be a compiler bug IMHO. This bug has been present in 1.0.4, 1.1.0 and it seems unfixed in 1.2.rc1. The numpy 1.1 test suite passed with that install, I did not run the 1.2.rc1 one yet since I do not have nose installed. gdb says nothing particularly helpful: -bash-3.00$ gdb python GNU gdb 6.6 Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-pc-solaris2.10"... (gdb) r Starting program: /home/mabshoff/sage-3.0.5-x86-solaris/local/bin/python warning: rw_common (): unable to read at addr 0xd3f08 warning: sol_thread_new_objfile: td_ta_new: Debugger service failed warning: Unable to find dynamic linker breakpoint function. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. Python 2.5.2 (r252:60911, Jul 14 2008, 15:20:38) [GCC 4.2.4] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> from numpy import * >>> a = array([1,0,0,0,0,-1],dtype=complex) >>> a array([ 1.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, 0.+0.j, -1.+0.j]) >>> roots(a) Program received signal SIGSEGV, Segmentation fault. 0xfed622b3 in csqrt () from /lib/libm.so.2 (gdb) bt #0 0xfed622b3 in csqrt () from /lib/libm.so.2 #1 0xfe681d5a in ?? () #2 0x00000000 in ?? () (gdb) quit Any pointers? Cheers, Michael

2 5

Re: [Numpy-discussion] array with named columns (or record arrays with homogenous types)
by Tony Yu 23 Sep '08

23 Sep '08

> > Message: 8 > Date: Mon, 22 Sep 2008 19:28:50 -0400 > From: Pierre GM <pgmdevlist(a)gmail.com> > Subject: Re: [Numpy-discussion] array with named columns (or record > arrays with homogenous types) > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <200809221928.50535.pgmdevlist(a)gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > On Monday 22 September 2008 19:25:16 Tony S Yu wrote: > >>>>> import numpy as np >>>>> >>>>> color = np.array([(1, 2, 3), (4, 5, 6)], >> >> ... dtype=[('r', '<f4'), ('g', '<f4'), ('b', >> '<f4')]) >> # get the average (grayscale) of the first row >> >>>>> color[0].mean() > > What about > color.view((float,3))[0].mean() ? Yeah, that works perfectly; by specifying the axis and dropping the index (`color.view(('<f4',3)).mean(axis=1)) gives the mean of all the records. BTW, is the tuple argument you use with `view` documented anywhere; I haven't seen it before and a quick search doesn't give any results. Thanks! -Tony

2 1

array with named columns (or record arrays with homogenous types)
by Tony S Yu 22 Sep '08

22 Sep '08

Is there any way to name the columns of an array so that they can be called with a string, but still used like an ordinary array? What I have in mind is very similar to a record array, but with homogenous types. Having different types implies that certain operations (e.g., mean, sum) don't make sense, in general. For example, I want to do something like: #~~~~~~~ >>> import numpy as np >>> color = np.array([(1, 2, 3), (4, 5, 6)], ... dtype=[('r', '<f4'), ('g', '<f4'), ('b', '<f4')]) # get the average (grayscale) of the first row >>> color[0].mean() TypeError: cannot perform reduce with flexible type #~~~~~~~ BTW, I actually want add the fields (columns) for all records (rows), not just each row individually, but `color.mean(axis=1)` doesn't make sense because `color` is actually a 1D array of records. Thanks, -Tony

2 1

New patch for new mgrid / ogrid functionality
by David M. Kaplan 22 Sep '08

22 Sep '08

Hi all, Attached is a newer version of my patch that adds new mgrid / ogrid functionality for working with arrays in addition to slices. In fact, I have attached two versions of the patch: index_tricks.patch, that is just the last version of the patch I sent, and index_tricks.new.patch, that has been modified so that it is backward compatible. In the last version, mgrid calls where all arguments are slices will return an array, otherwise it returns a list as ogrid does. This is the only reasonable way to have the new functionality and maintain backwards compatibility. My 2 cents - I personally think the version that always returns a list will ultimately be more transparent and cause fewer problems than the newer version. In either case, the plan should be to eventually have it always return a list as that is the only fully consistent option, the question is just when that switch should be made and by who. If it is done at the next major release, someone else will have to remember to ax the additional code and correct the documentation.... Other changes that would be nice: add a __call__ method, create an instance called ndgrid for matlab compatibility, and have meshgrid be reimplimented using an nd_grid instance. Cheers, David -- ********************************** David M. Kaplan Charge de Recherche 1 Institut de Recherche pour le Developpement Centre de Recherche Halieutique Mediterraneenne et Tropicale av. Jean Monnet B.P. 171 34203 Sete cedex France Phone: +33 (0)4 99 57 32 27 Fax: +33 (0)4 99 57 32 95 http://www.ur097.ird.fr/team/dkaplan/index.html **********************************

4 5

ANN: Enthought Python Distribution 4.0.300 Beta 2 available
by Travis Vaught 22 Sep '08

22 Sep '08

Greetings, We've recently posted the second beta release of the Enthought Python Distribution (EPD) for our upcoming general release of version 4.0.300 with Python 2.5. You may download the beta from here: http://www.enthought.com/products/epdbeta.php Please feel free to test it out and provide feedback on the EPD Trac instance: https://svn.enthought.com/epd You can check out the release notes here: http://www.enthought.com/products/epdbetareleasenotes.php About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python™ Programming Language, including over 60 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, database adapters, and a lot of other tools right out of the box. http://www.enthought.com/products/epd.php It is currently available as a single-click installer for Windows XP (x86), Mac OS X (a universal binary for Intel 10.4 and above) and RedHat EL3 (x86 and amd64). EPD is free for academic use. An annual Subscription and installation support are available for individual commercial use (http://www.enthought.com/products/epddownload.php ). An Enterprise Subscription with support for particular deployment environments is also available for commercial purchase (http://www.enthought.com/products/enterprise.php ). The Beta versions of EPD are available for indefinite free trial. Thanks, Travis

1 0

efficient way to do this?
by John Hunter 22 Sep '08

22 Sep '08

I have a an array of indices into a larger array where some condition is satisfied. I want to create a larger set of indices which *mark* all the indicies following the condition over some Nmark length window. In code: import numpy as np N = 1000 Nmark = 20 ind = np.nonzero(np.random.rand(N)<0.01)[0] marked = np.zeros(N, bool) for i in ind: marked[i:i+Nmark] = True I am going to have to do this over many arrays, and so I want to do it efficiently. Is there a way to do the above more efficiently, eg w/o the loop. In the real use case, there will be significant auto-correlation among the places where the condition is satisfied. Eg, if it is satisfied at some index, it is likely that it will be satisfied for many of its neighbors. Eg, the real case looks more like y = np.sin(2*np.pi*np.linspace(0, 2, N)) ind = np.nonzero(y>0.95)[0] marked2 = np.zeros(N, bool) for i in ind: marked2[i:i+Nmark] = True Thanks in advance for any hints, JDH

3 6