Mailman 3 May 2012 - NumPy-Discussion

Debian/Ubuntu patch help (was: ANN: NumPy 1.6.2 release candidate 1)
by Julian Taylor 04 Jun '12

04 Jun '12

> Hi, if there's anyone wants to have a look at the above issue this > week, >that would be great. > If there's a patch by this weekend I can create a second RC, so we can > still have the final release before the end of this month (needed for > Debian freeze). Otherwise a second RC won't be needed. bugfixes are still allowed during the debian freeze, so that should not be an issue for the release timing. I don't see the issue with the gcc --print-multiarch patch besides maybe some cleanup. --print-multiarch is a debian specific gcc patch, but multiarch is debian specific for now. It doesn't work in 11.04, but who cares, that will be end of life in 5 month anyway. Besides x11 almost nothing is multiarched in 11.04 anyway and that can still be covered by the currently existing method. gcc should be available for pretty much anything requiring numpy.distutils anyway so that should be not be an issue. On systems without --print-multiarch or gcc you just ignore the failing, there will be no repercussions as there will also not be any multiarched libraries. the only potential issue I see is that upstream gcc adds a --print-multiarch that does something completely different and harmful for distutils, but I don't consider that very likely. Hardcoding a bunch of paths defeats the whole purpose of multiarch. It will just to break in future (e.g. when the x32 abi comes to debian/ubuntu) and will make cross compiling harder (though I guess numpy distutils may not be built for that anyway)

3 6

fast access and normalizing of ndarray slices
by Wolfgang Kerzendorf 04 Jun '12

04 Jun '12

Dear all, I have an ndarray which consists of many arrays stacked behind each other (only conceptually, in truth it's a normal 1d float64 array). I have a second array which tells me the start of the individual data sets in the 1d float64 array and another one which tells me the length. Example: data_array = (conceptually) [[1,2], [1,2,3,4], [1,2,3]] = in reality [1,2,1,2,3,4,1,2,3, dtype=float64] start_pointer = [0, 2, 6] length_data = [2, 4, 3] I now want to normalize each of the individual data sets. I wrote a simple for loop over the start_pointer and length data grabbed the data and normalized it and wrote it back to the big array. That's slow. Is there an elegant numpy way to do that? Do I have to go the cython way? Cheers Wolfgang

4 5

slicing and aliasing
by Neal Becker 31 May '12

31 May '12

Will copying slices always work correctly w/r to aliasing? That is, will: u[a:b] = u[c:d] always work (assuming the ranges of a:b, d:d are equal, or course)

2 1

Bus error when using flat on sliced, memmap'd array
by chris farrow 31 May '12

31 May '12

Hi all, I encountered an odd bug today that I wanted to bring to everyone's attention. First the code: >>> import numpy as np >>> shape = (8, 8) >>> dtype = np.dtype(np.uint8) >>> image = np.random.randint(0, 256, shape).astype(dtype) >>> image.tofile("test_image.bin") >>> image = np.memmap("test_image.bin", dtype=dtype, shape=shape, mode='r') >>> arr = image[::2,::2] >>> np.sum(arr.flat) On my system (numpy 1.6.1, git revision 68538b74483009c2c2d1644ef00397014f95a696, on OSX, python 2.7.3 (32-bit)), this causes a bus error when run. Here's what I've discovered so far about this: - the bus error only occurs with mode 'r' - the dimensionality of the array appears to be irrelevant - if the array slice does not change the strides, the bus error does not occur - no 'arr.flat', no bus error - Other aggregating functions (e.g. fmin.reduce) will induce the error - Iterating over arr.flat will *not* cause a bus error Based on this, I suspect the issue is with the C-facing side of the flat iterator. Enjoy! Chris

3 2

Scientific Software and Web Developer Needed in Seattle, WA
by Chris Barker 30 May '12

30 May '12

Scientific Software and Web Developer Needed NOAA Emergency Response Division Help us develop our next-generation oil spill transport model. Background: The Emergency Response Division (ERD) of NOAA's Office of Response and Restoration (OR&R) provides scientific expertise to support the response to oil and chemical spills in the coastal environment. We played a major role in the recent Deepwater Horizon oil spill in the Gulf of Mexico. In order to fulfill our mission, we develop many of the software tools and models required to support a response to hazardous material spills. We are currently in the middle of a program to develop our next-generation oil spill transport model, taking into account lessons learned from years of response and recent major incidents. There are currently two positions available: one will focus on on the computational code in C++, Pyhton and Cython, and the other on a new web front end (Python, HTML, CSS, Javascript). General Characteristics: The incumbents in this position will provide software development services to support the mission of the Emergency Response Division of NOAA's Office of Response and Restoration. As part of his/her efforts, independent evaluation and application of development techniques, algorithms, software architecture, and programming patterns will be required. The incumbent will work with the staff of ERD to provide analysis on user needs and software, GUI, and library design. He/she will be expect to work primarily on site at NOAA's facility in Seattle. Knowledge: The incumbent must be able to apply modern concepts of software engineering and design to the development of computational code, web applications, and libraries. The incumbent will need to be able to design, write, refactor, and implement code for a complex web application and/or computational library. The incumbent will work with a multi-disciplinary team including scientists, users, and other developers, utilizing software development practices such as usability design, version control, bug and issue tracking, and unit testing. Good communication skills and the knowledge of working as part of a team are required. Direction received: The incumbent will participate on various research and development teams. While endpoints will be identified through Division management and some direct supervision will be provided, the incumbent will be responsible for progressively being able to take input from team meetings and design objectives and propose strategies for reaching endpoints. Typical duties and responsibilities: The incumbent will work with the oil and chemical spill modeling team to improve and develop new tools and models used in fate and transport forecasting. Different components of the project will be written in C++, Python, and Javascript. Education requirement, minimum: Bachelor's degree in a technical discipline. Experience requirement, minimum: One to five years experience in development of complex software systems in one or more full-featured programming languages (C, C++, Java, Python, Javaascript, Ruby, Fortran, etc.) The team requires experience in the following languages/disciplines. Each incumbent will need experience in some subset: * Computational/Scientific programming * Numerical Analysis/Methods * Parallel processing * Desktop GUI * Web services * Web clients: HTML/CSS/Javascript * Python * C/C++ * Python--C/C++ integration * Software development team leadership While the incumbent will work on-site at NOAA, directly with the NOAA team, this is a contract position with General Dynamics Information Technology: http://www.gdit.com/default.aspx For more information and to apply, use the GDIT web site: http://www.resumeware.net/gdns_rw/gdns_web/job_detail.cfm?recnum=1&totalrec… If that long url doesn't work, try: http://www.resumeware.net/gdns_rw/gdns_web/job_search.cfm and search for job ID: 199765 You can also send questions about employment issues to: Susan Bowley: susan.bowley(a)gdit.com And questions about the nature of the work to: Chris Barker: Chris.Barker(a)noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

1.6.2 no more unique for rows
by josef.pktd＠gmail.com 30 May '12

30 May '12

https://github.com/numpy/numpy/commit/74b9f5eef8fac643bf9012dbb2ac6b4b19f46… broke return_inverse for structured arrays, because of the use of mergesort I'm using structured dtypes to get uniques and return_inverse by rows >>> groups = np.random.randint(0,4,size=(10,2)) >>> groups_ = groups.view([('',groups.dtype)]*groups.shape[1]).flatten() >>> groups array([[0, 2], [1, 2], [1, 1], [3, 1], [3, 1], [2, 1], [1, 0], [3, 3], [3, 2], [0, 0]]) >>> groups_ array([(0, 2), (1, 2), (1, 1), (3, 1), (3, 1), (2, 1), (1, 0), (3, 3), (3, 2), (0, 0)], dtype=[('f0', '<i4'), ('f1', '<i4')]) >>> np.argsort(groups_) array([9, 0, 6, 2, 1, 5, 4, 3, 8, 7]) >>> np.argsort(groups_, kind='mergesort') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python26\lib\site-packages\numpy\core\fromnumeric.py", line 679, in argsort return argsort(axis, kind, order) TypeError: requested sort not available for type >>> uni, uni_idx, uni_inv = np.unique(groups_, return_index=True, return_inverse=True) >>> uni_inv array([1, 4, 3, 6, 6, 5, 2, 8, 7, 0]) exception in numpy 1.6.2rc2 (as reported by Debian for statsmodels) Josef

4 5

Numpy code in GPL package
by Henry Gomersall 30 May '12

30 May '12

I'd like to include the _cook_nd_args() function from fftpack in my GPL code. Is this possible? How should I modify my license file to satisfy the Numpy license requirements, but so it's clear which function it applies to? Thanks, Henry

2 2

f2py hiding callback function
by Marc Poinot 30 May '12

30 May '12

Hi all, I'm trying to hide the actual python callback function on the fortran side as well as on the python side. See the example: I want f1 to be the wrapper of my callback, f2 is a second level where the user has no arg to pass. But if I call f2, for example in f3, the callback is 'propagated' and I have to declare it the same way as in f2. Is there a trick to stop this propagation? subroutine f1(p,t) cf2py intent(callback) push_back_path cf2py optional push_back_path cf2py external push_back_path character(len=*) p character(len=*) t call push_back_path(p,t) return end subroutine f2() cf2py intent(callback) push_back_path cf2py optional push_back_path cf2py external push_back_path cf2py use f1__user__routines character(len=96) p character(len=32) t p='/usr' t='local' call f1(p,t) t='lib' call f1(p,t) return end subroutine f3() call f2() end The python test: # ----------------------------------------------- # f2py -c pathutils.for -m pathutils # python pathtest.py import pathutils import string def push_back_path(p,t): p=p.strip() t=t.strip() p=string.ljust(p+'/'+t,96) print 'PUSH_BACK_PATH [%s]'%p pathutils.push_back_path=push_back_path print 'F1' pathutils.f1('/usr','bin') print 'F2' # Hiding on python side is ok pathutils.f2() # Hiding on fortran side fails print 'F3' pathutils.f3() Which leads to: $ python pathtest.py F1 PUSH_BACK_PATH [/usr/bin ] F2 PUSH_BACK_PATH [/usr/local ] PUSH_BACK_PATH [/usr/lib ] F3 capi_return is NULL Call-back cb_push_back_path_in_f1__user__routines failed. capi_return is NULL Call-back cb_push_back_path_in_f1__user__routines failed. Traceback (most recent call last): File "pathtest.py", line 20, in <module> pathutils.f3() TypeError: push_back_path() takes exactly 2 arguments (0 given) TIA -MP- ----------------------------------------------------------------------- Marc POINOT [ONERA/DSNA] Tel:+33.1.46.73.42.84 Fax:+33.1.46.73.41.66 Avertissement/disclaimer http://www.onera.fr/onera-en/emails-terms

1 0

Re: [Numpy-discussion] [enhancement] sum_angle() and sum_polar()
by Robert Jördens 29 May '12

29 May '12

On Tue, May 29, 2012 at 11:03 AM, Stéfan van der Walt <stefan(a)sun.ac.za> wrote: > On Mon, May 28, 2012 at 11:53 AM, Travis Oliphant <travis(a)continuum.io> wrote: >> I could see these functions going into scipy.ndimage but again because they >> are not necessarily just image processing functions, and the fact that they >> are so simple, perhaps they are best put into NumPy itself. > > I'm wondering about the general applicability of these functions. Can > anyone suggest some use cases? An example from solid state physics: If you have a spin chain with some long-range interaction and you have the known, dense, coupling matrix J, sum_angle(J, pi/4) gives you a view at the distance dependence of the interaction. -- Robert Jordens.

2 2

[enhancement] sum_angle() and sum_polar()
by Robert Jördens 29 May '12

29 May '12

Hi everyone, I am proposing to add the the two following functions to numpy/lib/twodim_base.py: sum_angle() computes the sum of a 2-d array along an angled axis sum_polar() computes the sum of a 2-d array along radial lines or along azimuthal circles https://github.com/numpy/numpy/pull/230 Comments? When I was looking for a solution to these problems of calculating special sums of 2-d arrays I could not find anything and it took me a while to figure out a (hopefully) useful and consistent algorithm. I can see how one would extend these to higher dimensions but that would preclude using bincount() to do the heavy lifting. Looking at some other functions, the doctests might need to be split into real examples and unittests. Best, -- Robert Jordens.

5 5