Mailman 3 July 2011 - NumPy-Discussion

Change in behavior of PyArray_BYTES(
by Matthew Brett July 27, 2011

July 27, 2011

Hi, I was trying to compile matplotlib against current trunk, and hit an error with this line: char* row0 = PyArray_BYTES(matrix); https://github.com/matplotlib/matplotlib/blob/master/src/agg_py_transforms.… The error is: src/agg_py_transforms.cpp:30:26: error: invalid conversion from ‘void*’ to ‘char*’ It turned out that the output type of PyArray_BYTES has changed between 1.5.1 and current trunk In 1.5.1, ndarraytypes.h: #define PyArray_BYTES(obj) (((PyArrayObject *)(obj))->data) (resulting in a char *, from the char * bytes member of PyArrayObject) In current trunk we have this: #define PyArray_BYTES(arr) PyArray_DATA(arr) ifndef NPY_NO_DEPRECATED_API then this results in: #define PyArray_DATA(obj) ((void *)(((PyArrayObject_fieldaccess *)(obj))->data)) giving a void * ifdef NPY_NO_DEPRECATED_API then: static NPY_INLINE char * PyArray_DATA(PyArrayObject *arr) { return ((PyArrayObject_fieldaccess *)arr)->data; } resulting in a char * (for both PyArray_DATA and PyArray_BYTES. It seems to me that it would be safer to add back this line: #define PyArray_BYTES(obj) (((PyArrayObject *)(obj))->data) to ndarraytypes.h , within the ifndef NPY_NO_DEPRECATED_API block, to maintain compatibility. Do y'all agree? Best, Matthew

2 2

inconsistent semantics for double-slicing
by Alex Flint July 27, 2011

July 27, 2011

When applying two different slicing operations in succession (e.g. select a sub-range, then select using a binary mask) it seems that numpy arrays can be inconsistent with respect to assignment: For example, in this case an array is modified: In [6]: *A = np.arange(5)* In [8]: *A[:][A>2] = 0* In [10]: A Out[10]: *array([0, 1, 2, 0, 0])* Whereas here the original array remains unchanged In [11]: *A = np.arange(5)* In [12]: *A[[0,1,2,3,4]][A>2] = 0* In [13]: A Out[13]: *array([0, 1, 2, 3, 4])* This arose in a less contrived situation in which I was trying to copy a small image into a large image, modulo a mask on the small image. Is this meant to be like this? Alex

2 1

nanmin(<masked array>) fails with 'TypeError: cannot reduce a scalar'. Numpy 1.6.0 regression?
by Mark Dickinson July 27, 2011

July 27, 2011

In NumPy 1.6.0, I get the following behaviour: Python 2.7.2 |EPD 7.1-1 (32-bit)| (default, Jul 3 2011, 15:40:35) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "packages", "demo" or "enthought" for more information. >>> import numpy >>> numpy.nanmin(numpy.ma.masked_array([1,2,3,4])) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/7.1/lib/python2.7/site-packages/numpy/lib/function_base.py", line 1507, in nanmin return np.fmin.reduce(a.flat) TypeError: cannot reduce on a scalar >>> numpy.__version__ '1.6.0' In NumPy version 1.5.1: Python 2.7.2 |EPD 7.1-1 (32-bit)| (default, Jul 3 2011, 15:40:35) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "packages", "demo" or "enthought" for more information. >>> import numpy >>> numpy.nanmin(numpy.ma.masked_array([1,2,3,4])) 1 >>> numpy.__version__ '1.5.1' Was this change intentional? -- Mark

3 3

Numpy master breaking matplotlib build
by Michael Droettboom July 27, 2011

July 27, 2011

The return type of PyArray_BYTES in the old API compatibility code seems to have changed recently to (void *) which breaks matplotlib builds. This pull request changes it back. Is this correct? https://github.com/numpy/numpy/pull/121 Mike

2 1

lazy loading ndarrays
by Craig Yoshioka July 27, 2011

July 27, 2011

I want to subclass ndarray to create a class for image and volume data, and when referencing a file I'd like to have it load the data only when accessed. That way the class can be used to quickly set and manipulate header values, and won't load data unless necessary. What is the best way to do this? Are there any hooks I can use to load the data when an array's values are first accessed or manipulated? I tried some trickery with __array_interface__ but couldn't get it to work very well. Should I just use a memmapped array, and give up on a purely 'lazy' approach? Thanks, and cheers! -Craig

4 4

code review/build & test for datetime business day API
by Mark Wiebe July 26, 2011

July 26, 2011

These functions are now fully implemented and documented. As always, code reviews are welcome here: https://github.com/numpy/numpy/pull/87 and for those that don't want to dig into review C code, the commit for the documentation is here: https://github.com/m-paradox/numpy/commit/6b5a42a777b16812e774193b06da1b68b… This is probably also another good place to do a merge to master, so if people could test it on Mac/Windows/other platforms that would be much appreciated. Thanks, Mark On Fri, Jun 10, 2011 at 5:49 PM, Mark Wiebe <mwwiebe(a)gmail.com> wrote: > I've implemented the busday_offset function with support for the weekmask > and roll parameters, the commits are tagged 'datetime-bday' in the pull > request here: > > https://github.com/numpy/numpy/pull/87 > > -Mark > > > On Thu, Jun 9, 2011 at 5:23 PM, Mark Wiebe <mwwiebe(a)gmail.com> wrote: > >> Here's a possible design for a business day API for numpy datetimes: >> >> >> The 'B' business day unit will be removed. All business day-related >> calculations will be done using the 'D' day unit. >> >> A class *BusinessDayDef* to encapsulate the definition of the business >> week and holidays. The business day functions will either take one of these >> objects, or separate weekmask and holidays parameters, to specify the >> business day definition. This class serves as both a performance >> optimization and a way to encapsulate the weekmask and holidays together, >> for example if you want to make a dictionary mapping exchange names to their >> trading days definition. >> >> The weekmask can be specified in a number of ways, and internally becomes >> a boolean array with 7 elements with True for the days Monday through Sunday >> which are valid business days. Some different notations are for the 5-day >> week include [1,1,1,1,1,0,0], "1111100" "MonTueWedThuFri". The holidays are >> always specified as a one-dimensional array of dtype 'M8[D]', and are >> internally used in sorted form. >> >> >> A function *is_busday*(datearray, weekmask=, holidays=, busdaydef=) >> returns a boolean array matching the input datearray, with True for the >> valid business days. >> >> A function *busday_offset*(datearray, offsetarray, >> roll='raise', weekmask=, holidays=, busdaydef=) which first applies the >> 'roll' policy to start at a valid business date, then offsets the date by >> the number of business days specified in offsetarray. The arrays datearray >> and offsetarray are broadcast together. The 'roll' parameter can be >> 'forward'/'following', 'backward'/'preceding', 'modifiedfollowing', >> 'modifiedpreceding', or 'raise' (the default). >> >> A function *busday_count*(datearray1, datearray2, weekmask=, holidays=, >> busdaydef=) which calculates the number of business days between datearray1 >> and datearray2, not including the day of datearray2. >> >> >> For example, to find the first Monday in Feb 2011, >> >> >>>np.busday_offset('2011-02', 0, roll='forward', weekmask='Mon') >> >> or to find the number of weekdays in Feb 2011, >> >> >>>np.busday_count('2011-02', '2011-03') >> >> This set of three functions appears to be powerful enough to express the >> business-day computations that I've been shown thus far. >> >> Cheers, >> Mark >> > >

7 25

Build error on Windows
by Chris Barker July 26, 2011

July 26, 2011

Hi folks, I'm trying to build numpy HEAD on Windows in preparation for the SciPy sprints tomorrow. I've never built numpy on Windows, and I'm new to git, so I could be doing any number of things wrong. I think I have the latest code: C:\Documents and Settings\Chris\My Documents\SciPy\numpy_git\numpy>git log -1 commit 6fdfd9c070ce943415d75780702a22f4bbd8f837 Author: Ben Walsh <b(a)wumpster.com> Date: Tue Jul 12 14:52:01 2011 +0100 I tried to build it with a simple: python setup.py build_ext --inplace and got: C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL /nologo /INCRE MENTAL:NO /LIBPATH:c:\python27\libs /LIBPATH:c:\python27\PCbuild /LIBPATH:build\ temp.win32-2.7 /EXPORT:init_dummy build\temp.win32-2.7\Release\numpy\core\src\du mmymodule.obj /OUT:numpy\core\_dummy.pyd /IMPLIB:build\temp.win32-2.7\Release\nu mpy\core\src\_dummy.lib /MANIFESTFILE:build\temp.win32-2.7\Release\numpy\core\sr c\_dummy.pyd.manifest LINK : error LNK2001: unresolved external symbol init_dummy build\temp.win32-2.7\Release\numpy\core\src\_dummy.lib : fatal error LNK1120: 1 unresolved externals error: Command "C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DL L /nologo /INCREMENTAL:NO /LIBPATH:c:\python27\libs /LIBPATH:c:\python27\PCbuild /LIBPATH:build\temp.win32-2.7 /EXPORT:init_dummy build\temp.win32-2.7\Release\n umpy\core\src\dummymodule.obj /OUT:numpy\core\_dummy.pyd /IMPLIB:build\temp.win3 2-2.7\Release\numpy\core\src\_dummy.lib /MANIFESTFILE:build\temp.win32-2.7\Relea se\numpy\core\src\_dummy.pyd.manifest" failed with exit status 1120 Python 2.7.2, python.org build (32 bit) Visual Studio express 2008 That should work, yes? Thanks for any hints. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

7 7

Multi-taper spectral analysis package: pymutt. Critical update.
by martin smith July 26, 2011

July 26, 2011

The current release, version 0.82.0, contains fixes for two major bugs. The first bug is a show-stopping segmentation fault under some versions of Linux and arises from a variable type mismatch in calls to the numpy api. The second bug causes bad spectral values at the Nyquist frequency for series with even lengths and arises from an error in interfacing to a local version of fftpack. I strongly recommend upgrading to the current release. Pymutt is a module that implements Thomson's multi-taper spectral analysis algorithms. The code is based on a subroutine from Lees and Park and has, of course, a python interface. References are provided in the readme file. The code has seen substantial usage and should be fairly reliable. Examples are included. It's available at http://code.google.com/p/pymutt/. Martin L. Smith Blindgoat Geophysics

1 0

Get a 1D slice of a 3D data set?
by Nicolas Bigaouette July 25, 2011

July 25, 2011

Hi all, I have a 3D orthogonal and non-uniform grid representing a scalar field. I'm using matplotlib.image.NonUniformImage() to plot it similarly to imshow(). What I'd like to do is plot the values of the scalar field across a specific line (say, from point A to B). Any suggestion? Thanks!

1 0

Optimizing recursive loop
by Robert Elsner July 25, 2011

July 25, 2011

Hey Everybody, I am approximating the derivative of nonperiodic functions on [-1,1] using Chebyshev polynomials. The implementation is straightforward and works well but is painfully slow. The routine wastes most of its time on a trivial operation (see comment in the code) Unfortunately the spectral coefficients are calculated recursively and thus I haven't found a way to speed up the code. I am aware of list comprehensions, the speed advantage of calling native numpy functions etc. But no luck yet finding an alternate formulation that speeds up the calculation. I attached a stripped down variant of the code. I am very grateful for tips or directions where to look for a solution (well I know writing a small C extension might be the way to go but Id love to have speedy Python) Note: The DCT takes almost no time compared to the statement in the loop. Cheers Robert

4 7