Mailman 3 October 2011 - NumPy-Discussion

Example Usage of Neighborhood Iterator in Cython
by T J 18 Oct '11

18 Oct '11

I recently put together a Cython example which uses the neighborhood iterator. It was trickier than I thought it would be, so I thought to share it with the community. The function takes a 1-dimensional array and returns a 2-dimensional array of neighborhoods in the original area. This is somewhat similar to the functionality provided by segment_axis (http://projects.scipy.org/numpy/ticket/901), but I believe this slightly different in that neighborhood can extend to the left of the current index (assuming circular boundaries). Keep in mind that this is just an example, and normal usage probably is not concerned with creating a new array. External link: http://codepad.org/czRIzXQl -------------- import numpy as np cimport numpy as np cdef extern from "numpy/arrayobject.h": ctypedef extern class numpy.flatiter [object PyArrayIterObject]: cdef int nd_m1 cdef np.npy_intp index, size cdef np.ndarray ao cdef char *dataptr # This isn't exposed to the Python API. # So we can't use the same approach we used to define flatiter ctypedef struct PyArrayNeighborhoodIterObject: int nd_m1 np.npy_intp index, size np.PyArrayObject *ao # note the change from np.ndarray char *dataptr object PyArray_NeighborhoodIterNew(flatiter it, np.npy_intp* bounds, int mode, np.ndarray fill_value) int PyArrayNeighborhoodIter_Next(PyArrayNeighborhoodIterObject *it) int PyArrayNeighborhoodIter_Reset(PyArrayNeighborhoodIterObject *it) object PyArray_IterNew(object arr) void PyArray_ITER_NEXT(flatiter it) np.npy_intp PyArray_SIZE(np.ndarray arr) cdef enum: NPY_NEIGHBORHOOD_ITER_ZERO_PADDING, NPY_NEIGHBORHOOD_ITER_ONE_PADDING, NPY_NEIGHBORHOOD_ITER_CONSTANT_PADDING, NPY_NEIGHBORHOOD_ITER_CIRCULAR_PADDING, NPY_NEIGHBORHOOD_ITER_MIRROR_PADDING np.import_array() def windowed(np.ndarray[np.int_t, ndim=1] arr, bounds): cdef flatiter iterx = <flatiter>PyArray_IterNew(<object>arr) cdef np.npy_intp size = PyArray_SIZE(arr) cdef np.npy_intp* boundsPtr = [bounds[0], bounds[1]] cdef int hoodSize = bounds[1] - bounds[0] + 1 # Create the Python object and keep a reference to it cdef object niterx_ = PyArray_NeighborhoodIterNew(iterx, boundsPtr, NPY_NEIGHBORHOOD_ITER_CIRCULAR_PADDING, None) cdef PyArrayNeighborhoodIterObject *niterx = \ <PyArrayNeighborhoodIterObject *>niterx_ cdef int i,j cdef np.ndarray[np.int_t, ndim=2] hoods hoods = np.empty((arr.shape[0], hoodSize), dtype=np.int) for i in range(iterx.size): for j in range(niterx.size): hoods[i,j] = (niterx.dataptr)[0] PyArrayNeighborhoodIter_Next(niterx) PyArray_ITER_NEXT(iterx) PyArrayNeighborhoodIter_Reset(niterx) return hoods def test(): x = np.arange(10) print x print print windowed(x, [-1, 3]) print print windowed(x, [-2, 2]) ---------- If you run test(), this is what you should see: [0 1 2 3 4 5 6 7 8 9] [[9 0 1 2 3] [0 1 2 3 4] [1 2 3 4 5] [2 3 4 5 6] [3 4 5 6 7] [4 5 6 7 8] [5 6 7 8 9] [6 7 8 9 0] [7 8 9 0 1] [8 9 0 1 2]] [[8 9 0 1 2] [9 0 1 2 3] [0 1 2 3 4] [1 2 3 4 5] [2 3 4 5 6] [3 4 5 6 7] [4 5 6 7 8] [5 6 7 8 9] [6 7 8 9 0] [7 8 9 0 1]] windowed(x, [0, 2]) is almost like segment_axis(x, 3, 2, end='wrap').

3 3

float128 in fact float80
by Matthew Brett 17 Oct '11

17 Oct '11

Hi, After getting rather confused, I concluded that float128 on a couple of Intel systems I have, is in fact an 80 bit extended precision number: http://en.wikipedia.org/wiki/Extended_precision >>> np.finfo(np.float128).nmant 63 >>> np.finfo(np.float128).nexp 15 That is rather confusing. What is the rationale for calling this float128? It is not IEEE 754 float128, and yet it seems to claim so. Best, Matthew

6 16

yet another indexing question
by Neal Becker 17 Oct '11

17 Oct '11

suppose I have: In [10]: u Out[10]: array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) And I have a vector v: v = np.array ((0,1,0,1,0)) I want to form an output vector which selects items from u where v is the index of the row of u to be selected. In the above example, I want: w = [0,6,2,8,4] I can't seem to find a syntax that does this. Now, more importantly, I need the result to be a reference to the original array (not a copy), because I'm going to use it on the LHS of an assignment. Is this possible?

5 5

dtyping with .astype()
by Alex van der Spek 17 Oct '11

17 Oct '11

Beginner's question? I have this dictionary dtypes of names and types: >>>dtypes {'names': ['col1', 'col2', 'col3', 'col4', 'col5'], 'formats': [<type 'numpy.float16'>, <type 'numpy.float16'>, <type 'numpy.float16'>, <type 'numpy.float16'>, <type 'numpy.float16'>]} and this array y >>> y array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24], [25, 26, 27, 28, 29], [30, 31, 32, 33, 34], [35, 36, 37, 38, 39], [40, 41, 42, 43, 44], [45, 46, 47, 48, 49]]) But: >>>>z=y.astype(dtypes) gives me a confusing result. I only asked to name the columns and change their types to half precision floats. What am I missing? How to do this? Thank you in advance, Alex van der Spek

3 4

Type checking inconsistency
by Tony Yu 17 Oct '11

17 Oct '11

Hi, I noticed a type-checking inconsistency between assignments using slicing and fancy-indexing. The first will happily cast on assignment (regardless of type), while the second will throw a type error if there's reason to believe the casting will be unsafe. I'm not sure which would be the "correct" behavior, but the inconsistency is surprising. Best, -Tony Example: >>> import numpy as np >>> a = np.arange(10) >>> b = np.ones(10, dtype=np.uint8) # this runs without error >>> b[:5] = a[:5] >>> mask = a < 5 >>> b[mask] = b[mask] TypeError: array cannot be safely cast to required type

3 4

NumPy example list is corrupted
by Alan Frankel 16 Oct '11

16 Oct '11

I see it now. Thanks! Alan >>Couldn't someone add to the page an icon, >> or even a static link, that links to the "action=info" URL? >> Who's the right person to ask? >It's already there --- it's as part of standard Moinmoin wiki >configuration --- the one with the letter 'i' in it on the upper right >corner of the page. >-- >Pauli Virtanen

1 0

NumPy example list is corrupted
by Alan Frankel 16 Oct '11

16 Oct '11

I've been editing the "Tentative NumPy? Tutorial" and occasionally referring to the "NumPy? Example List" ( http://www.scipy.org/Numpy_Example_List ). In the process, I think I mistakenly corrupted the NumPy Example List. Since the website does not offer any wiki-type functionality for reverting changes or referring to a history of changes, there doesn't seem to be a way for me to fix the problem. Nor is there a "Contact Us" link on the page. I tried the scipy channel on irc/freenode, but no one was there. I eventually filed a bug (#1963), then later decided that maybe this mailing list would be another and perhaps better way to report the problem. In addition to fixing the immediate problem, it would be nice if someone could give registered users the ability for reverting changes to the page as well. That should prevent problems like this in the future. Thanks, Alan

3 4

float128 casting rounding as if it were float64
by Matthew Brett 16 Oct '11

16 Oct '11

Hi, While struggling with floating point precision, I ran into this: In [52]: a = 2**54+3 In [53]: a Out[53]: 18014398509481987L In [54]: np.float128(a) Out[54]: 18014398509481988.0 In [55]: np.float128(a)-1 Out[55]: 18014398509481987.0 The line above tells us that float128 can exactly represent 2**54+3, but the line above that says that np.float128(2**54+3) rounds upwards as if it were a float64: In [59]: np.float64(a) Out[59]: 18014398509481988.0 In [60]: np.float64(a)-1 Out[60]: 18014398509481988.0 Similarly: In [66]: np.float128('1e308') Out[66]: 1.000000000000000011e+308 In [67]: np.float128('1e309') Out[67]: inf Is it possible that float64 is being used somewhere in float128 casting? Best, Matthew

2 3

Printing individual array elements with at least 15 significant digits
by Hugo Gagnon 15 Oct '11

15 Oct '11

Hello, I need to print individual elements of a float64 array to a text file. However in the file I only get 12 significant digits, the same as with: >>> a = np.zeros(3) >>> a.fill(1./3) >>> print a[0] 0.333333333333 >>> len(str(a[0])) - 2 12 whereas >>> len(repr(a[0])) - 2 17 which makes more sense since I am expecting at least 15 significant digits… So how can I print a np.float64 with at least 15 significant digits (without repr!)?

2 1

ndarray with double comparison
by Chao YUE 15 Oct '11

15 Oct '11

Dear all, sorry for this stupid question but I cannot find it in numpy tutorial or google. suppose I have a=np.arange(11). In [32]: a < 8 Out[32]: array([ True, True, True, True, True, True, True, True, False, False, False], dtype=bool) In [34]: a > 4 Out[34]: array([False, False, False, False, False, True, True, True, True, True, True], dtype=bool) how can I have boolean index like 4 < a < 8 np.where(a>4 and a<8);or plainly input "a>4 and a<8" doesn't work. thanks, Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************

5 6

NumPy-Discussion October 2011