Mailman 3 August 2009 - NumPy-Discussion

saving incrementally numpy arrays
by Juan Fiol 12 Aug '09

12 Aug '09

Hi, I am creating numpy arrays in chunks and I want to save the chunks while my program creates them. I tried to use numpy.save but it failed (because it is not intended to append data). I'd like to know what is, in your opinion, the best way to go. I will put a few thousands every time but building up a file of several Gbytes. I do not want to put into memory all previous data each time. Also I cannot wait until the program finishes, I must save partial results periodically. Thanks, any help will be appreciated Juan

6 9

Masking an array with another array
by Gökhan SEVER 12 Aug '09

12 Aug '09

Hello, Could you please give me some hints about how to mask an array using another arrays like in the following example. In [14]: a = arange(5) In [15]: a Out[15]: array([0, 1, 2, 3, 4]) and my secondary array is "b" In [16]: b = array([2,3]) What I want to do is to mask a with b values and get an array of: array([False, False, True, True, False], dtype=bool) That is just an manually created array. I still don't know how to do this programmatically in Pythonic fashion or numpy's masked array functions. Thank you. Gökhan

6 10

identity
by Lars Bittrich 12 Aug '09

12 Aug '09

Hi, a colleague made me aware of a speed issue with numpy.identity. Since he was using numpy.diag(numpy.ones(N)) before, he expected identity to be at least as fast as diag. But that is not the case. We found that there was a discussion on the list (July, 20th; "My identity" by Keith Goodman). The presented solution was much faster. Someone wondered if the change was already made in the svn. But I got something different: In [1]:import numpy In [2]:numpy.__version__ Out[2]:'1.4.0.dev7301' In [3]:numpy.identity?? [...] def identity(n, dtype=None): """ [...] """ a = array([1]+n*[0],dtype=dtype) b = empty((n,n),dtype=dtype) # Note that this assignment depends on the convention that since the a # array is shorter than the flattened b array, then the a array will # be repeated until it is the appropriate size. Given a's construction, # this nicely sets the diagonal to all ones. b.flat = a return b instead of (mail by Keith Goodman): def myidentity(n, dtype=None): a = zeros((n,n), dtype=dtype) a.flat[::n+1] = 1 return a Did I look at the wrong place or is there a reason to keep the slow version of identity? Lars

5 6

NumPy C-API - copying from Fortran to C order
by Nichols A. Romero 12 Aug '09

12 Aug '09

Hi, I am working on a *very* simple Python interface to ScaLAPACK using the NumPy C-API. I am not using f2py at all. Simple question: How can I copy a C-order NumPy array into a Fortran-order NumPy array within the C-API? (This is trivial in Python, it is simply A = A.copy("Fortran")) I would like to do this with the minimal amount of memory use. The matrices are 2-d and rectangular, but not square. It looks like I use PyArray_CopyInto or PyArray_MoveInto, but I would need to create another array using PyArray_EMPTY(2, dims, NPY_DOUBLE, NPY_F_CONTIGUOUS) Thanks in advance for your assistance. -- Nichols A. Romero, Ph.D. Argonne Leadership Computing Facility Argonne, IL 60490 (630) 252-3441 (O) (630) 470-0462 (C)

1 0

Strange Error with NumPy Addendum
by Nanime Puloski 12 Aug '09

12 Aug '09

Thank you for your responses so far. What I also do not understand is why sin(2**64) works with the standard Python math module, but fails to do so with NumPy? To Robert Kern: Can't 2^64 be represented in C as a long double? It seems to work well on my machine.

3 5

memmap capability
by Tom Kuiper 12 Aug '09

12 Aug '09

If this appears twice, forgive me. I sent it previously (7:13 am PDT) via a browser interface to JPL's Office Outlook. I have doubts about this system. This time, from Iceweasel through our SMTP server. There are two things I'd like to do using memmap. I suspect that they are impossible but maybe I'm missing some subtlety. 1) I would like to append rows to a memmap array and have the modified array changed on disk also. 2) I would like to have the memory view of the array on disk change, i.e., modify the offset for an opened array. The only way I can think of involves opening and closing arrays repeatedly. Regards Tom Kuiper

2 1

How to preserve number of array dimensions when taking a slice?
by Dr. Phillip M. Feldman 12 Aug '09

12 Aug '09

I'd like to be able to make a slice of a 3-dimensional array, doing something like the following: Y= X[A, B, C] where A, B, and C are lists of indices. This works, but has an unexpected side-effect. When A, B, or C is a length-1 list, Y has fewer dimensions than X. Is there a way to do the slice such that the number of dimensions is preserved, i.e., I'd like Y to be a 3-dimensional array, even if one or more dimensions is unity. Is there a way to do this? -- View this message in context: http://www.nabble.com/How-to-preserve-number-of-array-dimensions-when-takin… Sent from the Numpy-discussion mailing list archive at Nabble.com.

3 2

is this a chararray decode bug
by josef.pktd＠gmail.com 12 Aug '09

12 Aug '09

(copied from the lengthy unicode thread in scipy-dev, so it doesn't get lost) this looks like a bug ? or is it a known limitation that chararrays cannot be 0-d >>> b0= np.array(u'\xe9','<U1').view(np.chararray) >>> print b0.encode('cp1252') Traceback (most recent call last): File "<pyshell#47>", line 1, in <module> print b0.encode('cp1252') File "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py", line 217, in encode return self._generalmethod('encode', broadcast(self, encoding, errors)) File "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py", line 162, in _generalmethod newarr[:] = res ValueError: cannot slice a 0-d array Josef

1 0

Question about bounds checking
by Rich E 12 Aug '09

12 Aug '09

Dear all, I am having a few issues with indexing in numpy and wondered if you could help me out. If I define an array a = zeros(( 4)) a array([ 0., 0., 0., 0.]) Then I try and reference a point beyond the bounds of the array a[4] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index out of bounds but if I use the slicing format to reference the point I get a[0:4] array([ 0., 0., 0., 0.]) a[0:10] array([ 0., 0., 0., 0.]) it returns a[ 0 : 3 ], with no error raised. If I then ask for the shape of the array, I get a.shape (4,) but if I use a[0:3].shape (3,) which is one less than I would have expected. a[0:4].shape (4,) This is numpy 1.2.1 on python 2.5 Thanks in advance for you help, Rich

3 2

Howto create a record array from arrays without copying their data
by Ralph Heinkel 12 Aug '09

12 Aug '09

Hi, I'm creating (actually calculating) a set of very large 1-d arrays (vectors), which I would like to assemble into a record array so I can access the data row-wise. Unfortunately it seems that all data of my original 1-d arrays are getting copied in memory during that process. Is there a way to get around that? Basically what I do is: arr1 = numpy.array([1, 4, 5], dtype=int) arr2 = numpy.array([5.5, 6.6, 9.9], dtype=float) recarray = numpy.core.rec.fromarrays([arr1, arr2], names=['col1', 'col2']) When I now make a change in recarray like recarray.col1[0] = 5000 I cannot see the change in arr1. (So that's why I assume that the data is copied). Also in the numy book it tells that would be copied. Thanks, Ralph

3 2