Mailman 3 January 2009 - NumPy-Discussion

make latex in numpy/doc failed
by Nils Wagner 29 Jan '09

29 Jan '09

Hi all, a make latex in numpy/doc failed with ... Intersphinx hit: PyObject http://docs.python.org/dev/c-api/structures.html writing... Sphinx error: too many nesting section levels for LaTeX, at heading: numpy.ma.MaskedArray.__lt__ make: *** [latex] Fehler 1 I am using sphinxv0.5.1 BTW, make html works fine here. Nils

2 1

optimise operation in array with datetime objects
by Timmie 29 Jan '09

29 Jan '09

Hello, I have an array of datetime objects. What is the most efficient way of creating a new array with only the hours or minutes out of it? Here is an example: ### imports import numpy as np import datetime as dt ### create some data d = dt.datetime.now() dates_li = [] count = 0 for i in range(0, 24): count = count +1 fact = count * 3600 date_new = d + dt.timedelta(0, fact) print date_new dates_li.append(date_new) ### the array with datetime objects dates_array = np.array(dates_li) ### this is the loop I would like to optimize: ### looping over arrays is considered inefficient. ### what could be a better way? hours_array = dates_array.copy() for i in range(0, dates_array.size): hours_array[i] = dates_array[i].hour hours_array = hours_array.astype('int') #### Thanks in advance for any hints! Kind regards, Timmie

2 3

recfunctions.stack_arrays
by Ryan May 28 Jan '09

28 Jan '09

Pierre (or anyone else who cares to chime in), I'm using stack_arrays to combine data from two different files into a single array. In one of these files, the data from one entire record comes back missing, which, thanks to your recent change, ends up having a boolean dtype. There is actual data for this same field in the 2nd file, so it ends up having the dtype of float64. When I try to combine the two arrays, I end up with the following traceback: data = stack_arrays((old_data, data)) File "/home/rmay/.local/lib64/python2.5/site-packages/metpy/cbook.py", line 260, in stack_arrays output = ma.masked_all((np.sum(nrecords),), newdescr) File "/home/rmay/.local/lib64/python2.5/site-packages/numpy/ma/extras.py", line 79, in masked_all a = masked_array(np.empty(shape, dtype), ValueError: two fields with the same name Which is unsurprising. Do you think there is any reasonable way to get stack_arrays() to find a common dtype for fields with the same name? Or another suggestion on how to approach this? If you think coercing one/both of the fields to a common dtype is the way to go, just point me to a function that could figure out the dtype and I'll try to put together a patch. Thanks, Ryan P.S. Thanks so much for your work on putting those utility functions in recfunctions.py It makes it so much easier to have these functions available in the library itself rather than needing to reinvent the wheel over and over. -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma

2 3

numpy ndarray questions
by Jochen 28 Jan '09

28 Jan '09

Hi all, I just wrote ctypes bindings to fftw3 (see http://projects.scipy.org/pipermail/scipy-user/2009-January/019557.html for the post to scipy). Now I have a couple of numpy related questions: In order to be able to use simd instructions I create an ndarray subclass, which uses fftw_malloc to allocate the memory and fftw_free to free the memory when the array is deleted. This works fine for inplace operations however if someone does something like this: a = fftw3.AlignedArray(1024,complex) a = a+1 a.ctypes.data points to a different memory location (this is actually an even bigger problem when executing fftw plans), however type(a) still gives me <class 'fftw3.planning.AlignedArray'>. I think I understand the reason for this is that python does a copy internally (it does not call the __copy__ or __deepcopy__ methods?). Is there a way that I get a different object type? Or even better is there a way to prevent operations like a=a+1 or make them automatically in-place operations? I realise that I could change the __add__ ... methods but that would also prevent b=a+1 operations. My second comment is with respect to the documentation for numpy.ctypeslib.ndpointer The documentation says: An ndpointer instance is used to describe an ndarray in restypes and argtypes specifications. however if I specify a restype to a function e.g. clib.malloc.restype = ctypeslib.ndpointer(flags='contiguous,aligned',shape=shape,dtype=dtype) Calling clib.malloc(1024) results in a TypeError: TypeError: default __new__ takes no parameters Am I correct in assuming that the documentation is incorrect and ndpointer can only be used for argtypes? Or am I missing something? Cheers Jochen

4 18

Slicing and structured arrays question
by Hanno Klemm 27 Jan '09

27 Jan '09

Hi, I have the following question, that I could not find an answer to in the example list, or by googling: I have a record array with dtype such as: dtype([('times', '<f8'), ('sensors', '|S8'), ('prop1', '<f8'), ('prop2', '<f8'), ('prop3', '<f8'), ('prop4', '<f8')]) I would now like to calculate the mean and median for each of the properties 'props'. Is there a way to do this similarly to a conventional array with a[:,2:].mean(axis=0) or do I have to use a loop over the names of the properties? Thanks in advance, Hanno -- Hanno Klemm klemm(a)phys.ethz.ch

2 1

PyArray_Zeros
by Hanni Ali 27 Jan '09

27 Jan '09

Hi, I have been having trouble with the PyArray_Zeros/PyArray_ZEROS functions. I cannot seem to create an array using these functions. resultArray = PyArray_ZEROS(otherArray->nd, otherArray->dimensions, NPY_DOUBLE, 0); I would have thought this would have created an array the same shape as the otherArray, just filled with zeros... But I seem to get an error. Am I doing something obviously wrong? Hanni

1 0

Usage of numpy.where and pylab.find with strings
by Ariel Rokem 27 Jan '09

27 Jan '09

Hi - I am trying to find a string in a list with strings and have come across the following state of affairs: In [228]: subjects Out[228]: ['KAA', 'CCS', 'EJS', 'MNM', 'JHS', 'LJL', 'DVA', 'FCL', 'CNC', 'KFM', 'APM', 'GMC'] In [229]: subjects[0] Out[229]: 'KAA' In [230]: subjects[0] == 'KAA' Out[230]: True In [231]: np.where(subjects == 'KAA') Out[231]: () In [232]: pylab.find(subjects == 'KAA') Out[232]: array([], dtype=int32) It doesn't seem to matter if I make the list into an array: In [233]: np.array(subjects) Out[233]: array(['KAA', 'CCS', 'EJS', 'MNM', 'JHS', 'LJL', 'DVA', 'FCL', 'CNC', 'KFM', 'APM', 'GMC'], dtype='|S3') In [234]: pylab.find(subjects == 'KAA') Out[234]: array([], dtype=int32) In [235]: np.where(subjects == 'KAA') Out[235]: () What am I doing wrong? What does it mean that the dtype is IS3? Thanks a bunch -- Ariel

3 3

Bug with mafromtxt
by Ryan May 27 Jan '09

27 Jan '09

Pierre, I've found what I consider to be a bug in the new mafromtxt (though apparently it existed in earlier versions as well). If you have an entire column of data in a file that contains only masked data, and try to get mafromtxt to automatically choose the dtype, the dtype gets selected to be object type. In this case, I'd think the better behavior would be float, but I'm not sure how hard it would be to make this the case. Here's a test case: import numpy as np from StringIO import StringIO s = StringIO('1 2 3\n4 5 6\n') a = np.mafromtxt(s, missing='2,5', dtype=None) print a.dtype Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma

2 4

Re: [Numpy-discussion] Academic citation ?
by jh＠physics.ucf.edu 27 Jan '09

27 Jan '09

> What is the most up-to-date way to cite Numpy and Scipy in an academic > journal ? Cite our conference articles here: http://conference.scipy.org/proceedings/SciPy2008/index.html It would be nice if someone involved in the proceedings could post a bibtex on the citations page. And link the citations page to...something...easily navigated to from the front page. This brings up a related point: When someone goes to scipy.org, there is no way to navigate to conferences.scipy.org from scipy.org except by finding the link buried in the intro text. Ipython and all the whatever.scipy.org domains, except for docs.scipy.org, are completely absent; you have to know about them to find them. I don't even know where to find a complete list of these. They should all have a presence on at least the front page and maybe the navigation. --jh--

5 7

Array of matrices - Inverse and dot
by Jean-Baptiste Rudant 26 Jan '09

26 Jan '09

Hello, I would like to operate in an easy and efficient way (without python loop) with arrays of matrices. Suppose a and b are some arrays of N1*N2 matrices of size 3*3, I would like to calculate inv_a and dot_ab, which would be arrays of N1*N2 (3*3)-matrices, such as : inv_a[i, j] = np.linalg.inv(a[i, j]) dot_ab[i, j] = np.dot(a[i, j], b[i, j]) (where a and b could be : N1 = 5 N2 = 6 a = np.random((N1, N2, 3, 3) b = np.random((N1, N2, 3, 3) ). Thank you, (Sorry to ask the list : I think it is quite a basic stuff, but searching for "array of matrices" on google didn't help me so much.) Jean-Baptiste Rudant

4 4