Mailman 3 November 2007 - NumPy-Discussion

Numpy matrix and Blitz Matrix
by Paolo Tenconi 22 Nov '07

22 Nov '07

Hi Everyone, *) numpy arrays are automatically converted to blitz arrays. That's fine. *) I need to work with blitz Matrix objects, but I noticed that numpy matrix objects are not converted to them and I get a compilation error too 3) Does someone on the list succeeded doing that? Or is there a workaround to create quickly a blitz matrix object from a numpy 2D array or numpy-matrix without wasting time in casting operations? It would be nice having an automatic translation of numpy.matrix to blitz Matrix as matrices are used frequently in scientific computing. Any suggestion or help would be very glad. Paolo #------------------ EXAMPLE CODE ----------------------------------- import numpy import scipy #Dot product with arrays (it works) x=numpy.array([[1,2,],[3,4]]) y=numpy.zeros((2,2)) scipy.weave.inline("""y=x*x;""",['x','y'],type_converters=weave.converters.blitz,compiler='gcc',force=1) #Matrix multiplication with matrix (gives compilation error) X=numpy.matlib.mat('1 2; 3 4') Y=numpy.matlib.zeros((2,2)) scipy.weave.inline("""Y=X*X;""",['X','Y'],type_converters=weave.converters.blitz,compiler='gcc') #--------------------------------------------------------------------

1 0

Re: [Numpy-discussion] numpy : your experiences?
by Jan Strube 22 Nov '07

22 Nov '07

> >> a) Can you guys tell me briefly about the kind of problems you are >> tackling with numpy and scipy? > > I'm using python with numpy,scipy, pytables and matplotlib for data > analysis in the field of high energy particle physics. Most of the > work is histograming millions of events, fitting functions to the > distributions or applying cuts to yield optimized signal/background > ratios. I often use the random number and optimization facilities for > these purposes. > Cool! Me, too! In addition I am using Paida (http:// paida.sourceforge.net) for histogramming. I am interfacing that with matplotlib for the plotting, though. > Most of my colleagues use ROOT (root.cern.ch) which has also a python > binding, however, I love the simplicity of numpy's ufuncs and indexing > capabilities, which makes the code much denser and readable. In addition to more reliable in my experience. > In particular for the simulation yes, depending on the level of detail > of course. But only parts, eg. random number generation for certain > distributions had to be coded in C/C++. > Are you saying you extended the scipy/numpy tools for this? Do you think it would make sense to put some of that stuff on the wiki? > > Cheers! Bernhard Cheers, Jan

2 1

Re: [Numpy-discussion] OT: A Way to Approximate and Compress a 3DSurface
by Nadav Horesh 21 Nov '07

21 Nov '07

Wouldn't a random or regular subsampling of the set will do the job? For data interpolation: 2D-Delaunay triangulation based method (I think you can find one in the scipy cookbook). Nadav. -----Original Message----- From: numpy-discussion-bounces(a)scipy.org on behalf of Geoffrey Zhu Sent: Tue 20-Nov-07 19:50 To: Discussion of Numerical Python Subject: [Numpy-discussion] OT: A Way to Approximate and Compress a 3DSurface Hi Everyone, This is off topic for this mailing list but I don't know where else to ask. I have N tabulated data points { (x_i, y_i, z_i) } that describes a 3D surface. The surface is pretty "smooth." However, the number of data points is too large to be stored and manipulated efficiently. To make it easier to deal with, I am looking for an easy method to compress and approximate the data. Maybe the approximation can be described by far fewer number of coefficients. If you can give me some hints about possible numpy or non-numpy solutions or let me know where is better to ask this kind of question, I would really appreciate it. Many thanks, Geoffrey _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

4 3

numpy distutils patch
by Jarrod Millman 21 Nov '07

21 Nov '07

Hey, I was looking at numpy/distutils/ccompiler.py and noticed that it has a fix for distutils.util.split_quoted. Here is the relevant code from split_quoted in numpy.distutils.ccompiler: ----------------------------------- def split_quoted(s): <snip> if _has_white_re.search(s[beg+1:end-1]): s = s[:beg] + s[beg+1:end-1] + s[end:] pos = m.end() - 2 else: # Keeping quotes when a quoted word does not contain # white-space. XXX: send a patch to distutils pos = m.end() <snip> ----------------------------------- Here is the relevant code from split_quoted in distutils.util: ----------------------------------- def split_quoted(s): <snip> s = s[:beg] + s[beg+1:end-1] + s[end:] pos = m.end() - 2 <snip> ----------------------------------- Does anyone know if a patch was ever submitted upstream? If not, is there any reason that a patch shouldn't be submitted? Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

2 3

Building installers using bdist_mpkg on Leopard
by Chris 21 Nov '07

21 Nov '07

There appears to be a bug in bdist_mpkg on Leopard: it requires an "admin" group that does not exist by default. The mpkg build dies with a ValueError saying that the group does not exist. Is it best just to add this group, or should the code be changed?

2 1

OT: A Way to Approximate and Compress a 3D Surface
by Geoffrey Zhu 21 Nov '07

21 Nov '07

Hi Everyone, This is off topic for this mailing list but I don't know where else to ask. I have N tabulated data points { (x_i, y_i, z_i) } that describes a 3D surface. The surface is pretty "smooth." However, the number of data points is too large to be stored and manipulated efficiently. To make it easier to deal with, I am looking for an easy method to compress and approximate the data. Maybe the approximation can be described by far fewer number of coefficients. If you can give me some hints about possible numpy or non-numpy solutions or let me know where is better to ask this kind of question, I would really appreciate it. Many thanks, Geoffrey

4 3

ndim argsort question
by Vincent Schut 20 Nov '07

20 Nov '07

Hi, I'm getting a bit lost in numpy's fancy indexing, using argsort on a slice of a 4d arrays, and then using the resulting indices to sort other slices of that same array, preferably in one go. I have a data[d, b, y, x] array (for the curious: date, band, y, x: satellite image) Then I create a sorting index from band 0: idx = data[:, 0, :, :].argsort(axis=0) Now I want to sort all 7 bands accoring to this index, without looping, or at least avoid looping as far as possible. Something like sortedData = data[idx], but then for each 'b'. However, I cant get multidim indexing using the argsort result to work. Any hints? Hard to explain. I hope someone understands it, otherwise please let me know and I'll try to refrase :) Cheers, Vincent Schut.

1 0

Exception handling in ufuncs
by Bock, Oliver BGI SYD 20 Nov '07

20 Nov '07

Is it possible to trap non-floating point exceptions? For example, I store strings in ndarrays as objects, but I would like the result of invalid operations to be None. from numpy import * a = array(["a", "b"], dtype=object) b = array(["A", None], dtype=object) print a + b Traceback (most recent call last): File "C:\workspace\AlphaExplorer\ae1\test\_fiddle.py", line 225, in ? print a + b TypeError: cannot concatenate 'str' and 'NoneType' objects I have tried seterr() and seterrcall(), but these are not triggered (which did not surprise me because the documentation says they are used for floating point exceptions). Of course I can always write my own ufunc, but if a better way exists... def f(a, b): if a is None or b is None: return None return a + b print vectorize(f, otypes=[object])(a, b) -- This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.

1 0

SegFault/double free with simple array mask operation
by Achim Gaedke 18 Nov '07

18 Nov '07

Hello everybody! Please have a look at the program below: # start import numpy t_array=numpy.ones(2048, dtype=numpy.float32) sinc_array=numpy.array((len(t_array),),dtype=numpy.float32) sinc_array[(t_array > 0.)]=1.0 # end If you execute this program, it crashes with Segmentation Fault or *** glibc detected *** python: double free or corruption (out): 0x081fe470 *** It depends on the circumstances, which error occurs, e.g. you must quit your interpreter if you are in interactvie mode. Obviously numpy.array() should be numpy.zeros() or numpy.empty() .... But this program should not crash with a core dump. Used Linux Versions are: Debian Testing with numpy 1.0.3, Debian Stable with numpy 1.0.1, Ubuntu Linux 6.10 with numpy 1.0 Also numpy-1.0.4 crashes. Yours, Achim

5 7

[ANN] Release of the first PyTables video
by Ivan Vilata i Balaguer 18 Nov '07

18 Nov '07

===================================== Release of the first PyTables video ===================================== `Carabos <http://www.carabos.com/>`_ is very proud to announce the first of a series of videos dedicated to introducing the main features of PyTables to the public in a visual and easy to grasp manner. http://www.carabos.com/videos/pytables-1-intro `PyTables <http://www.pytables.org/>`_ is a Free/Open Source package designed to handle massive amounts of data in a simple, but highly efficient way, using the HDF5 file format and NumPy data containers. This first video is an introductory overview of PyTables, covering the following topics: * HDF5 file creation * the object tree * homogeneous array storage * natural naming * working with attributes With a running length of little more than 10 minutes, you may sit back and watch it during any short break. More videos about PyTables will be published in the near future. Stay tuned on www.pytables.org for the announcement of the new videos. We would like to hear your opinion on the video so we can do it better the next time. We are also open to suggestions for the topics of future videos. Best regards, :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""

2 2