Mailman 3 January 2012 - NumPy-Discussion

Fix for ticket #1973
by Charles R Harris 25 Jan '12

25 Jan '12

I've put up a pull request for a fix to ticket #1973. Currently the fix simply propagates the maskna flag when the *.astype method is called. A more complicated option would be to add a maskna keyword to specify whether the output is masked or not or propagates the type of the source, but that seems overly complex to me. Thoughts? Chuck

3 5

Unexpected behavior with np.min_scalar_type
by Kathleen M Tacina 25 Jan '12

25 Jan '12

I was experimenting with np.min_scalar_type to make sure it worked as expected, and found some unexpected results for integers between 2**63 and 2**64-1. I would have expected np.min_scalar_type(2**64-1) to return uint64. Instead, I get object. Further experimenting showed that the largest integer for which np.min_scalar_type will return uint64 is 2**63-1. Is this expected behavior? On python 2.7.2 on a 64-bit linux machine: >>> import numpy as np >>> np.version.full_version '2.0.0.dev-55472ca' >>> np.min_scalar_type(2**8-1) dtype('uint8') >>> np.min_scalar_type(2**16-1) dtype('uint16') >>> np.min_scalar_type(2**32-1) dtype('uint32') >>> np.min_scalar_type(2**64-1) dtype('O') >>> np.min_scalar_type(2**63-1) dtype('uint64') >>> np.min_scalar_type(2**63) dtype('O') I get the same results on a Windows XP machine running python 2.7.2 and numpy 1.6.1. Kathy

3 2

einsum evaluation order
by Søren Gammelmark 24 Jan '12

24 Jan '12

Dear all, I was just looking into numpy.einsum and encountered an issue which might be worth pointing out in the documentation. Let us say you wish to evaluate something like this (repeated indices a summed) D[alpha, alphaprime] = A[alpha, beta, sigma] * B[alphaprime, betaprime, sigma] * C[beta, betaprime] with einsum as einsum('abs,cds,bd->ac', A, B, C) then it is not exactly clear which order einsum evaluates the contractions (or if it does it in one go). This can be very important since you can do it in several ways, one of which has the least computational complexity. The most efficient way of doing it is to contract e.g. A and C and then contract that with B (or exchange A and B). A quick test on my labtop says 2.6 s with einsum and 0.13 s for two tensordots with A and B being D x D x 2 and C is D x D for D = 96. This scaling seems to explode for higher dimensions, whereas it is much better with the two independent contractions (i believe it should be O(D^3)).For D = 512 I could do it in 5 s with two contractions, whereas I stopped waiting after 60 s for einsum (i guess einsum probably is O(D^4) in this case). I had in fact thought of making a function similar to einsum for a while, but after I noticed it dropped it. I think, however, that there might still be room for a tool for evaluating more complicated expressions efficiently. I think the best way would be for the user to enter an expression like the one above which is then evaluated in the optimal order. I know how to do this (theoretically) if all the repeated indices only occur twice (like the expression above), but for the more general expression supported by einsum I om not sure how to do it (haven't thought about it). Here I am thinking about stuff like x[i] = a[i] * b[i] and their more general counterparts (at first glance this seems to be a simpler problem than full contractions). Do you think there is a need/interest for this kind of thing? In that case I would like the write it / help write it. Much of it, I think, can be reduced to decomposing the expression into existing numpy operations (e.g. tensordot). How to incorporate issues of storage layout etc, however, I have no idea. In any case I think it might be nice to write explicitly how the expression in einsum is evaluated in the docs. Søren Gammelmark PhD-student Department of Physics and Astronomy Aarhus University

2 1

bug in numpy.mean() ?
by K.-Michael Aye 24 Jan '12

24 Jan '12

I know I know, that's pretty outrageous to even suggest, but please bear with me, I am stumped as you may be: 2-D data file here: http://dl.dropbox.com/u/139035/data.npy Then: In [3]: data.mean() Out[3]: 3067.0243839999998 In [4]: data.max() Out[4]: 3052.4343 In [5]: data.shape Out[5]: (1000, 1000) In [6]: data.min() Out[6]: 3040.498 In [7]: data.dtype Out[7]: dtype('float32') A mean value calculated per loop over the data gives me 3045.747251076416 I first thought I still misunderstand how data.mean() works, per axis and so on, but did the same with a flattenend version with the same results. Am I really soo tired that I can't see what I am doing wrong here? For completion, the data was read by a osgeo.gdal dataset method called ReadAsArray() My numpy.__version__ gives me 1.6.1 and my whole setup is based on Enthought's EPD. Best regards, Michael

7 8

installing matplotlib in MacOs 10.6.8.
by Alex Ter-Sarkissov 24 Jan '12

24 Jan '12

hi everyone, I run python 2.7.2. in Eclipse (recently upgraded from 2.6). I have a problem with installing matplotlib (I found the version for python 2.7. MacOs 10.3, no later versions). If I run python in terminal using arch -i386 python, and then from matplotlib.pylab import * and similar stuff, everything works fine. If I run python in eclipse or just without arch -i386, I can import matplotlib as from matplotlib import * but actually nothing gets imported. If I do it in the same way as above, I get the message no matching architecture in universal wrapper which means there's conflict of versions or something like that. I tried reinstalling the interpreter and adding matplotlib to forced built-ins, but nothing helped. For some reason I didn't have this problem with numpy and tkinter. Any suggestions are appreciated.

3 2

'Advanced' save and restore operation
by Emmanuel Mayssat 24 Jan '12

24 Jan '12

After having saved data, I need to know/remember the data dtype to restore it correctly. Is there a way to save the dtype with the data? (I guess the header parameter of savedata could help, but they are only available in v2.0+ ) I would like to save several related structured array and a dictionary of parameters into a TEXT file. Is there an easy way to do that? (maybe xml file, or maybe archive zip file of other files, or ..... ) Any recommendation is helpful. Regards, -- Emmanuel

4 3

Course "Python for Scientists and Engineers" in Chicago
by Mike Müller 24 Jan '12

24 Jan '12

Course "Python for Scientists and Engineers" in Chicago ======================================================= There will be a comprehensive Python course for scientists and engineers in Chicago end of February / beginning of March 2012. It consists of a 3-day intro and a 2-day advanced section. Both sections can be taken separately or combined. More details below and here: http://www.dabeaz.com/chicago/science.html Please let friends or colleagues who might be interested in such a course know about it. 3-Day Intro Section ------------------- - Overview of Scientific and Technical Libraries for Python. - Numerical Calculations with NumPy - Storage and Processing of Large Amounts of Data - Graphical Presentation of Scientific Data with matplotlib - Object Oriented Programming for Scientific and Technical Projects - Open Time for Problem Solving 2-Day Advanced Section ---------------------- - Extending Python with Other Languages - Unit Testing - Version Control with Mercurial The Details ----------- The course is hosted by David Beazley (http://www.dabeaz.com). Date: Feb 27 - Mar 2, 2012 Location: Chicago, IL, USA Trainer: Mike Müller Course Language: English Link: http://www.dabeaz.com/chicago/science.html

1 0

Strange error raised by scipy.special.erf
by Nadav Horesh 24 Jan '12

24 Jan '12

With N.seterr(all='raise'): >>> from scipy import special >>> import scipy >>> special.erf(26.6) 1.0 >>> scipy.__version__ '0.11.0.dev-81dc505' >>> import numpy as N >>> N.seterr(all='raise') {'over': 'warn', 'divide': 'warn', 'invalid': 'warn', 'under': 'ignore'} >>> special.erf(26.5) 1.0 >>> special.erf(26.6) Traceback (most recent call last): File "<pyshell#7>", line 1, in <module> special.erf(26.6) FloatingPointError: underflow encountered in erf >>> special.erf(26.7) 1.0 What is so special in 26.6? I have this error also with previous versions of scipy Nadav.

2 2

Working with MATLAB
by Jaidev Deshpande 23 Jan '12

23 Jan '12

Dear List, I frequently work with MATLAB and it is necessary for me many a times to adapt MATLAB codes for NumPy arrays. While for most practical purposes it works fine, I think there might be a lot of 'under the hood' things that I might be missing when I make the translations from MATLAB to Python. Are there any 'best practices' for working on this transition? Thanks

1 1

Saving and loading a structured array from a TEXT file
by Emmanuel Mayssat 23 Jan '12

23 Jan '12

Is there a way to save a structured array in a text file? My problem is not so much in the saving procedure, but rather in the 'reloading' procedure. See below In [3]: import numpy as np In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')]) In [5]: r.tofile('toto.txt',sep='\n') bash-4.2$ cat toto.txt ('1', 1, 1.0) ('1', 1, 1.0) ('1', 1, 1.0) In [7]: r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/cls1fs/clseng/10/<ipython-input-7-b07ba265ede7> in <module>() ----> 1 r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype) ValueError: Unable to read character files of that array type -- Emmanuel

2 2