Mailman 3 September 2010 - NumPy-Discussion

restrictions on fancy indexing
by Neal Becker Sept. 17, 2010

Sept. 17, 2010

It's nice I can do: f = np.linspace (0, 1, 100) u[f<.1] = 0 cool, this seems to work also: u[np.abs(f)<.1] = 0 cool! But exactly what kind of expressions are possible here? Certainly not arbitrary code.

4 3

buffer arg to ndarray
by Neal Becker Sept. 17, 2010

Sept. 17, 2010

np.ndarray description says: buffer : object exposing buffer interface It's nice that it works with mmap: b = mmap.mmap (...) u = np.ndarray (buffer=b ...) but you wouldn't know it from the above description. It doesn't look to me that the object returned by mmap exposes the buffer interface, does it? In [26]: dir(b) Out[26]: ['__add__', '__class__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__getslice__', '__hash__', '__init__', '__len__', '__mul__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'find', 'flush', 'move', 'read', 'read_byte', 'readline', 'resize', 'rfind', 'seek', 'size', 'tell', 'write', 'write_byte']

2 1

Re: [Numpy-discussion] Viewer for 2D Numpy arrays (GUI)
by Joe Harrington Sept. 17, 2010

Sept. 17, 2010

There is a stand-alone program in wide use by astronomers that does 2D floating-point image display called ds9. It has a 2-way numpy interface and a rich set of image viewing/annotating features, including interactive colormapping, zoom/pan of very large images, n-up displays, movies, live row and column plots, etc. See http://hea-www.harvard.edu/RD/ds9/ There is also the lighter, native-Python ntv routine, which uses matplotlib: http://planets.ucf.edu/resources/open-source-software/ http://code.google.com/p/ntv/ This is a new package that can display 2D floating-point arrays as images and do some astro-specific stuff like point-source centering and extraction (aka photometry) from images. The author, Nate Lust, is eager for contributions, such as help in adding features and improving the web pages for it. It would probably be straightforward to add a table-viewing/editing feature to this, such that you would click in an image and get a spreadsheet-like interface centered where you clicked. --jh-- Date: Fri, 17 Sep 2010 10:46:51 +0530 From: Mayank P Jain <mayankpj(a)gmail.com> Subject: [Numpy-discussion] Viewer for 2D Numpy arrays (GUI) To: numpy-discussion <numpy-discussion(a)scipy.org> Message-ID: <AANLkTimqv8jkMRN-i+bcW48ZdXctN5O_kC8TgDCCR0MR(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Currently I am exporting them to csv files, but I wonder if there is a viewer that can be used with native numpy array files to view and preferably modify the 2D arrays. Any help would be appreciated. Regards Mayank P Jain V R TechNiche Transportation Modeler/Planner Phone: +91 901 356 0583

1 0

loadtxt stop
by Neil Hodgson Sept. 17, 2010

Sept. 17, 2010

Hi, I been looking around and could spot anything on this. Quite often I want to read a homogeneous block of data from within a file. The skiprows option is great for missing out the section before the data starts, but if there is anything below then loadtxt will choke. I wondered if there was a possibility to put an endmarker= ? For example, if I want to load text from a large! file that looks like this header line header line 1 2.0 3.0 2 4.5 5.7 ... 500 4.3 5.4 END more headers more headers 1 2.0 3.0 3.14 1.1414 2 4.5 5.7 1.14 3.1459 ... 500 4.3 5.4 0.000 0.001 END Then I can use skiprows=2, but loadtxt will choke when it gets to 'END'. To read t

1 0

Silent DeprecationWarnings under Python 2.7 onwards
by Peter Sept. 17, 2010

Sept. 17, 2010

Hi all, NumPy currently makes extensive use of the DeprecationWarning class to alert users when some feature is going to be withdrawn. However, as of Python 2.7, the DeprecationWarning is silent by default, see: http://docs.python.org/library/warnings.html#updating-code-for-new-versions… This makes sense to me for deprecation warnings from Python itself or the standard library - after all, Python 2.7 is the last of the Python 2.x series. However, I believe that a silent DeprecationWarning is bad news for third party libraries like NumPy where many end users are coders and we want them to see these warnings by default. Is anyone else concerned about this? A typical NumPy user (on Python 2.7+ or Python 3.2+) may never see the warnings (because they have to deliberately turn them on), the first they'll know about it is when they upgrade to a new release and their code suddenly stops working. Potentially NumPy would have to introduce its own NumPy specific DeprecationWarning warning class, and use that. Peter

3 4

Re-pinging two old issues
by Friedrich Romstedt Sept. 16, 2010

Sept. 16, 2010

I'd really love to see these two things fixed: bugreport1.py When comparing objects using the numpy mechanism for == or != (only those apparently), everything is cast to an ndarray, even if this results in an dtype=numpy.object array. E.g. for addition, this doesn't happen. Installing loggers ... Creating array a: ... [1 2] Calling a + 1 ... args = (array([1, 2]), 1) Calling a == 1 ... args = (array([1, 2]), array(1)) Calling a == X() ... args = (array([1, 2]), array(<__main__.X instance at 0x716300>, dtype=object)) bugreport2.py This one is really crucial to me. I have to overload the operators with callables. But when doing so, at least my Python crashes at exit with different outcome. Sometimes it even does not crash. I'd like to get some feedback from other users whether they can confirm the crashiness or if it is specific to my numpy. I'm using numpy 1.4.1, but I strongly believe this things are widely apart from the regions where fixes took place. My system is Mac OS X 10.6 Intel-based, 64bit, with a 32bit Python, and self-compiled numpy, and 8 GB RAM, ca. 1/3 free, and rebooting is rare for me. Friedrich

1 0

GSoC Article for PSF and my blog
by Mike Driscoll Sept. 16, 2010

Sept. 16, 2010

Hi, I am working on an article for the Python Software Foundation's blog, http://pyfound.blogspot.com/, about the various Python projects that were worked on during this year's Google Summer of Code program. They want me to write up something about what projects were worked on and what the results were. I found your project information here: http://wiki.python.org/moin/SummerOfCode/2010 Anyway, since the PSF blog article will be brief, I thought I would also write up a longer article about your projects on my personal blog as well. The information I found on the Python wiki page was pretty brief, so I would appreciate it if you could tell me the following: 1) What was worked on during the GSoC project 2) How many students helped with a guess at how many hours were put in 3) Your experiences being a mentor If possible, I would like the student's perspective too. Feel free to forward my information to them. Also feel free to opt out and I won't write anything more than the already public info I can find. Thanks a lot for your help! -- ----------------- Mike Driscoll Blog: http://blog.pythonlibrary.org

1 0

Two questions on indexing
by Mark Fenner Sept. 16, 2010

Sept. 16, 2010

One method of using indices seems to be as follows: In [19]: a = N.array(range(6)).reshape(3,2) In [20]: i = N.indices([3,2]) In [21]: r,c = i In [22]: a[r,c] Out[22]: array([[0, 1], [2, 3], [4, 5]]) In [23]: a[tuple(i)] Out[23]: array([[0, 1], [2, 3], [4, 5]]) *****Question 1:***** For using the results of argmax, how would one "pad" the indices returned by argmax to be usable as follows (note, I resorted to method I found on numpy-discussion ... google "take_axis" ... that revolves around rolling the desired axis to the first position and then using the indices ... I'm curious how to make something like what I have below work). Incidentally, the number of dimensions can be arbitrary (for my needs): x = N.array([[2, 3], [1, 5]]) for a in [0, 1]: indices = N.argmax(x, axis=a) # perhaps something similar to the following? # grid = N.indices(x.shape) # grid[a] = indices # <--- this is the idea, but it fails to get the necessary result # usable = tuple(grid) assert x[usableIndices] == x.max(axis=a) # or similar *****Question 2:***** A separate question. Suppose I have a slice for indexing that looks like: [:, :, 2, :, 5] How can I get an indexing slice for all OTHER dimension values besides those specified. Conceptually, something like: [:, :, all but 2, :, all but 5] Incidentally, the goal is to construct a new array with all those "other" spots filled in with zero and the specified spots with their original values. Would it be easier to construct a 0-1 indicator array with 1s in the [:,:,2,:,5] positions and multiply it out? Humm, I may have just answered my own question. For argument sake, how would you do it with indexing/slicing? I suppose one develops some intuition as one gains experience with numpy with regards to when to (1) use clever matrix ops and when to (2) use clever slicing and when to (3) use a combination of both. Thanks, Mark

3 2

Can we freeze the subversion repository and move to github this week?
by Travis Oliphant Sept. 16, 2010

Sept. 16, 2010

Are we ready to do this yet? I know there were some outstanding questions. Are there major concerns remaining? Or, is it just picking a time to freeze subversion when someone like David or Pauli who know how to move the repository to github correctly a time to do it. Thanks, -Travis

11 30

weighted mean; weighted standard error of the mean (sem)
by cpblpublic Sept. 16, 2010

Sept. 16, 2010

I am looking for some reaally basic statistical tools. I have some sample data, some sample weights for those measurements, and I want to calculate a mean and a standard error of the mean. Here are obvious places to look: numpy scipy.stats statsmodels It seems to me that numpy's "mean" and "average" functions have their names backwards. That is, often a mean is defined more generally than average, and includes the possibility of weighting, but in this case it is "average" that has a weights argument. Can these functions be merged/renamed/deprecated in the future? It's clear to me that "mean" should allow for weights. None of these modules, above, offer standard error of the mean which incorporates weights. scipy.stats.sem() doesn't, and that's the closest thing. numpy's "var" doesn't allow weights. There aren't any weighted variances in the above modules. Again, are there favoured codes for these functions? Can they be incorporated appropriately in the future? Most immediately, I'd love to get code for weighted sem. I'll write it otherwise, but it might be crude and dumb... Thanks! Chris Barrington-Leigh UBC

5 7