Mailman 3 July 2009 - NumPy-Discussion

numpy subclass
by Jacopo 23 Jul '09

23 Jul '09

I am facing the opposite problem described here: http://docs.scipy.org/doc/numpy/user/basics.subclassing.html In short, I want to create a class Dummy, inherited from np.ndarray, which returns a plain array whenever an instance is sliced or viewed. I cannot figure out how. I would really appreciate your help, here is a chunk of code, what is commented was an attempt which didn’t work out. Thanks a lot, Jacopo p.s. I can be more wordy in explain the problem i just dont want to overwhelm you. import numpy as np class Dummy(np.ndarray): def __new__(cls, array): obj=array.view(cls) return obj def __array_finalize__(self, obj): #self=self.view(np.ndarray) #self=np.asarray(self) d=Dummy(np.ones(5)) d2=d[:3] print type(d2) # I wish this was np.array

3 3

String to integer array of ASCII values
by Peter 23 Jul '09

23 Jul '09

Dear all, I've looked over some of the documentation for creating an array, e.g. http://docs.scipy.org/doc/numpy/user/basics.creation.html#arrays-creation http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html However, I don't see an example quite like what I want to do. I want to be able to take a python string (e.g. "ABCDEF") and turn it into an array of the ASCII values (i.e. [65, 66, 67, 68, 69, 70] for this example). >>> import numpy >>> numpy.__version__ '1.1.1' I can get the result I want like this, but I would like a faster way: >>> numpy.array([ord(letter) for letter in "ABCDEF"]) array([65, 66, 67, 68, 69, 70]) I know in C that a string can been regarded as an array of unsigned integers - so I'd like to get NumPy to do that for me. I'm guessing there is a magic data type I can specify. Using "c" appears to mean characters which is close but isn't what I want: >>> numpy.array("ABCDEF", "c") array(['A', 'B', 'C', 'D', 'E', 'F'], dtype='|S1') I eventually found this works: >>> numpy.frombuffer("ABCDEF", numpy.byte) array([65, 66, 67, 68, 69, 70], dtype=int8) But why don't these work too? >>> numpy.array("ABCDEF", numpy.byte) Traceback (most recent call last): ... ValueError: setting an array element with a sequence. >>> numpy.fromiter("ABCDEF", numpy.byte, count=6) Traceback (most recent call last): ... ValueError: setting an array element with a sequence. So, is using frombuffer the only or best option? Thanks, Peter

2 4

Getting 95%/99% margin of ndarray
by Johannes Bauer 23 Jul '09

23 Jul '09

Hello list, is there some possibilty to get a p-dynamic of an array, i.e. if p=1 then the result would be (arr.min(), arr.max()), but if 0 < p < 1, then the result is so that the pth percentile of the picture is withing the range given? I cannot explain this very well, so please let me illustrate. Let's say we have an array: 0 0 0 0 1 2 3 4 5 2000 p = 0.9 -> (0, 5) This means that 90% of the pixels are within the [0; 5] range p = 0.5 -> (1, 5) p = 0.3 -> (2, 4) It it clear what I want? Is there the possibility to achieve that? If I have to implement it myself, how can I do that efficiently in Python? Kind regards, Johannes

4 6

Comparing the precision of dtypes?
by Hans Meine 23 Jul '09

23 Jul '09

Hi, Ullrich Köthe found an interesting way to compute a promoted dtype, given two arrays a and b: type = min(float32, a.dtype.type, b.dtype.type) How hackish is this? Is this likely to break on other platforms/numpy versions? Is there a better API for type promotion? Have a nice day, Hans

3 5

sorting and nans, timings.
by Charles R Harris 23 Jul '09

23 Jul '09

Hi All, I changed the sort routines to sort nans to the end and got some timings. Sorting 100000 random doubles 100 times yields: current nan version quicksort 1.17 sec 1.29 sec mergesort 1.37 sec 1.36 sec heapsort 1.83 sec 2.12 sec Curiously, mergesort doesn't seem to suffer at all. This is using x != x for nan detection, using npy_isnan is notably slower with my compiler (gcc 4.3.0). So, what do folks want to do about sorting nans? Chuck

2 2

Re: [Numpy-discussion] String manipulation
by David Goldsmith 22 Jul '09

22 Jul '09

By the way, Chris, what's it gonna take to get you contributing to the doc? ;-) DG --- On Wed, 7/22/09, Christopher Barker <Chris.Barker(a)noaa.gov> wrote: > From: Christopher Barker <Chris.Barker(a)noaa.gov> > Subject: Re: [Numpy-discussion] String manipulation > To: "Discussion of Numerical Python" <numpy-discussion(a)scipy.org> > Date: Wednesday, July 22, 2009, 11:33 AM > Stéfan van der Walt wrote: > > You cannot use view on an array scalar, so you'll have > to convert to > > an array first: > > > > x = x.reshape((1,)) > > x.view('S1') > > Bingo! > > Thanks, Stéfan. > > I don't know why one can't uwe view on a scalar -- there is > still a dat pointer isn't there?, and it can be reshaped. > Butno matter, now I have a one liner that splits a > string up into fixed fields, and returns a float array of > the numbers in those fields: > > np.array((line,)).view(dtype='S%i'%field_len).astype(np.float) > > I suspect this is faster than the list comp method, though > I haven't tested it -- I"ll leave that to someone that > actually has this problem to solve ;-) > > I've enclosed a bit of test code with a few other > approaches as well. > > -Chris > > > > > -- Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R > (206) 526-6959 voice > 7600 Sand Point Way NE (206) > 526-6329 fax > Seattle, WA 98115 > (206) 526-6317 main > reception > > Chris.Barker(a)noaa.gov > > -----Inline Attachment Follows----- > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

1 0

performance matrix multiplication vs. matlab
by David Paul Reichert 22 Jul '09

22 Jul '09

Hi all, I would be glad if someone could help me with the following issue: From what I've read on the web it appears to me that numpy should be about as fast as matlab. However, when I do simple matrix multiplication, it consistently appears to be about 5 times slower. I tested this using A = 0.9 * numpy.matlib.ones((500,100)) B = 0.8 * numpy.matlib.ones((500,100)) def test(): for i in range(1000): A*B.T I also used ten times larger matrices with ten times less iterations, used xrange instead of range, arrays instead of matrices, and tested it on two different machines, and the result always seems to be the same. Any idea what could go wrong? I'm using ipython and matlab R2008b. Thanks, David -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

19 68

Overloading numpy's ufuncs for better type coercion?
by Hans Meine 22 Jul '09

22 Jul '09

Hi! (This mail is a reply to a personal conversation with Ullrich Köthe, but is obviously of a greater concern. This is about VIGRA's new NumPy-based python bindings.) Ulli considers this behaviour of NumPy to be a bug: In [1]: a = numpy.array([200], numpy.uint8) In [2]: a + a Out[2]: array([144], dtype=uint8) However, this is well-known, often-discussed, and IMHO not really unexpected for computer programmers who ever worked with C-like languages (even Java has many such problems). Christian even said this is what he wants. OTOH, I agree that it is a missing feature that NumPy performs "coercion before the operation" (to be more precise: the temporary data type should be promoted from the operand types, and /then/ the coercion - which can also reduce the number of bits - should happen), to fix this strange behaviour: In [3]: numpy.add(a, a, numpy.empty((1, ), dtype = numpy.uint32)) Out[3]: array([144], dtype=uint32) Now, our opinions differ on how to deal with this - Ulli planned to overwrite (more or less) all ufuncs in vigranumpy in order to return float32 (which is sort of a common denominator and the type nearly all other vigranumpy functions should accept). I see two main disadvantages here: a) Choosing float32 seems to be arbitrary, and I'd like as much as possible of vigranumpy to be independent from VIGRA and its particular needs. I have seen so many requests (e.g. on the c++-sig mailing list) for *good* C++/boost- python <-> numpy bindings that it would be a pity IMO to add special cases for VIGRA by overloading __add__ etc. b) Also, I find it unexpected and undesirable to change the behaviour of such basic operations as addition on our ndarray-derived image types. IMO this brings the danger of new users being confused about the different behaviours, and even experienced vigranumpy users might eventually fall in the trap when dealing with plain ndarrays and our derived types side-by-side. Ideally, I'd like numpy to be "fixed" - I hope that the "only" obstacle is that someone needs to do it, but I am afraid of someone mentioning the term "backward-compatibility" (Ulli would surely call it "bug-compatibility" here ;-) ). But in the end, I wonder how bad this really is for the VIGRA. AFAICS, the main problem is that one needs to decide upon the pixel types for which to export algorithms (in most cases, we'll use just float32, at least for a start), and that when one loads images into arrays of the data type used in the image file, one will often end up with uint8 arrays which cannot be passed into many algorithms without an explicit conversion. However, is this really a bad problem? For example, the conversion would typically have to be performed only once (after loading), no? Then, why not simplify things further by adding a dtype= parameter to importImage()? This could even default to float32 then. Looking forward to your opinions, Hans

4 5

branches/datetime
by Peter Alexander 22 Jul '09

22 Jul '09

Hi all, I see life in a feature I've been dreaming about for years now. :-) I'm wondering how stable this branch is and if it's ready for playing with. I ask because I'm (once again) about to write an cython extension to process vectors of epochs and records of struct tm. Naturally, I'd love to be doing this right in numpy itself. Thanks, Pete

2 1

ANN: SciPy 2009 early registration ends today
by Jarrod Millman 22 Jul '09

22 Jul '09

Today is the last day to register for SciPy 2009 at the early bird rates. Please register (http://conference.scipy.org/to_register ) by the end of the day to take advantage of the reduced early registration rate. The conference schedule is available here: http://conference.scipy.org/schedule The special group rate for the Marriot Hotel is no longer available. However, there are a number of closer and less expensive choices still available: http://admissions.caltech.edu/visiting/accommodations I've been staying at the Vagabond Inn for the last several years: http://www.vagabondinn-pasadena-hotel.com/ It is within easy walking distance of the conference and has just been completely renovated. Rooms at the Vagabond start at $79/night. About the conference -------------------- SciPy 2009, the 8th Python in Science conference, will be held from August 18-23, 2009 at Caltech in Pasadena, CA, USA. The conference starts with two days of tutorials to the scientific Python tools. There will be two tracks, one for introduction of the basic tools to beginners, and one for more advanced tools. The tutorials will be followed by two days of talks. Both days of talks will begin with a keynote address. The first day’s keynote will be given by Peter Norvig, the Director of Research at Google; while, the second keynote will be delivered by Jon Guyer, a Materials Scientist in the Thermodynamics and Kinetics Group at NIST. The program committee will select the remaining talks from submissions to our call for papers. All selected talks will be included in our conference proceedings edited by the program committee. After the talks each day we will provide several rooms for impromptu birds of a feather discussions. Finally, the last two days of the conference will be used for a number of coding sprints on the major software projects in our community. For the 8th consecutive year, the conference will bring together the developers and users of the open source software stack for scientific computing with Python. Attendees have the opportunity to review the available tools and how they apply to specific problems. By providing a forum for developers to share their Python expertise with the wider commercial, academic, and research communities, this conference fosters collaboration and facilitates the sharing of software components, techniques, and a vision for high level language use in scientific computing. For further information, please visit the conference homepage: http://conference.scipy.org. Important Dates --------------- * Friday, July 3: Abstracts Due * Wednesday, July 15: Announce accepted talks, post schedule * Wednesday, July 22: Early Registration ends * Tuesday-Wednesday, August 18-19: Tutorials * Thursday-Friday, August 20-21: Conference * Saturday-Sunday, August 22-23: Sprints * Friday, September 4: Papers for proceedings due Executive Committee ------------------- * Jarrod Millman, UC Berkeley, USA (Conference Chair) * Gaël Varoquaux, INRIA Saclay, France (Program Co-Chair) * Stéfan van der Walt, University of Stellenbosch, South Africa (Program Co-Chair) * Fernando Pérez, UC Berkeley, USA (Tutorial Chair)

1 0