Mailman 3 June 2013 - NumPy-Discussion

Avoiding messy code in Python
by Bakhtiyor Zokhidov 12 Jun '13

12 Jun '13

Hi, I have some code more or less 500 lines, but very messy code. All codes containing several functions are in one module, besides, without documentation and testing. Could anyone give me some advice to organize my messy code in an accurate style including test function as well? Any answers will be appreciated, -- Bakhtiyor Zokhidov

3 2

Re: [Numpy-discussion] Weird problem with subtracting ndarrays
by Moroney, Catherine M (398D) 12 Jun '13

12 Jun '13

Thanks for the tip. I thought it must be something simple like that. When I convert the arrays to numpy.int32 things behave normally. Another question though: a numpy.all() on the signed-int arrays shows that they are equal. So why would the subtraction of the unsigned arrays wrap around like that? Catherine On Jun 12, 2013, at 12:25 PM, <numpy-discussion-request(a)scipy.org> wrote: > Message: 9 > Date: Wed, 12 Jun 2013 15:30:22 -0400 > From: Warren Weckesser <warren.weckesser(a)gmail.com> > Subject: Re: [Numpy-discussion] weird problem with subtracting > ndarrays > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <CAGzF1udhTAmxPWwHsd9xQ386CBz2iXymmYC4kJJ3AOTRPZQzyg(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > On Wed, Jun 12, 2013 at 3:25 PM, Moroney, Catherine M (398D) < > Catherine.M.Moroney(a)jpl.nasa.gov> wrote: > >> Hello, >> >> I've got two arrays of the same shape that I read in from a file, and I'm >> trying to >> difference them. Very simple stuff, but I'm getting weird answers. >> >> Here is the code: >> >>>>> counts1 = hfile1.read_grid_field("CFbA", >> "TerrainReferencedRCCMFraction_Num") >>>>> counts2 = hfile2.read_grid_field("CFbA", >> "TerrainReferencedRCCMFraction_Num") >>>>> counts1.max(), counts2.max() >> (13, 13) >>>>> counts1.min(), counts2.min() >> (0, 0) >>>>> numpy.all(counts1 == counts2) >> False >>>>> diff = counts1 - counts2 >>>>> diff.max() >> 4294967295 !! WHAT IS HAPPENING HERE ?? >>>>> sum = counts1 + counts2 >>>>> sum.max() >> 26 >> >> As you can see, the range of values in both arrays is 0 to 13, and the sum >> behaves normally, but the difference gives this weird number. >> >> When I create dummy arrays, the subtraction works fine. So there must be >> some funny value >> lurking in either the counts1 or counts2 array, but the numpy.isnan() test >> returns False. >> >> Any ideas for how I debug this? >> >> Catherine >> >> > Check the dtype of the arrays. They are probably unsigned integers, and > the subtraction leads to wrap-around in some cases. > > For example: > > In [1]: x = np.array([0, 1, 2], dtype=np.uint32) > > In [2]: y = np.array([1, 1, 1], dtype=np.uint32) > > In [3]: x - y > Out[3]: array([4294967295, 0, 1], dtype=uint32) > > > Warren > >

2 1

weird problem with subtracting ndarrays
by Moroney, Catherine M (398D) 12 Jun '13

12 Jun '13

Hello, I've got two arrays of the same shape that I read in from a file, and I'm trying to difference them. Very simple stuff, but I'm getting weird answers. Here is the code: >>> counts1 = hfile1.read_grid_field("CFbA", "TerrainReferencedRCCMFraction_Num") >>> counts2 = hfile2.read_grid_field("CFbA", "TerrainReferencedRCCMFraction_Num") >>> counts1.max(), counts2.max() (13, 13) >>> counts1.min(), counts2.min() (0, 0) >>> numpy.all(counts1 == counts2) False >>> diff = counts1 - counts2 >>> diff.max() 4294967295 !! WHAT IS HAPPENING HERE ?? >>> sum = counts1 + counts2 >>> sum.max() 26 As you can see, the range of values in both arrays is 0 to 13, and the sum behaves normally, but the difference gives this weird number. When I create dummy arrays, the subtraction works fine. So there must be some funny value lurking in either the counts1 or counts2 array, but the numpy.isnan() test returns False. Any ideas for how I debug this? Catherine

3 2

np.asarray atleast_float64 ?
by josef.pktd＠gmail.com 12 Jun '13

12 Jun '13

Is there anything to require a numpy array with a minimum numeric dtype? To avoid lower precision calculations and be upwards compatible, something like x = np.asarray(x, >=np.float64) that converts ints, bool and lower precision to float64 but leaves higher precision float and complex double alone. Josef

2 3

t-statistic
by Sudheer Joseph 12 Jun '13

12 Jun '13

Dear experts, I am doing a project involving regression of a model variable with observed variable and wanted to find t-values dynamically as the number of available observations involved in comparison changes. Is there a tool in numpy/scipy which gives the appropriate t-value if we give number of samples ? t = 2.31 # appropriate t value (where n=9, two tailed 95%) with best regards, Sudheer *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

2 1

datetime64 constructor ignores dtype argument?
by Dave Hirschfeld 12 Jun '13

12 Jun '13

The example below demonstrates the fact that the datetime64 constructor ignores the dtype argument if passed in. Is this conscious design decision or a bug/oversight? In [55]: from datetime import datetime ...: d = datetime.now() ...: In [56]: d Out[56]: datetime.datetime(2013, 6, 12, 11, 3, 27, 861000) In [57]: np.datetime64(d) Out[57]: numpy.datetime64('2013-06-12T12:03:27.861000+0100') In [58]: np.datetime64(d).dtype Out[58]: dtype('<M8[us]') In [59]: np.datetime64(d, dtype='M8[D]') Out[59]: numpy.datetime64('2013-06-12T12:03:27.861000+0100') In [60]: np.datetime64(d, dtype='M8[D]').dtype Out[60]: dtype('<M8[us]') <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ??? In [61]: np.datetime64(d).astype('M8[D]') Out[61]: numpy.datetime64('2013-06-12') Thanks, Dave

2 1

genfromtxt and gzip
by Ted To 11 Jun '13

11 Jun '13

Hi all, >From the list archives (2011), I noticed that there is a bug in the python gzip module that causes genfromtxt to fail with python 2 but this bug is not a problem for python 3. When I tried to use genfromtxt and python 3 with a gzip'ed csv file, I instead got: IOError: Mode rbU not supported Is this a bug? I am using python 3.2.3 and numpy 1.7.1 from the experimental Debian repository. Thanks!

2 1

empty_like for masked arrays
by Nathaniel Smith 11 Jun '13

11 Jun '13

Hi all, Is there anyone out there using numpy masked arrays, who has an opinion on how empty_like (and its friends ones_like, zeros_like) should handle the mask? Right now apparently if you call np.ma.empty_like on a masked array, you get a new masked array that shares the original array's mask, so modifying one modifies the other. That's almost certainly wrong. This PR: https://github.com/numpy/numpy/pull/3404 makes it so instead the new array has values that are all set to empty/zero/one, and a mask which is set to match the input array's mask (so whenever something was masked in the original array, the empty/zero/one in that place is also masked). We don't know if this is the desired behaviour for these functions, though. Maybe it's more intuitive for the new array to match the original array in shape and dtype, but to always have an empty mask. Or maybe not. None of us really use np.ma, so if you do and have an opinion then please speak up... -n

4 4

adding booleans
by josef.pktd＠gmail.com 08 Jun '13

08 Jun '13

Interesting observation, (while lurking on a pull request) >>> np.add.reduce(np.arange(5)<3) 3 >>> np.add((np.arange(5)<3), (np.arange(5)<3)) array([ True, True, True, False, False], dtype=bool) I often use summing of an array of boolean but didn't know the second behavior Josef

5 10

What's the difference between calling __mul__ and *?
by Will Lee 07 Jun '13

07 Jun '13

Can somebody tell me why these operations are not the same in numpy? In [2]: a = numpy.array([1, 2, 3.]) In [4]: matrix = numpy.matrix([[1, 2, 3.], [4, 5, 6], [7, 8, 9]]) In [5]: a.__mul__(matrix) matrix([[ 1., 4., 9.], [ 4., 10., 18.], [ 7., 16., 27.]]) In [6]: a * matrix matrix([[ 30., 36., 42.]]) Essentially I'm trying to extend from numpy.ndarray. From my subclass __mul__ I'd like to call the parent's __mul__ method. I ran into problem when I'm trying to call super(SubArrayClass, self).__mul__() method when working with a matrix. I also can't think of a way to use operator.mul() due to the subclass nature. Is there any way to make this work? Any help is greatly appreciated. Thanks, Will

3 3