Mailman 3 December 2011 - NumPy-Discussion

numpy.array() of mixed integers and strings can truncate data
by Thouis (Ray) Jones 05 Dec '11

05 Dec '11

Is this expected behavior? >>> np.array([-345,4,2,'ABC']) array(['-34', '4', '2', 'ABC'], dtype='|S3') >>> np.version.full_version '1.6.1' >>> np.version.git_revision '68538b74483009c2c2d1644ef00397014f95a696' Ray Jones

7 11

stacking scalars with column vector
by Santhosh R 05 Dec '11

05 Dec '11

Hi , I am trying to learn python to convert some of my mlab codes to python.Am new to python.Some help here would be appreciated I am trying to make a column vector by stacking two scalars-->Xstart,Xend and second column of n X 2 2Darray-->A In matlab the code is b=[Xstart;A(:,1);Xend] I have made the following code import numpy as np x=np.array([[Xstart]]) arr_temp=np.array([A[:,0]]) x=np.vstack((x,arr_temp.T)) x=np.vstack((x,[[Xend]])) This works, but is there a more elegant way of doing this. Also if i define arr_temp as arr_temp=A[:,0], the array arr_temp which is supposed to be of shape (n,1) shows as (n,) .Why is this Thanks Santhosh

2 2

bug in PyArray_GetCastFunc
by Geoffrey Irving 05 Dec '11

05 Dec '11

When attempting to cast to a user defined type, PyArray_GetCast looks up the cast function in the dictionary but doesn't check if the entry exists. This causes segfaults. Here's a patch. Geoffrey diff --git a/numpy/core/src/multiarray/convert_datatype.c b/numpy/core/src/multiarray/convert_datatype.c index 818d558..4b8f38b 100644 --- a/numpy/core/src/multiarray/convert_datatype.c +++ b/numpy/core/src/multiarray/convert_datatype.c @@ -81,7 +81,7 @@ PyArray_GetCastFunc(PyArray_Descr *descr, int type_num) key = PyInt_FromLong(type_num); cobj = PyDict_GetItem(obj, key); Py_DECREF(key); - if (NpyCapsule_Check(cobj)) { + if (cobj && NpyCapsule_Check(cobj)) { castfunc = NpyCapsule_AsVoidPtr(cobj); } }

2 3

Convert datetime64 to python datetime.datetime in numpy 1.6.1?
by Warren Weckesser 04 Dec '11

04 Dec '11

In numpy 1.6.1, what's the most straightforward way to convert a datetime64 to a python datetime.datetime? E.g. I have In [1]: d = datetime64("2011-12-03 12:34:56.75") In [2]: d Out[2]: 2011-12-03 12:34:56.750000 I want the same time as a datetime.datetime instance. My best hack so far is to parse repr(d) with datetime.datetime.strptime: In [3]: import datetime In [4]: dt = datetime.datetime.strptime(repr(d), "%Y-%m-%d %H:%M:%S.%f") In [5]: dt Out[5]: datetime.datetime(2011, 12, 3, 12, 34, 56, 750000) That works--unless there are no microseconds, in which case ".%f" must be removed from the format string--but there must be a better way. Warren

2 2

"upsample" or scale an array
by Robin Kraft 04 Dec '11

04 Dec '11

Thanks Warren, this is great, and even handles giant arrays just fine if you've got enough RAM. I also just found this StackOverflow post with another solution. a.repeat(2, axis=0).repeat(2, axis=1). http://stackoverflow.com/questions/7525214/how-to-scale-a-numpy-array np.kron lets you do more, but for my simple use case the repeat() method is faster and more ram efficient with large arrays. In [3]: a = np.random.randint(0, 255, (2400, 2400)).astype('uint8') In [4]: timeit a.repeat(2, axis=0).repeat(2, axis=1) 10 loops, best of 3: 182 ms per loop In [5]: timeit np.kron(a, np.ones((2,2), dtype='uint8')) 1 loops, best of 3: 513 ms per loop Or for a 43200x4800 array: In [6]: a = np.random.randint(0, 255, (2400*18, 2400*2)).astype('uint8') In [7]: timeit a.repeat(2, axis=0).repeat(2, axis=1) 1 loops, best of 3: 6.92 s per loop In [8]: timeit np.kron(a, np.ones((2, 2), dtype='uint8')) 1 loops, best of 3: 27.8 s per loop In this case repeat() peaked at about 1gb of ram usage while np.kron hit about 1.7gb. Thanks again Warren. I'd tried way too many variations on reshape and rollaxis, and should have come to the Numpy list a lot sooner! -Robin On Dec 3, 2011, at 12:51 AM, Warren Weckesser wrote: > On Sat, Dec 3, 2011 at 12:35 AM, Robin Kraft wrote: > > > I need to take an array - derived from raster GIS data - and upsample or > > scale it. That is, I need to repeat each value in each dimension so that, > > for example, a 2x2 array becomes a 4x4 array as follows: > > > > [[1, 2], > > [3, 4]] > > > > becomes > > > > [[1,1,2,2], > > [1,1,2,2], > > [3,3,4,4] > > [3,3,4,4]] > > > > It seems like some combination of np.resize or np.repeat and reshape + > > rollaxis would do the trick, but I'm at a loss. > > > > Many thanks! > > > > -Robin > > > > > Just a day or so ago, Josef Perktold showed one way of accomplishing this > using numpy.kron: > > In [14]: a = arange(12).reshape(3,4) > > In [15]: a > Out[15]: > array([[ 0, 1, 2, 3], > [ 4, 5, 6, 7], > [ 8, 9, 10, 11]]) > > In [16]: kron(a, ones((2,2))) > Out[16]: > array([[ 0., 0., 1., 1., 2., 2., 3., 3.], > [ 0., 0., 1., 1., 2., 2., 3., 3.], > [ 4., 4., 5., 5., 6., 6., 7., 7.], > [ 4., 4., 5., 5., 6., 6., 7., 7.], > [ 8., 8., 9., 9., 10., 10., 11., 11.], > [ 8., 8., 9., 9., 10., 10., 11., 11.]]) > > > Warren

3 6

"upsample" or scale an array
by Robin Kraft 03 Dec '11

03 Dec '11

I need to take an array - derived from raster GIS data - and upsample or scale it. That is, I need to repeat each value in each dimension so that, for example, a 2x2 array becomes a 4x4 array as follows: [[1, 2], [3, 4]] becomes [[1,1,2,2], [1,1,2,2], [3,3,4,4] [3,3,4,4]] It seems like some combination of np.resize or np.repeat and reshape + rollaxis would do the trick, but I'm at a loss. Many thanks! -Robin

2 1

scipy.org still says source in some subversion repo -- should be git !?
by Sebastian Haase 02 Dec '11

02 Dec '11

google search for: numpy browse source points here: http://new.scipy.org/download.html which talks about: svn co http://svn.scipy.org/svn/numpy/trunk numpy the link should probably go here: https://github.com/numpy/numpy as already stated here: http://www.scipy.org/Developer_Zone HTH, Sebastian Haase

4 4

build numpy matrix out of smaller matrix
by jonasr 02 Dec '11

02 Dec '11

Hi, is there any possibility to define a numpy matrix, via a smaller given matrix, i.e. in matlab i can do this like a=[1 2 ; 3 4 ] A=[a a ; a a ] so that i finally get A=[ [1,2,1,2] [3,4,3,4] [1,2,1,2] [3,4,3,4]] i tried different things on numpy which didn't work any ideas ? thank you -- View this message in context: http://old.nabble.com/build-numpy-matrix-out-of-smaller-matrix-tp32896004p3… Sent from the Numpy-discussion mailing list archive at Nabble.com.

4 4

np.dot and array order
by josef.pktd＠gmail.com 01 Dec '11

01 Dec '11

np.__version__ '1.5.1' official win32 installer (playing with ipython for once) I thought np.dot is Lapack based and favors fortran order, but if the second array is fortran ordered, then dot takes twice as long. The order of the first array seems irrelevant (or maybe just with my shapes, in case it matters: the first array is float64, the second is bool, and I'm low in left over memory) In [93]: %timeit np.dot(x.T, indi) 1 loops, best of 3: 1.33 s per loop In [94]: %timeit np.dot(xf.T, indi) 1 loops, best of 3: 1.27 s per loop In [95]: %timeit np.dot(xf.T, indif) 1 loops, best of 3: 3 s per loop In [100]: %timeit np.dot(x.T, indif) 1 loops, best of 3: 3.05 s per loop In [96]: x.flags.c_contiguous Out[96]: True In [97]: xf.flags.c_contiguous Out[97]: False In [98]: indi.flags.c_contiguous Out[98]: True In [99]: indif.flags.c_contiguous Out[99]: False In [101]: x.shape Out[101]: (200000, 20) In [102]: indi.shape Out[102]: (200000, 500) It's just the way it is, or does it depend on ....? Josef

2 1

what statistical module to use for python?
by Chao YUE 01 Dec '11

01 Dec '11

Hi all, I just want to broadly ask what statistical package are you guys using? I mean routine statistical function like linear regression, GLM, ANOVA... etc. I know there is SciKits packages like statsmodels, but are there more general and complete ones? thanks to all, Chao -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************

2 2