Mailman 3 March 2014 - NumPy-Discussion

GSoC - what's next
by Ralf Gommers 24 Mar '14

24 Mar '14

Hi all, Just a short update, now that the deadline for submitting GSoC proposals has passed. We received four proposals: 1. Leo Mao, "Numpy: Vector Math Library Integration" 2. Janani Padmanbhan, "SciPy/NumPy- enhancements in scipy.special (hyp2f1, sph_harm) " 3. Ankit Agrawal, "SciPy : Discrete Wavelet Transforms and related algorithms" 4. Richard Tsai, "SciPy: Rewrite and improve cluster package in Cython" In principle we have enough mentors for all these proposals, although it looks like I'll have to chase them a bit to sign up in Melange etc. We're going to have to rank these proposals in the next week or so and communicate our preferences to the PSF organizers. I'll be in touch with all potential mentors about that. The announcement by Google of which students were accepted will follow on April 21. Thanks to the four students who spent a lot of effort on creating solid proposals, and to all who helped them by giving feedback. Looks like it's going to be a productive summer! Cheers, Ralf

2 1

Drop support for Python 3.2?
by Charles R Harris 24 Mar '14

24 Mar '14

Hi All, The suggestion has been made the we drop Python 3.2 support in numpy 1.9 and scipy 0.15. The advantage, from my point of view, to supporting Python >= 3.3 is that the u'unicode' syntax is supported in 3.3 and this makes it easier to maintain compatibility with Python 2.6 and 2.7. However, it may be a bit early to make this move, so feedback is welcome. Chuck

3 2

why sort does not accept a key?
by Alan G Isaac 24 Mar '14

24 Mar '14

I'm wondering if `sort` intentially does not accept a `key` or if this is just a missing feature? (I suppose that if the `order` argument is specified it would have to accept a sequence of keys ...) Thanks, Alan Isaac

4 7

Implementing elementary matrices
by Matt Pagan 24 Mar '14

24 Mar '14

Greetings! I made a patch for NumPy that adds a function for easily creating elementary matrices. Sorry for not knowing the process for submitting patches. Is this function something the NumPy community could see adding to the codebase? Are there ways I can improve on this? diff --git a/numpy/lib/twodim_base.py b/numpy/lib/twodim_base.py index 12c0f9b..10073af 100644 --- a/numpy/lib/twodim_base.py +++ b/numpy/lib/twodim_base.py @@ -967,3 +967,85 @@ def triu_indices_from(arr, k=0): if not (arr.ndim == 2 and arr.shape[0] == arr.shape[1]): raise ValueError("input array must be 2-d and square") return triu_indices(arr.shape[0], k) + +def elem(N, i, j=None, t=None, dtype=float): + """ + Return an elementary matrix. + + Parameters + ---------- + N : int + The size of the NxN array to be returned. Elementary matrices + should be square. + i : int + The index of the first row on which operations are to be + performed. + j : int + If set, the index of the second row of which operations are to + be performed. + t : scalar + If set, the factor by which a given row will be multiplied. + + Returns + ------- + m: ndarray of shape (NxN) + The identity matrix after a single row operation has been + performed on it. + + See also + -------- + eye, identity + + Examples + ------- + To swap the the first and third rows of a 4x4 identity matirx: + + >>> L = elem(4, 0, 2) + >>> L + array([[ 0., 0., 1., 0.], + [ 0., 1., 0., 0.], + [ 1., 0., 0., 0.], + [ 0., 0., 0., 1.]]) + + This array then becomes quite useful for matrix multiplication. + + >>> H = np.matrix([[ 2, 3, 5, 7], + [11, 13, 17, 19], + [23, 29, 31, 37], + [41, 43, 47, 53]]) + >>> L*H + matrix([[ 23., 29., 31., 37.], + [ 11., 13., 17., 19.], + [ 2., 3., 5., 7.], + [ 41., 43., 47., 53.]]) + + When the elemntary matrix is multiplied by the given matrix, the + result is the given matrix with it's first and third rows swapped. + + If the given matrix is multiplied by the elementary matrix (i.e., + the multiplication takes place in reverse order, the result is + the given matrix with its first and third columns swapped. + + >>> H*L + matrix([[ 5., 3., 2., 7.], + [ 17., 13., 11., 19.], + [ 31., 29., 23., 37.], + [ 47., 43., 41., 53.]]) + + """ + m=eye(N, dtype=dtype) + if j==None and t==None: + raise ValueError("One or more of %s and %s must be set." % \ + ('j', 't')) + return None + elif t==None: + swap = np.array(m[i]) + m[i] = m[j] + m[j] = swap + return m + elif j==None: + m[i] *= t + return m + else: + m[j] += (t * m[i]) + return m -- Matt Pagan matt(a)pagan.io PGP: 0xE9284418E360583C

2 2

numpy sum each month and then numpy mean of all months
by questions anon 24 Mar '14

24 Mar '14

Hello all, I have netcdf files that contain hourly rainfall data. Each netcdf file includes one months worth of hours and I have 10 years worth of data. I would like to calculate the sum of each month and then the mean of these summed months across all of the years. I have no problem firstly calculating the sum of each month but then I come up with a mask-size error when I try to calculate the mean of those months. So somehow I am not combining my summed months correctly? Below is the code I am using (focusing on just january at this stage) and below that is the error. Any feedback will be greatly appreciated. from netCDF4 import Dataset import numpy as N import matplotlib.pyplot as plt from mpl_toolkits.basemap import Basemap import os shapefile1="/Users/REGIONS" OutputFolder=r"/Users/rainmonthlysummarystats/" fileforlatlon=Dataset("/Users/WRFsample/IDV71000_VIC_T_SFC.nc", 'r+', 'NETCDF4') LAT=fileforlatlon.variables['latitude'][:] LON=fileforlatlon.variables['longitude'][:] def summaryplots(variable): if variable=='RAIN': ncvariablename='PCP_SFC' MainFolder=r"/Data/WRFmonthly/" ticks=[0, 25, 50, 75, 100, 125, 150, 175, 200] cmap=plt.cm.jet Jan="01" monthseason="Jan" summonthlyrain_all=[] all_variabledata=[] for (path, dirs, files) in os.walk(MainFolder): if os.path.basename(path)==Jan: for ncfile in files: fileext=ncvariablename+'.nc' if ncfile.endswith(fileext): print "dealing with ncfiles:", path+ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') variable=ncfile.variables[ncvariablename][:,:,:] ncfile.close() all_variabledata.append(variable) #combine all data from the chosen variable to make one array for analyses big_array=N.ma.concatenate(all_variabledata) SUM=big_array.sum(axis=0) summonthlyrain_all.append(SUM) del all_variabledata[:] big_array_sumrain=N.ma.concatenate(summonthlyrain_all) MEAN=big_array_sumrain.mean(axis=0) #plot output summary stats map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33, llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') map.drawcoastlines() map.drawstates() map.readshapefile(shapefile1, 'DSE_REGIONS') x,y=map(*N.meshgrid(LON,LAT)) plottitle=ncvariablename+'_mean_'+monthseason plt.title(plottitle) CS = map.contourf(x,y,MEAN, ticks, cmap=cmap) l,b,w,h =0.1,0.1,0.8,0.8 cax = plt.axes([l+w+0.025, b, 0.025, h]) plt.colorbar(CS,cax=cax, drawedges=True) plt.savefig((os.path.join(OutputFolder, plottitle+'.png'))) plt.show() plt.close() summaryplots('RAIN') MaskError Traceback (most recent call last) /Applications/Canopy.app/appdata/canopy-1.3.0.1715.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/IPython/utils/py3compat.pyc in execfile(fname, *where) 202 else: 203 filename = fname --> 204 __builtin__.execfile(filename, *where) /Users/slburns/Dropbox/Python_code/WRFoutputs/plot_variable_sum_monthlyrain_percentiles_test.py in <module>() 71 72 ---> 73 summaryplots('RAIN') 74 75 /Users/slburns/Dropbox/Python_code/WRFoutputs/plot_variable_sum_monthlyrain_percentiles_test.py in summaryplots(variable) 62 plottitle=ncvariablename+'_mean_'+monthseason 63 plt.title(plottitle) ---> 64 CS = map.contourf(x,y,MEAN, ticks, cmap=cmap) 65 l,b,w,h =0.1,0.1,0.8,0.8 66 cax = plt.axes([l+w+0.025, b, 0.025, h]) /Users/slburns/Applications/User/lib/python2.7/site-packages/mpl_toolkits/basemap/__init__.pyc in with_transform(self, x, y, data, *args, **kwargs) 519 # convert lat/lon coords to map projection coords. 520 x, y = self(x,y) --> 521 return plotfunc(self,x,y,data,*args,**kwargs) 522 return with_transform 523 /Users/slburns/Applications/User/lib/python2.7/site-packages/mpl_toolkits/basemap/__init__.pyc in contourf(self, x, y, data, *args, **kwargs) 3670 # combine with data mask. 3671 mask = np.logical_or(ma.getmaskarray(data),xymask) -> 3672 data = ma.masked_array(data,mask=mask) 3673 CS = ax.contourf(x,y,data,*args,**kwargs) 3674 except: /Users/slburns/Applications/User/lib/python2.7/site-packages/numpy/ma/core.pyc in __new__(cls, data, mask, dtype, copy, subok, ndmin, fill_value, keep_mask, hard_mask, shrink, **options) 2708 msg = "Mask and data not compatible: data size is %i, " + 2709 "mask size is %i." -> 2710 raise MaskError(msg % (nd, nm)) 2711 copy = True 2712 # Set the mask to the new value MaskError: Mask and data not compatible: data size is 193, mask size is 20458.

1 0

1.9.0 release runup
by Charles R Harris 23 Mar '14

23 Mar '14

Hi All, It is time to start looking forward to the 1.9.0 release. Currently there are some 76 open PRs and they keep rolling in, which is good, but we need to decide on what is important for 1.9 and what can be put off to 1.10 because otherwise we will never finish. The datetime problems and some of the deprecations/futurewarnings that were present in 1.8 need to be dealt with. The nanmedian stuff will make a nice addition to the nan functions. Apart from those, if you have a PR or fix that you think needs to be in 1.9, please make it known. Chuck

3 4

Numpy 1.8.1 release
by Charles R Harris 23 Mar '14

23 Mar '14

Hi All, It is time for the 1.8.1 release to go forward. I'm on the fence as to whether to do an rc2 or just release and do a 1.8.2 if needed. The problems noted with the 1.8.1rc1 should be fixed, but if you are in a position to test the current 1.8.x branch, please give it a try. Chuck

3 4

Re: [Numpy-discussion] Resolving the associativity/precedence debate for @
by Christophe Bal 23 Mar '14

23 Mar '14

The left associativity should be the less disturbing choice. Using parenthesis to force right associativity will be not too painful. How many matrices in long product are involved concretely in Numpy projects ? On the other hand, Nathaniel proposed a new way to evaluate associative operators, and I really think that would be the best way to manage products of matrices. The idea is fir example to see A@B@C@D as __atmul__(A, B, C, D). Christophe BAL

1 0

unique return_index order?
by Alan G Isaac 21 Mar '14

21 Mar '14

The documentation of numpy.unique http://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html does not seem to promise that return_index=True will always index the *first* occurrence of each unique item, which I believe is the current behavior. A promise would be nice. Is it intended? Alan Isaac

3 6

Numpy Installation Problem Asking
by Chia Jing Yi 20 Mar '14

20 Mar '14

Hi, I plan to plot a sashimi plot to view the alternative splicing event of my interested "gene" by using MISO. After follow the following link, http://genes.mit.edu/burgelab/miso/docs/and read through the forum. Unfortunately, I still face some problems to run the complete set of MISO successful. I can run some of the python script but I still fail to run some of it :( It shown the following error message when I try some of the MISO python script (compare_miso.py, run_miso.py, sashimi_plot.py, etc). *.* *.* *.* *ImportError: No module named _ufuncs* I can successful run some of the important MISO script (index_gff.py, sam_to_bam.py). I'm downloading everything by using Python-2.7. I suspect it might due to numpy or scipy installation problem. Unfortunately, I still fail to figure it out :( I have try to use Python-3.3. But it will show the following error message when I try to run all the python script under MISO : *.* *.* *SyntaxError: invalid syntax* I have browse through forum and seqanswer etc. Unfortunately I still fail to figure out why it happen :( I have to manual download all the package and install it separately as I don't have the purposely through access network through server. It is due to the network security of server at University. Thanks and looking forward to hear from any of you. best regards edge

2 2