Mailman 3 January 2012 - NumPy-Discussion

[JOB] Extracting subset of dataset using latitude and longitude
by Jeremy Lounds 13 Jan '12

13 Jan '12

Hello, I am looking for some help extracting a subset of data from a large dataset. The data is being read from a wgrib2 (World Meterological Organization standard gridded data) using the pygrib library. The data values, latitudes and longitudes are in separate lists (arrays?), and I would like a regional subset. The budget is not very large, but I am hoping that this is pretty simple job. I am just way too green at Python / numpy to know how to proceed, or even what to search for on Google. If interested, please e-mail jlounds(a)dynamiteinc.com Thank you! Jeremy Lounds DynamiteInc.com 1-877-762-7723, ext 711 Fax: 877-202-3014

2 1

Python for Scientists - courses in Germany and US
by Mike Müller 13 Jan '12

13 Jan '12

Learn NumPy and Much More ========================= Scientists like Python. If you would like to learn more about important libraries for scientific applications, you might be interested in these courses. The course in Germany covers: - Overview of libraries - NumPy - Data storage with text files, Excel, netCDF and HDF5 - matplotlib - Object oriented programming for scientists - Problem solving session The course in the USA covers all this plus: - Extending Python in other languages - Version control - Unit testing More details below. If you have any questions about the courses, please contact me. Mike Python for Scientists and Engineers (Germany) --------------------------------------------- A three-day course covering all the basic tools scientists and engineers need. This course requires basic Python knowledge. Date: 19.01.-21.01.2012 Location: Leipzig, Germany Trainer: Mike Müller Course Language: English Link: http://www.python-academy.com/courses/python_course_scientists.html Python for Scientists and Engineers (USA) ----------------------------------------- This is an extend version of our well-received course for scientists and engineers. Five days of intensive training will give you a solid basis for using Python for scientific an technical problems. The course is hosted by David Beazley (http://www.dabeaz.com). Date: 27.02.-02.03.2012 Location: Chicago, IL, USA Trainer: Mike Müller Course Language: English Link: http://www.dabeaz.com/chicago/science.html

2 1

Question on F/C-ordering in numpy svd
by Ivan Oseledets 13 Jan '12

13 Jan '12

Dear all! I quite new to numpy and python. I am a matlab user, my work is mainly on multidimensional arrays, and I have a question on the svd function from numpy.linalg It seems that u,s,v=svd(a,full_matrices=False) returns u and v in the F-contiguous format. That is not in a good agreement with other numpy stuff, where C-ordering is default. For example, matrix multiplication, dot() ignores ordering and returns result always in C-ordering. (which is documented), but the svd feature is not documented. With best wishes, Ivan

2 1

Numpy 'groupby'
by Michael Hull 12 Jan '12

12 Jan '12

Hi Everyone, First off, thanks for all your hard work on numpy, its a really great help! I was wondering if there was a standard 'groupby' in numpy, that similar to that in itertools. I know its not hard to write with np.diff, but I have found myself writing it on more than a couple of occasions, and wondered if there was a 'standarised' version I was missing out on?? Thanks, Mike

3 2

Index update
by Mads Ipsen 10 Jan '12

10 Jan '12

Hi, Suppose you have N items, say N = 10. Now a subset of these items are selected given by a list A of indices. Lets say that items A = [2,5,7] are selected. Assume now that you delete some of the items given by the indices S = [1,4,8]. This means that the list of indices A must be updated, since items have been deleted. For this particular case the updated selection list A becomes A = [1,3,5]. Is there some smart numpy way of doing this index update of the selected items in A without looping? Typically N is large. Best regards, Mads -- +-----------------------------------------------------+ | Mads Ipsen | +----------------------+------------------------------+ | Gåsebæksvej 7, 4. tv | | | DK-2500 Valby | phone: +45-29716388 | | Denmark | email: mads.ipsen(a)gmail.com | +----------------------+------------------------------+

2 2

Ufuncs and flexible types, CAPI
by Val Kalatsky 10 Jan '12

10 Jan '12

Hi folks, First post, may not follow the standards, please bear with me. Need to define a ufunc that takes care of various type. Fixed - no problem, userdef - no problem, flexible - problem. It appears that the standard ufunc loop does not provide means to deliver the size of variable size items. Questions and suggestions: 1) Please no laughing: I have to code for NumPy 1.3.0. Perhaps this issue has been resolved, then the discussion becomes moot. If so please direct me to the right link. 2) A reasonable approach here would be to use callbacks and to give the user (read programmer) a chance to intervene at least twice: OnInit and OnFail (OnFinish may not be unreasonable as well). OnInit: before starting the type resolution the user is given a chance to do something (e.g. check for that pesky type and take control then return a flag indicating a stop) before the resolution starts OnFail: the resolution took place and did not succeed, the user is given a chance to fix it. In most of the case these callbacks are NULLs. I could patch numpy with a generic method that does it, but it's a shame not to use the good ufunc machine. Thanks for tips and suggestions. Val Kalatsky

2 2

SParse feature vector generation
by Dhruvkaran Mehta 10 Jan '12

10 Jan '12

Hi numpy users, *Is there a convenient way in numpy to go from "string" features like:* "uc_berkeley", "google", 1 "stanford", "intel", 1 . . . "uiuc", "texas_instruments", 0 *to a numpy matrix like:* "uc_berkeley", "stanford", ..., "uiuc", "google", "intel", "texas_instruments", "bool" 1 0 ... 0 1 0 0 1 0 1 ... 0 0 1 0 1 : 0 0 ... 1 0 0 1 0 I really appreciate you taking the time to help! Thanks! --Dhruv

2 1

find location of maximum values
by questions anon 10 Jan '12

10 Jan '12

I have a netcdf file that contains hourly temperature data for a whole month. I would like to find the maximum temperature within that file and also the corresponding Latitude and Longitude and Time and then plot this. Below is the code I have so far. I think everything is working except for identifying the corresponding latitude and longitude. The latitude and longitude always seems to be in the same area (from file to file) and they are not where the maximum values are occurring. Any feedback will be greatly appreciated. from netCDF4 import Dataset import matplotlib.pyplot as plt import numpy as N from mpl_toolkits.basemap import Basemap from netcdftime import utime from datetime import datetime from numpy import ma as MA import os TSFCall=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", path+ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] TIME=ncfile.variables['time'][:] LAT=ncfile.variables['latitude'][:] LON=ncfile.variables['longitude'][:] ncfile.close() TSFCall.append(TSFC) big_array=N.ma.concatenate(TSFCall) tmax=MA.max(TSFC) print tmax t=TIME[tmax] indexTIME=N.where(TIME==t) TSFCmax=TSFC[tmax] print t, indexTIME print LAT[tmax], LON[tmax], TSFCmax cdftime=utime('seconds since 1970-01-01 00:00:00') ncfiletime=cdftime.num2date(t) print ncfiletime timestr=str(ncfiletime) d = datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S') date_string = d.strftime('%Y%m%d_%H%M') map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33,llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i') x,y=map(*N.meshgrid(LON,LAT)) map.tissot(LON[tmax], LAT[tmax], 0.15, 100, facecolor='none', edgecolor='black', linewidth=2) map.readshapefile(shapefile1, 'DSE_REGIONS') map.drawcoastlines(linewidth=0.5) map.drawstates() plt.title('test'+' %s'%date_string) ticks=[-5,0,5,10,15,20,25,30,35,40,45,50] CS = map.contourf(x,y,TSFCmax, 15, cmap=plt.cm.jet) l,b,w,h =0.1,0.1,0.8,0.8 cax = plt.axes([l+w+0.025, b, 0.025, h], ) cbar=plt.colorbar(CS, cax=cax, drawedges=True) plt.show() plt.close()

7 16

Error in numpy.load example?
by Sébastien Barthélémy 08 Jan '12

08 Jan '12

Hi all, the doc http://docs.scipy.org/doc/numpy/reference/generated/numpy.load.html contains the following example: Store compressed data to disk, and load it again: >>> np.savez('/tmp/123.npz', a=np.array([[1, 2, 3], [4, 5, 6]]), b=np.array([1, 2])) >>> data = np.load('/tmp/123.npy') However http://docs.scipy.org/doc/numpy/reference/generated/numpy.savez.html says: numpy.savez(file, *args, **kwds)¶ Save several arrays into a single file in uncompressed .npz format. Moreover, this last page points to an undocumented numpy.savez_compressed function, which is also non-existent in my version of numpy (1.5.1-2ubuntu2). That's quite confusing. >From the following thread, it seems the arrays are stored uncompressed in a zipfile. http://article.gmane.org/gmane.comp.python.numeric.general/38378 Can somebody confirm that/fix the docs? Cheers -- Sébastien

2 1

ANN: Numexpr 2.0.1 released
by Francesc Alted 08 Jan '12

08 Jan '12

========================== Announcing Numexpr 2.0.1 ========================== Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It wears multi-threaded capabilities, as well as support for Intel's VML library, which allows for squeezing the last drop of performance out of your multi-core processors. What's new ========== In this release, better docstrings for `evaluate` and reduction methods (`sum`, `prod`) is in place. Also, compatibility with Python 2.5 has been restored (2.4 is definitely not supported anymore). In case you want to know more in detail what has changed in this version, see: http://code.google.com/p/numexpr/wiki/ReleaseNotes or have a look at RELEASE_NOTES.txt in the tarball. Where I can find Numexpr? ========================= The project is hosted at Google code in: http://code.google.com/p/numexpr/ You can get the packages from PyPI as well: http://pypi.python.org/pypi/numexpr Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy! -- Francesc Alted

2 2