Mailman 3 April 2012 - NumPy-Discussion

Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer?
by Dave Fugate April 23, 2012

April 23, 2012

Thanks Ralf! I'm interested in unattended/silent installations. My best, Dave --------------------------------------------------- Date: Sat, 21 Apr 2012 10:48:36 +0200 From: Ralf Gommers <ralf.gommers(a)googlemail.com> Subject: Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer? To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <CABL7CQjHRun1BOndM0+adgS8msNkS4UR1KHUddc7vVYwTt=prg(a)mail.gmail.com> Content-Type: text/plain; charset="windows-1252" On Fri, Apr 20, 2012 at 8:05 PM, Dave Fugate <dfugate(a)microsoft.com> wrote: > Hi, is there any documentation available on exactly which command line > options are available from NumPy?s ?superpack? installers on Windows? > E.g., http://docs.scipy.org/doc/numpy/user/install.html mentions an > ?/arch? flag, but I?m not seeing anything else called out. > Other than arch selection I think it's a fairly standard NSIS installer. No idea what else you can do with it though from the command line. Are you looking to accomplish some specific task? Ralf

2 1

the state of NA/masking
by Nathaniel Smith April 22, 2012

April 22, 2012

Hi all, Travis, Mark, and I talked on Skype this week about how to productively move forward with the NA debate, and I got picked to summarize for the list :-). There are three main things we discussed: 1) About process: We seem to agree that this discussion has been ineffective for a variety of reasons, and that it would be best to back up and try the consensus-based approach. Maybe not everyone agrees... I'm not sure how we go about building consensus on whether we need consensus? And we noted that we may not actually all mean the same thing by that. To start a discussion, I'll write up separately what I understand by that term. 2) If we require consensus on our NA implemention, then we have a problem for the 1.7.0 release. The problem is this: -- We have some kind of commitment to keeping compatibility between releases -- Therefore, if we release with NA masks, then we have some kind of commitment to continuing to support these in some form going forward -- But as per above, we can't make such a commitment until we have consensus, and we don't have consensus. Even if we end up deciding that the current code is the best thing ever, we haven't done that yet. Therefore, we have a kind of constrained optimization problem: we need to find the best way to adjust our "some kind of commitment", or the current code, or both, so that we can release 1.7. Alternatively we could delay the release until we have reached and implemented consensus, but I have an allergy to putting such amorphous things on our critical path, and I suspect I'm not the only one. (If it turns out that consensus is quick and the release is slow for other reasons, then that'd be great, of course, but why depend on it if we don't have to?) I'll also send a separate email to try and lay out the main options here, as a basis for discussion. 3) And, in the long run, there's the actual question of what we kind of NA support we actually want in numpy. A major problem here is that it's very difficult for anyone who hasn't spent huge amounts of time wading through the mailing list to actually understand what the points of contention are. So, Mark and I are going to *co*-write a document explaining what we see as the main problems, and trying to clarify our disagreements. Of course, this still won't include everyone's point of view, but hopefully it will serve as a good starting point for... you guessed it... discussion. Cheers, -- Nathaniel

1 0

Euroscipy 2012 - abstract deadline soon (April 30) + sprints
by Emmanuelle Gouillart April 22, 2012

April 22, 2012

Hello, this is a reminder of the approaching deadline for abstract submission at the Euroscipy 2012 conference: the deadline is April 30, in one week. Euroscipy 2012 will be held in **Brussels**, **August 23-27**, at the Université Libre de Bruxelles (ULB, Solbosch Campus). The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research and industry. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. More information about the conference, including practical information, are found on the conference website http://www.euroscipy.org/conference/euroscipy2012 We are soliciting talks and posters that discuss topics related to scientific computing using Python. These include applications, teaching, future development directions, and research. We welcome contributions from the industry as well as the academic world. Submission guidelines are found on http://www.euroscipy.org/card/euroscipy2012_call_for_contributions Also, rooms are available at the ULB for sprints on Tuesday August 28th and Wednesday 29th. If you wish to organize a sprint at Euroscipy, please get in touch with Berkin Malkoc (malkocb(a)itu.edu.tr). Any other questions should be addressed exclusively to org-team(a)lists.euroscipy.org We apologize for the inconvenience if you received this e-mail through several mailing-lists. -- Emmanuelle, for the organizing team

1 0

1.7 blockers
by Charles R Harris April 22, 2012

April 22, 2012

Hi All, There several problems with numpy master that need to be fixed before a release can be considered. 1. Datetime on windows with mingw. 2. Bus error on SPARC, ticket #2076. 3. NA and real/complex views of complex arrays. Number 1 has been proved to be particularly difficult, any help or suggestions for that would be much appreciated. The current work has been going in pull request 214 <https://github.com/numpy/numpy/pull/214>. This isn't to say that there aren't a ton of other things that need fixing or that we can skip out on the current stack of pull requests, but I think it is impossible to consider a release while those three problems are outstanding. Chuck

14 48

ANN: Cython 0.16
by Dag Sverre Seljebotn April 21, 2012

April 21, 2012

[From Mark Florisson, release manager for 0.16. PS: Note that the git branch is "release", not "master": https://github.com/cython/cython/tree/release Dag] We are pleased to announce a new version of Cython, 0.16 (http://cython.org/release/Cython-0.16.tar.gz). It comes with new features, improvements and bug fixes, including - super() without arguments - fused types - memoryviews - more Python-like functions Many thanks to the many contributors of this release and to all bug reporters and supporting users! A more comprehensive list of features and contributors can be found here: http://wiki.cython.org/ReleaseNotes-0.16 , and an overview of bug fixes can be found here: http://trac.cython.org/cython_trac/query?status=closed&group=component&orde… Enjoy!

1 0

Problem in Generalized Ufunc with mixed data type signatures
by kishor.iitr April 21, 2012

April 21, 2012

Hi, I am trying to create Generalized ufunc with following signature: static char func_signatures[] = { NPY_LONG, NPY_DOUBLE, PyArray_INTP }; gufunc_signature = "(m), (n), -> (n)"; I am getting following warning while calling the function: TypeError: function not supported for these types, and can't coerce safely to supported types If I change m => NPY_DOUBLE and n -> NPY_DOBULE function works fine but somehow when I mix the data types it gives me warning mentioned above? I didn't find any example in the Numpy Library where we mix the data types. Can you please help me? Thanks, -Kishor -- View this message in context: http://old.nabble.com/Problem-in-Generalized-Ufunc-with-mixed-data-type-sig… Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 0

Command-line options for (Windows) NumPy Installer?
by Dave Fugate April 21, 2012

April 21, 2012

Hi, is there any documentation available on exactly which command line options are available from NumPy's 'superpack' installers on Windows? E.g., http://docs.scipy.org/doc/numpy/user/install.html mentions an "/arch" flag, but I'm not seeing anything else called out. Thanks! Dave

2 1

Getting C-function pointers from Python to C
by Travis Oliphant April 19, 2012

April 19, 2012

Hi all, Some of you are aware of Numba. Numba allows you to create the equivalent of C-function's dynamically from Python. One purpose of this system is to allow NumPy to take these functions and use them in operations like ufuncs, generalized ufuncs, file-reading, fancy-indexing, and so forth. There are actually many use-cases that one can imagine for such things. One question is how do you pass this function pointer to the C-side. On the Python side, Numba allows you to get the raw integer address of the equivalent C-function pointer that it just created out of the Python code. One can think of this as a 32- or 64-bit integer that you can cast to a C-function pointer. Now, how should this C-function pointer be passed from Python to NumPy? One approach is just to pass it as an integer --- in other words have an API in C that accepts an integer as the first argument that the internal function interprets as a C-function pointer. This is essentially what ctypes does when creating a ctypes function pointer out of: func = ctypes.CFUNCTYPE(restype, *argtypes)(integer) Of course the problem with this is that you can easily hand it integers which don't make sense and which will cause a segfault when control is passed to this "function" We could also piggy-back on-top of Ctypes and assume that a ctypes function-pointer object is passed in. This allows some error-checking at least and also has the benefit that one could use ctypes to access a c-function library where these functions were defined. I'm leaning towards this approach. Now, the issue is how to get the C-function pointer (that npy_intp integer) back and hand it off internally. Unfortunately, ctypes does not make it very easy to get this address (that I can see). There is no ctypes C-API, for example. There are two potential options: 1) Create an API for such Ctypes function pointers in NumPy and use the ctypes object structure. If ctypes were to ever change it's object structure we would have to adapt this API. Something like this is what is envisioned here: typedef struct { PyObject_HEAD char *b_ptr; } _cfuncptr_object; then the function pointer is: (*((void **)(((_sp_cfuncptr_object *)(obj))->b_ptr))) which could be wrapped-up into a nice little NumPy C-API call like void * Npy_ctypes_funcptr(obj) 2) Use the Python API of ctypes to do the same thing. This has the advantage of not needing to mirror the simple _cfuncptr_object structure in NumPy but it is *much* slower to get the address. It basically does the equivalent of ctypes.cast(obj, ctypes.c_void_p).value There is working code for this in the ctypes_callback branch of my scipy fork on github. I would like to propose two things: * creating a Npy_ctypes_funcptr(obj) function in the C-API of NumPy and * implement it with the simple pointer dereference above (option #1) Thoughts? -Travis

6 30

f2py with int8
by John Mitchell April 19, 2012

April 19, 2012

Hi, I am using f2py to pass a numpy array of type numpy.int8 to fortran. It seems like I am misunderstanding something because I just can't make it work. Here is what I am doing. PYTHON b=numpy.array(numpy.zeros(shape=(10,),dtype=numpy.int8),order='F') b[0]=1 b[2]=1 b[3]=1 b array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0], dtype=int8) FORTRAN subroutine print_bit_array(bits,n) use iso_fortran_env integer,intent(in)::n integer(kind=int8),intent(in),dimension(n)::bits print*,'bits = ',bits end subroutine print_bit_array RESULT when calling fortran from python bits = 1 0 0 0 0 0 0 0 1 0 Any Ideas? thanks, John

4 7

mask array and add to list
by questions anon April 18, 2012

April 18, 2012

I am trying to mask an array and then add the array to a list, so I can then go on and calculate the max, min and mean of that list. The mask seems to work when I check each array. I check each array by finding the max, mean and mean and comparing with the unmasked array (they are different). However, when I add the array to the list and then check the max, min and mean and compare to the list of unmasked array I find that they are the same (which they shouldn't be). The problem seems to be occurring when I add the masked array to the list. Is there something I am doing wrong or not doing?? Any feedback will be greatly appreciated. import numpy as np import matplotlib.pyplot as plt from numpy import ma as MA from mpl_toolkits.basemap import Basemap from datetime import datetime import os from StringIO import StringIO from osgeo import gdal, gdalnumeric, ogr, osr import glob from datetime import date, timedelta import matplotlib.dates as mdates import time shapefile=r"d:/Vic/Vic_dissolve.shp" ## Create masked array from shapefile xmin,ymin,xmax,ymax=[111.975,-9.975, 156.275,-44.525] ncols,nrows=[886, 691] maskvalue = 1 xres=(xmax-xmin)/float(ncols) yres=(ymax-ymin)/float(nrows) geotransform=(xmin,xres,0,ymax,0, -yres) 0 src_ds = ogr.Open(shapefile) src_lyr=src_ds.GetLayer() dst_ds = gdal.GetDriverByName('MEM').Create('',ncols, nrows, 1 ,gdal.GDT_Byte) dst_rb = dst_ds.GetRasterBand(1) dst_rb.Fill(0) #initialise raster with zeros dst_rb.SetNoDataValue(0) dst_ds.SetGeoTransform(geotransform) err = gdal.RasterizeLayer(dst_ds, [maskvalue], src_lyr) dst_ds.FlushCache() mask_arr=dst_ds.GetRasterBand(1).ReadAsArray() np.set_printoptions(threshold='nan') mask_arr[mask_arr == 255] = 1 newmask=MA.masked_equal(mask_arr,0) ### calculate monthly summary stats for VIC Only rainmax=[] rainmin=[] rainmean=[] rainmaxaust=[] rainminaust=[] rainmeanaust=[] yearmonthlist=[] yearmonth_int=[] GLOBTEMPLATE = r"e:/Rainfall/rainfall-{year}/r{year}{month:02}??.txt" def accumulate_month(year, month): files = glob.glob(GLOBTEMPLATE.format(year=year, month=month)) monthlyrain=[] monthlyrainaust=[] for ifile in files: f=np.genfromtxt(ifile,skip_header=6) viconly_f=np.ma.masked_array(f, mask=newmask.mask) #print "f:", f.max(), f.mean() #print "viconly_f:", viconly_f.max(), viconly_f.mean() monthlyrain.append(viconly_f) monthlyrainaust.append(f) yearmonth=str(year)+str(month) yearmonthlist.append(yearmonth) r_max, r_mean, r_min=np.max(monthlyrain), np.mean(monthlyrain), np.min(monthlyrain) ra_max, ra_mean, ra_min=np.max(monthlyrainaust), np.mean(monthlyrainaust), np.min(monthlyrainaust) rainmax.append(r_max) rainmean.append(r_mean) rainmean.append(r_min) rainmaxaust.append(ra_max) rainminaust.append(ra_min) rainmeanaust.append(ra_mean) print "viconly:", yearmonth,r_max, r_mean, r_min print "aust:", yearmonth,ra_max, ra_mean, ra_min ###loop through months and years stop_month = datetime(2011, 10, 01) month = datetime(2011, 01, 01) while month < stop_month: accumulate_month(month.year, month.month) month += timedelta(days=32) month = month.replace(day=01)

2 2