Mailman 3 October 2014 - NumPy-Discussion

numpy.mean slicing in a netCDF file
by Fadzil Mnor Oct. 15, 2014

Oct. 15, 2014

Hi all, I wrote a script and plot monthly mean zonal wind (from a netcdf file names uwnd.mon.mean.nc) and I'm not sure if I'm doing it correctly. What I have: ******************************************************************************** *#this part calculates mean values for january only, from 1980-2010; thus the index looks like this 396:757:12* def ujan(): f = nc.Dataset('~/data/ncep/uwnd.mon.mean.nc') u10_1 = f.variables['uwnd'] u10_2 = np.mean(u10_1[396:757:12,:,38,39:43],axis=0) return u10_2 uJan = ujan()* #calling function* *#this part is only to define lon, lat and level * q = nc.Dataset('~/data/ncep/uwnd.mon.mean.nc') lon=q.variables['lon'] lat=q.variables['lat'] lev=q.variables['level'] *#for some reason I need to define this unless it gave error "length of x must be number of column in z"* lon=lon[39:43] *#begin plotting* clevs=np.arange(-10.,10.,0.5) fig = plt.figure(figsize=(11, 8)) fig.clf() ax = fig.add_subplot(111) ax.axis([97.5, 105., 1000., 10.]) ax.tick_params(direction='out', which='both') ax.set_xlabel('Lon (degrees)') ax.set_ylabel('Pressure (mb)') ax.set_xticks(np.arange(97.5, 105., .5)) ax.set_yticks([1000, 700, 500, 300, 100, 10]) cs=ax.contourf(lon, lev, uJan, clevs, extend='both',cmap='seismic') plt.title('Zonal winds average (Jan, 1981-2010)') cax = fig.add_axes([0.99, 0.1, 0.03, 0.8]) aa=fig.colorbar(cs,cax=cax,orientation='vertical') aa.set_label('m/s') plt.savefig('~/uwind-crossection-test.png', bbox_inches='tight') ******************************************************************************* the result is attached. I have no idea how to confirm the result (at least until this email is written) , but I believe the lower altitude values should be mostly negative, because over this region, the zonal wind are usually easterly (thus,negative values), but I got positive values. Put the information above aside, *I just want to know if my slicing in the ujan() function is correct*. If it is, then, there must be nothing wrong(except my above mentioned assumption). The data file dimension is: *[time,level,latitude,longitude]* This part: *u10_2 = np.mean(u10_1[396:757:12,:,38,39:43],axis=0)* The line above will calculate the mean of zonal wind (uwnd) in a range of time index 396 to 757 for each year (january only), for all vertical level, at latitude index 38 (5 N) and in between longitude index 39 to 43 (97.5E-105E). I assume it will calculate a 30-year average of zonal wind for january only. Is this correct? Thank you. Fadzil

4 5

Best way to expose std::vector to be used with numpy
by Daniele Nicolodi Oct. 15, 2014

Oct. 15, 2014

Hello, I have a C++ application that collects float, int or complex data in a possibly quite large std::vector. The application has some SWIG generated python wrappers that expose this vector to python. However, the standard way in which SWIG exposes the data is to create a touple and pass this to python, where it is very often converted to a numpy array for processing. Of course this is not efficient. I would like therefore to generate a smarter python interface. In python 3 I would without doubt go for implementing the buffer protocol, which enables seamless interfacing with numpy. However, I need to support also python 2 and there the buffer protocol is not as nice. What is the best thing to do to expose the data buffer to python 2, without introducing a dependency on numpy in the application? Bonus points if the proposed solution comes with a pointer on how to implement in in SWIG (which I don't know much and feel a bit too magic for my taste). Thank you! Cheers, Daniele

10 12

Endianness not detected in OS X Lion
by Nicola Oct. 15, 2014

Oct. 15, 2014

Hi, I am trying to install numpy 1.9.0 on OS X Lion 10.7.5 and I get the same error as reported in this old thread: http://thread.gmane.org/gmane.comp.python.numeric.general/47711/ that is, numpy/core/src/npymath/npy_math_private.h:78:3: error: conflicting types for 'ieee_double_shape_type' } ieee_double_shape_type; ^ numpy/core/src/npymath/npy_math_private.h:64:3: note: previous declaration of 'ieee_double_shape_type' was here } ieee_double_shape_type; ^ In file included from numpy/core/src/npymath/npy_math.c.src:56:0: numpy/core/src/npymath/npy_math_private.h:78:3: error: conflicting types for 'ieee_double_shape_type' } ieee_double_shape_type; ^ numpy/core/src/npymath/npy_math_private.h:64:3: note: previous declaration of 'ieee_double_shape_type' was here } ieee_double_shape_type; ^ no matter whether I use the system's clang or gcc 4.9 (installed with Homebrew). Is there a workaround for this? Nicola

2 2

array indexing question
by Neal Becker Oct. 14, 2014

Oct. 14, 2014

I'm using np.nonzero to construct the tuple: (array([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2]), array([1, 3, 5, 7, 2, 3, 6, 7, 4, 5, 6, 7])) Now what I want is the 2-D index array: [1,3,5,7, 2,3,6,7, 4,5,6,7] Any ideas? -- -- Those who don't understand recursion are doomed to repeat it

2 1

Any interest in a generalized piecewise function?
by Per.Brodtkorb＠ffi.no Oct. 13, 2014

Oct. 13, 2014

I have worked on a generalized piecewise function (genpiecewise) that are simpler and more general than the current numpy.piecewise implementation. The new generalized piecewise function allows functions of the type f(x0, x1,.. , xn) i.e. to have arbitrary number of input arguments that are evaluated conditionally. The generalized piecewise function passes all the tests for numpy.piecewise function except the undocumented features of numpy.piecewise which allows condlist to be a single bool list/array or a single int array. A new keyword "fillvalue" opens up the possibility to specify "out of bounds values" to other values than 0 eg. Np.nan. Examples: Example 1) >>> x = numpy.linspace(-2,2,5) >>> numpy.piecewise(x, x<1, [1]) # = numpy.where(x<0, 1, 0) array([ 1., 1., 1., 0., 0.]) # can be written as >>> genpiecewise([x<1],[1]) array([1, 1, 1, 0, 0]) # or >>> genpiecewise([x<1],[1], x) array([ 1., 1., 1., 0., 0.]) # or >>> genpiecewise([x<1],[1.0]) array([ 1., 1., 1., 0., 0.]) Example 2) >>> numpy.piecewise(x, [x < 0, x >= 0], [lambda x: -x, lambda x: x]) array([ 2., 1., 0., 1., 2.]) # can be written as >>> genpiecewise([x < 0, x >= 0], [lambda x: -x, lambda x: x], (x,)) array([ 2., 1., 0., 1., 2.]) # or genpiecewise([x < 0,], [lambda x: -x, lambda x: x], x) array([ 2., 1., 0., 1., 2.]) Example 3) # New functionality >>> X,Y = numpy.meshgrid(x,x) >>> genpiecewise([X*Y<0,], [lambda x,y: -x*y, lambda x,y: x*y], xi=(X,Y)) array([[ 4., 2., -0., 2., 4.], [ 2., 1., -0., 1., 2.], [-0., -0., 0., 0., 0.], [ 2., 1., 0., 1., 2.], [ 4., 2., 0., 2., 4.]]) >>> genpiecewise([X*Y<-0.5, X*Y>0.5], [lambda x,y: -x*y, lambda x,y: x*y], xi=(X,Y), fillvalue=numpy.nan) array([[ 4., 2., nan, 2., 4.], [ 2., 1., nan, 1., 2.], [ nan, nan, nan, nan, nan], [ 2., 1., nan, 1., 2.], [ 4., 2., nan, 2., 4.]]) >>> genpiecewise([X*Y<-0.5, X*Y>0.5], [lambda x,y: -x*y, lambda x,y: x*y, numpy.nan], (X,Y)) array([[ 4., 2., nan, 2., 4.], [ 2., 1., nan, 1., 2.], [ nan, nan, nan, nan, nan], [ 2., 1., nan, 1., 2.], [ 4., 2., nan, 2., 4.]]) My question is: are there any interest in the community for such a function? Could some or all of this functionality replace the current numpy.piecewise? (This function could replace the function _lazywhere (in scipy.stats._distn_infrastructure) which is heavily used in scipy.stats) Per A. Brodtkorb The code looks like this: def valarray(shape, value=np.nan, typecode=None): """Return an array of all value. """ out = ones(shape, dtype=bool) * value if typecode is not None: out = out.astype(typecode) if not isinstance(out, np.ndarray): out = asarray(out) return out def genpiecewise(condlist, funclist, xi=None, fillvalue=0, args=(), **kw): """ Evaluate a piecewise-defined function. Given a set of conditions and corresponding functions, evaluate each function on the input data wherever its condition is true. Parameters ---------- condlist : list of bool arrays Each boolean array corresponds to a function in `funclist`. Wherever `condlist[i]` is True, `funclist[i](x0,x1,...,xn)` is used as the output value. Each boolean array in `condlist` selects a piece of `xi`, and should therefore be of the same shape as `xi`. The length of `condlist` must correspond to that of `funclist`. If one extra function is given, i.e. if ``len(funclist) - len(condlist) == 1``, then that extra function is the default value, used wherever all conditions are false. funclist : list of callables, f(*(xi + args), **kw), or scalars Each function is evaluated over `x` wherever its corresponding condition is True. It should take an array as input and give an array or a scalar value as output. If, instead of a callable, a scalar is provided then a constant function (``lambda x: scalar``) is assumed. xi : tuple input arguments to the functions in funclist, i.e., (x0, x1,...., xn) fillvalue : scalar fillvalue for out of range values. Default 0. args : tuple, optional Any further arguments given here passed to the functions upon execution, i.e., if called ``piecewise(..., ..., args=(1, 'a'))``, then each function is called as ``f(x0, x1,..., xn, 1, 'a')``. kw : dict, optional Keyword arguments used in calling `piecewise` are passed to the functions upon execution, i.e., if called ``piecewise(..., ..., lambda=1)``, then each function is called as ``f(x0, x1,..., xn, lambda=1)``. Returns ------- out : ndarray The output is the same shape and type as x and is found by calling the functions in `funclist` on the appropriate portions of `x`, as defined by the boolean arrays in `condlist`. Portions not covered by any condition have undefined values. See Also -------- choose, select, where Notes ----- This is similar to choose or select, except that functions are evaluated on elements of `xi` that satisfy the corresponding condition from `condlist`. The result is:: |-- |funclist[0](x0[condlist[0]],x1[condlist[0]],...,xn[condlist[0]]) out = |funclist[1](x0[condlist[1]],x1[condlist[1]],...,xn[condlist[1]]) |... |funclist[n2](x0[condlist[n2]],x1[condlist[n2]],...,xn[condlist[n2]]) |-- Examples -------- Define the sigma function, which is -1 for ``x < 0`` and +1 for ``x >= 0``. >>> x = np.linspace(-2.5, 2.5, 6) >>> genpiecewise([x < 0, x >= 0], [-1, 1]) array([-1., -1., -1., 1., 1., 1.]) Define the absolute value, which is ``-x`` for ``x <0`` and ``x`` for ``x >= 0``. >>> genpiecewise([x < 0, x >= 0], [lambda x: -x, lambda x: x], (x,)) array([ 2.5, 1.5, 0.5, 0.5, 1.5, 2.5]) """ def otherwise_condition(condlist): return ~np.logical_or.reduce(condlist, axis=0) def check_shapes(condlist, funclist): nc, nf = len(condlist), len(funclist) if nc not in [nf-1, nf]: raise ValueError("function list and condition list" + " must be the same length") check_shapes(condlist, funclist) if xi is not None and not isinstance(xi, tuple): xi = (xi,) condlist = np.broadcast_arrays(*condlist) if len(condlist) == len(funclist)-1: condlist.append(otherwise_condition(condlist)) arrays = dtype = None if xi is not None: arrays = np.broadcast_arrays(*xi) dtype = np.result_type(*arrays) else: # funclist is a list of scalars only dtype = np.result_type(*funclist) out = valarray(condlist[0].shape, fillvalue, dtype) for cond, func in zip(condlist, funclist): if isinstance(func, collections.Callable): temp = tuple(np.extract(cond, arr) for arr in arrays) + args np.place(out, cond, func(*temp, **kw)) else: # func is a scalar value np.place(out, cond, func) return out

2 3

non-linear rebin function recipe?
by RayS Oct. 13, 2014

Oct. 13, 2014

Most of my work has used the Fourrier based method for "linear" rebin of evenly sampled time data of length m (say 1500) to a new number of samples n (say 2048); the delta time change per sample is a constant over the array. I'd like to test the effect a non-constant delta t, ie, stretching some regions of data while leaving others static, and transitioning in a smooth way. - Ray Schumacher

1 0

numpy 1.9.1 rc1 probably next weekend
by Julian Taylor Oct. 13, 2014

Oct. 13, 2014

hi, we have collected a couple bugfixes in the 1.9 branch that should get released soon. The most important bugs fixes will be: - fix matplotlibs build (#5067) - avoid triggering a compiler bug on redhat 5, gcc 4.1.2 (#5163) - fix compatibility of npy files between py2 and py3 on win32/64 (#5170) There is still a bug in nanmedian if you have infinities in the array (#5138) which I hope will be fixed soon. After that I'd like to cut a 1.9.1rc1 with final release the week after. If there is something missing in the 1.9.x branch please report or fix it soon. Cheers, Julian

1 0

Detect if array has been transposed
by Mads Ipsen Oct. 13, 2014

Oct. 13, 2014

Hi, In part of my C++ code, I often do void foo(PyObject * matrix) { do stuff } where matrix is a numpy mxn matrix created on the Python side, where foo() eg. is invoked as a = numpy.array([[1,2],[3,5]]) foo(a) However, if you call transpose() on a, some care should be taken, since numpy's internal matrix data first gets transposed on demand. In that case I must do a = numpy.array([[1,2],[3,5]]) a.transpose() foo(a.copy()) to make sure the correct data of the array gets transferred to the C++ side. Is there any way for me to detect (on the Python side) that transpose() has been invoked on the matrix, and thereby only do the copy operation when it really is needed? For example if a_has_transposed_data: foo(a.copy()) else: foo(a) Best regards, Mads -- +---------------------------------------------------------+ | Mads Ipsen | +----------------------+----------------------------------+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby | email: mads.ipsen(a)gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +----------------------+----------------------------------+

4 5

[ANN] bcolz 0.7.2
by Valentin Haenel Oct. 12, 2014

Oct. 12, 2014

====================== Announcing bcolz 0.7.2 ====================== What's new ========== This is a maintenance release that fixes various bits and pieces. Importantly, compatibility with Numpy 1.9 and Cython 0.21 has been fixed and the test suit no longer segfaults on 32 bit UNIX. Feature-wise a new ``carray.view()`` method has been introduced which allows carrays to share the same raw data. ``bcolz`` is a renaming of the ``carray`` project. The new goals for the project are to create simple, yet flexible compressed containers, that can live either on-disk or in-memory, and with some high-performance iterators (like `iter()`, `where()`) for querying them. Together, bcolz and the Blosc compressor, are finally fulfilling the promise of accelerating memory I/O, at least for some real scenarios: http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/queryi… For more detailed info, see the release notes in: https://github.com/Blosc/bcolz/wiki/Release-Notes What it is ========== bcolz provides columnar and compressed data containers. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, a high-performance compressor that is optimized for binary data. bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations. bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems. Installing ========== bcolz is in the PyPI repository, so installing it is easy:: $ pip install -U bcolz Resources ========= Visit the main bcolz site repository at: http://github.com/Blosc/bcolz Manual: http://bcolz.blosc.org Home of Blosc compressor: http://blosc.org User's mail list: bcolz(a)googlegroups.com http://groups.google.com/group/bcolz License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt ---- **Enjoy data!**

1 0

parallel compilation with numpy.distutils in numpy 1.10
by Julian Taylor Oct. 10, 2014

Oct. 10, 2014

hi, To speed up compilation of extensions I have made a PR to compile extension files in parallel: https://github.com/numpy/numpy/pull/5161 It adds the --jobs/-j flags to the build command of setup.py which defines the number of parallel compile processes. E.g. python setup.py build --jobs 4 install --prefix /tmp/local Additionally it adds the environment variable NPY_NUM_BUILD_JOBS which is used if no commandline is set. This helps e.g. with pip installations, travis builds (which give you 1.5 cpus) or to put in your .bashrc. The parallelization is only with the files of an extension so it is not super efficient but an uncached numpy build goes down from 1m40s to 1m00s with 3 cores on my machine which is quite decent. Building scipy from scratch decreased from 10minutes to 6m30s on my machine. Unfortunately projects using cython will not profit as cython tends to build an extension from one single file. (You may want to look into gccs internal parallelization for that, -flto=jobs)- Does some see issues with the interface I have currently set? Please speak up soon. There is still one problem in regards to parallelizing fortran 90. The ccompiler.py contains following comment: # build any sources in same order as they were originally specified # especially important for fortran .f90 files using modules This indicates the f90 builds cannot be trivially parallelized. I do not know much fortran, can someone explain to me when ordering of single file compiles is an issue in f90? Cheers, Julian Taylor

2 5

Copyright status of NumPy binaries on Windows/OS X
by Andrew Collette Oct. 10, 2014

Oct. 10, 2014

Hi all, I am working with the HDF Group on a new open-source viewer program for HDF5 files, powered by NumPy, h5py, and wxPython. On Windows, since people don't typically have Python installed, we are looking to distribute the application using PyInstaller, which embeds dependencies like NumPy. Likewise for OS X (using Py2App). We would like to make sure we don't accidentally include non-open-source components... I recall there was some discussion here about using the Intel math libraries for binary releases on various platforms. Do the releases on SourceForge or PyPI use any proprietary code? We'd like to avoid building NumPy ourselves if we can avoid it. Apologies if this is explained somewhere, but I couldn't find it. Thanks! Andrew Collette

6 11

fsin on Intel
by Alan G Isaac Oct. 10, 2014

Oct. 10, 2014

This is not NumPy specific but may still interest list members: http://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-boun… Alan Isaac

1 0

use ufunc for arbitrary positional arguments?
by Benjamin Root Oct. 10, 2014

Oct. 10, 2014

I have a need to "and" together an arbitrary number of boolean arrays. np.logical_and() expects only two positional arguments. There has got to be some sort of easy way to just and these together using the ufunc mechanism, right? Cheers! Ben Root

3 3

Re: [Numpy-discussion] Extracting individual columns in Numpy (suchith)
by Juan Oct. 9, 2014

Oct. 9, 2014

> Date: Thu, 09 Oct 2014 11:12:31 +0530 > From: suchith <suchithjn22(a)gmail.com> > Subject: [Numpy-discussion] Extracting individual columns in Numpy > To: numpy-discussion(a)scipy.org > Message-ID: <54362047.10102(a)gmail.com> > Content-Type: text/plain; charset="utf-8" > > How to extract individual columns from a numpy array? > For example, consider this script > > import numpy as np > a = np.array([[1,2,3],[4,5,6],[7,8,9]]) > a[0][:] > a[:][0] > > Now both a[:][0] and a[0][:] are outputting the same result, i.e > np.array([1,2,3]). If I want to extract the array [[1],[4],[7]] then > what should I do? Is it possible to add this feature? If so, which > file(s) should I edit? > > I know C,C++ and Python programming and I am new to open-source software > development. Please help me. > Try: In [2]: import numpy as np In [3]: a = np.array([[1,2,3],[4,5,6],[7,8,9]]) In [4]: a[0][:] Out[4]: array([1, 2, 3]) In [5]: a[:][0] Out[5]: array([1, 2, 3]) In [6]: a[0,:] Out[6]: array([1, 2, 3]) In [7]: a[:,0] Out[7]: array([1, 4, 7]) Syntax is different for numpy arrays. Regards, Juan

1 0

Extracting individual columns in Numpy
by suchith Oct. 9, 2014

Oct. 9, 2014

How to extract individual columns from a numpy array? For example, consider this script import numpy as np a = np.array([[1,2,3],[4,5,6],[7,8,9]]) a[0][:] a[:][0] Now both a[:][0] and a[0][:] are outputting the same result, i.e np.array([1,2,3]). If I want to extract the array [[1],[4],[7]] then what should I do? Is it possible to add this feature? If so, which file(s) should I edit? I know C,C++ and Python programming and I am new to open-source software development. Please help me. I have attached the screenshot with this mail. Thanks Suchith.J.N

2 2

Re: [Numpy-discussion] Extracting individual columns in Numpy
by Andrew Nelson Oct. 9, 2014

Oct. 9, 2014

>import numpy as np a = np.array([[1,2,3],[4,5,6],[7,8,9]]) a[0][:] a[:][0] Now both a[:][0] and a[0][:] are outputting the same result, i.e np.array([1,2,3]). If I want to extract the array [[1],[4],[7]] then what should I do? Is it possible to add this feature? The feature is already there: a[:, 0] -- _____________________________________ Dr. Andrew Nelson _____________________________________

1 0

ANN: Pandas 0.15.0 Release Candiate 1
by Jeff Reback Oct. 7, 2014

Oct. 7, 2014

Hi, I'm pleased to announce the availability of the first release candidate of Pandas 0.15.0. Please try this RC and report any issues here: Pandas Issues <https://github.com/pydata/pandas/issues> We will be releasing officially in 1-2 weeks or so. This is a major release from 0.14.1 and includes a number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. Highlights include: - Drop support for numpy < 1.7.0 - The Categorical type was integrated as a first-class pandas type - New scalar type Timedelta, and a new index type TimedeltaIndex - New DataFrame default display for df.info() to include memory usage - New datetimelike properties accessor .dt for Series - Split indexing documentation into Indexing and Selecting Data and MultiIndex / Advanced Indexing - Split out string methods documentation into Working with Text Data - read_csv will now by default ignore blank lines when parsing - API change in using Indexes in set operations - Internal refactoring of the Index class to no longer sub-class ndarray - dropping support for PyTables less than version 3.0.0, and numexpr less than version 2.1 Here are the full whatsnew and documentation links: v0.15.0 Whatsnew <http://pandas.pydata.org/pandas-docs/version/0.15.0/whatsnew.html> v0.15.0 Documentation Page <http://pandas.pydata.org/pandas-docs/version/0.15.0/> Source tarballs, and windows builds are available here: Pandas v0.15.0rc1 Release <https://github.com/pydata/pandas/releases> A big thank you to everyone who contributed to this release! Jeff

1 0

f2py and debug mode
by Bayard Oct. 6, 2014

Oct. 6, 2014

Hello to all. I'm aiming to wrap a Fortran program into Python. I started to work with f2py, and am trying to setup a debug mode where I could reach breakpoints in Fortran module launched by Python. I've been looking in the existing post, but not seeing things like that. I'm used to work with visual studio 2012 and Intel Fortran compiler, I have tried to get that point doing : 1) Run f2py -m to get *.c wrapper 2) Embed it in a C Project in Visual Studio, containing also with fortranobject.c and fortranobject.h, 3) Create a solution which also contains my fortran files compiled as a lib 4) Generate in debug mode a "dll" with extension pyd (to get to that point name of the "main" function in Fortran by "_main"). I compiled without any error, and reach break point in C Wrapper, but not in Fortran, and the fortran code seems not to be executed (whereas it is when compiling with f2py -c). Trying to understand f2py code, I noticed that f2py is not only writing c-wrapper, but compiling it in a specific way. Is there a way to get a debug mode in Visual Studio with f2py (some members of the team are used to it) ? Any alternative tool we should use for debugging ? Thanks for answering Ferdinand --- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

3 2

Round away from zero (towards +/- infinity)
by T J Oct. 4, 2014

Oct. 4, 2014

Is there a ufunc for rounding away from zero? Or do I need to do x2 = sign(x) * ceil(abs(x)) whenever I want to round away from zero? Maybe the following is better? x_ceil = ceil(x) x_floor = floor(x) x2 = where(x >= 0, x_ceil, x_floor) Python's round function goes away from zero, so I am looking for the NumPy equivalent (and using vectorize() seems undesirable). In this sense, it seems that having a ufunc for this type of rounding could be helpful. Aside: Is there interest in a more general around() that allows users to specify alternative tie-breaking rules, with the default staying 'round half to nearest even'? [1] Also, what is the difference between NumPy's fix() and trunc() functions? It seems like they achieve the same goal. trunc() was added in 1.3.0. So is fix() just legacy? --- [1] http://stackoverflow.com/questions/16000574/tie-breaking-of-round-with-numpy

3 4

0/0 == 0?
by T J Oct. 4, 2014

Oct. 4, 2014

Hi, I'm using NumPy 1.8.2: In [1]: np.array(0) / np.array(0) Out[1]: 0 In [2]: np.array(0) / np.array(0.0) Out[2]: nan In [3]: np.array(0.0) / np.array(0) Out[3]: nan In [4]: np.array(0.0) / np.array(0.0) Out[4]: nan In [5]: 0/0 --------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) <ipython-input-6-6549dea6d1ae> in <module>() ----> 1 0/0 ZeroDivisionError: integer division or modulo by zero Out[1] seems odd. I get the right value in 1.8.1. Was this fixed for 1.9.0?

9 21

skip samples in random number generator
by Brad Buran Oct. 3, 2014

Oct. 3, 2014

Given the following: from numpy import random rs = random.RandomState(seed=1) # skip the first X billion samples x = rs.uniform(0, 10) How do I accomplish "skip the first X billion samples" (e.g. 7.2 billion)? I see that there's a numpy.random.RandomState.set_state which accepts (among other parameters) a value called "pos". This sounds promising, but the other parameters I'm not sure how to compute (e.g. the 1D array of 624 unsigned integers, etc.). I need to be able to skip ahead in the sequence to reproduce some signals that were generated for experiments. I could certainly consume and discard the first X billion samples; however, that seems to be computationally inefficient. Brad

4 8

Problem building 64 bit numpy using MKL on Windows
by Reynolds, Jay Oct. 3, 2014

Oct. 3, 2014

Hi, I've built numpy 64 bit using vc11, the Intel Fortran compiler and the MKL 'mkl_rt' library. *why? (see end of message for the reason, if interested) Any advice or assistance would be greatly appreciated. If I can offer additional information, I will happily do so. The build appears to go just fine (no errors noted), and numpy loads into python just fine as well. (I note a warning: ### Warning: python_xerbla.c is disabled ### -- however, it doesn't appear to be problematic?) I have also confirmed that numpy sees the mkl blas and lapack libs. >>> numpy.__config__.show() lapack_opt_info: libraries = ['mkl_lapack', 'mkl_rt'] library_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\lib\\intel64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\include'] blas_opt_info: libraries = ['mkl_rt'] library_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\lib\\intel64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\include'] openblas_lapack_info: NOT AVAILABLE lapack_mkl_info: libraries = ['mkl_lapack', 'mkl_rt'] library_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\lib\\intel64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\include'] blas_mkl_info: libraries = ['mkl_rt'] library_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\lib\\intel64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\include'] mkl_info: libraries = ['mkl_rt'] library_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\lib\\intel64'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['C:\\Program Files (x86)\\Intel\\Composer XE 2015\\mkl\\include'] Everything *looks* to be in order upon casual inspection (*I think*, please correct me if I'm wrong!) However, there is no performance boost when running a few different tests in numpy (singular value decomposition, for example), and only a single thread appears to be in play. Running numpy.test('full') reveals 21 errors. For instance, LINK : fatal error LNK1104: cannot open file 'ifconsol.lib' And, the other being a recurring error with f2py, ERROR: test_size.TestSizeSumExample.test_transpose ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Program Files\Side Effects Software\Houdini 13.0.509\python27\lib\site-packages\nose\case.py", line 371, in setUp try_run(self.inst, ('setup', 'setUp')) File "C:\Program Files\Side Effects Software\Houdini 13.0.509\python27\lib\site-packages\nose\util.py", line 478, in try_run return func() File "C:\Program Files\Side Effects Software\Houdini 13.0.509\python27\lib\site-packages\numpy\f2py\tests\util.py", line 353, in setUp module_name=self.module_name) File "C:\Program Files\Side Effects Software\Houdini 13.0.509\python27\lib\site-packages\numpy\f2py\tests\util.py", line 80, in wrapper raise ret RuntimeError: Running f2py failed: ['-m', '_test_ext_module_5403', 'c:\\users\\jareyn~1\\appdata\\local\\temp\\tmpvykewl \\foo.f90'] Reading .f2py_f2cmap ... Mapping "real(kind=rk)" to "double" Succesfully applied user defined changes from .f2py_f2cmap Everything that requires configuration appears to be in agreement with this Intel Application Note, minus use of the Intel C++ compiler: https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl I have also referenced the Windows build docs on scipy.org: http://www.scipy.org/scipylib/building/windows.html#building-scipy Some info about my configuration: site.cfg: include_dirs = C:\Program Files (x86)\Intel\Composer XE 2015\mkl\include library_dirs = C:\Program Files (x86)\Intel\Composer XE 2015\mkl\lib\intel64 mkl_libs = mkl_rt PATH = (paths separated by line for easy reading) C:\Program Files\Side Effects Software\Houdini 13.0.509\python27; C:\Program Files\Side Effects Software\Houdini 13.0.509\python27\Scripts; C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_amd64; C:\Program Files (x86)\Intel\Composer XE 2015\bin\intel64 LD_LIBRARY_PATH = C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_amd64; C:\Program Files (x86)\Intel\Composer XE 2015\bin\intel64; C:\Program Files (x86)\Intel\Composer XE 2015\mkl\lib\intel64; C:\Program Files (x86)\Intel\Composer XE 2015\compiler\lib\intel64 Thank you in advance for your time, -Jay ===== *why am I doing this? The reason I'm doing this is because I need numpy with MKL to run with the version of python that comes packaged with Houdini (Python 2.7.5 (default, Oct 24 2013, 17:49:49) [MSC v.1700 64 bit (AMD64)] on win32). So, downloading a prebuilt 64 bit numpy isn't an option due to the unavailability of a compatible compiler version.

1 0

Re: [Numpy-discussion] Interpolation using `np.interp()` with periodic x-coordinates (Saullo Castro)
by Saullo Castro Oct. 2, 2014

Oct. 2, 2014

Jaime has helped me and PR#5117 is getting ready to be merged <https://github.com/numpy/numpy/pull/5117>... 2014-10-02 19:00 GMT+02:00 <numpy-discussion-request(a)scipy.org>: > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Re: Interpolation using `np.interp()` with periodic > x-coordinates (Charles R Harris) > 2. Re: Proposal: add ndarray.keys() to return dtype.names > (Sebastian Wagner) > 3. Re: Proposal: add ndarray.keys() to return dtype.names > (John Zwinck) > 4. skip samples in random number generator (Brad Buran) > 5. Re: skip samples in random number generator (Robert Kern) > 6. Re: skip samples in random number generator (Nathaniel Smith) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 1 Oct 2014 18:12:56 -0600 > From: Charles R Harris <charlesr.harris(a)gmail.com> > Subject: Re: [Numpy-discussion] Interpolation using `np.interp()` with > periodic x-coordinates > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <CAB6mnxJwuw8k=NCkkUyKioCvLUy3XfUQkR= > o-DgkxWQfvWOB6A(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Wed, Sep 24, 2014 at 3:57 PM, Saullo Castro <saullogiovani(a)gmail.com> > wrote: > > > From the closed pull request PR #5109: > > > > https://github.com/numpy/numpy/pull/5109 > > > > it came out that the a good implementation would be adding a parameter > > `period`. I would like to know about the community's interest for this > > implementation. > > > > The modification are shown here: > > > > > https://github.com/saullocastro/numpy/compare/interp_with_period?expand=1 > > > > Please, let me know about your feedback. > > > > I don't have any problem with allowing periodic interpolation. > > Chuck >

1 0

Proposal: add ndarray.keys() to return dtype.names
by John Zwinck Oct. 2, 2014

Oct. 2, 2014

I first proposed this on GitHub: https://github.com/numpy/numpy/issues/5134 ; jaimefrio requested that I bring it to this list for discussion. My proposal is to add a keys() method to NumPy's array class ndarray. The behavior would be to return self.dtype.names, i.e. the "column names" for a structured array (and None when dtype.names is None, which it is for pure numeric arrays without named columns). I originally proposed to add a values() method also, but I am tabling that for now so we needn't discuss it in this thread. The motivation is to enhance the ability to use duck typing with NumPy arrays, Python dicts, and other types like Pandas DataFrames, h5py Files, and more. It's a fairly common thing to want to get the "keys" of a container, where "keys" is understood to be a sequence of values one can pass to __getitem__(), and this is exactly what I'm aiming at. Thoughts? John Zwinck

5 12

Interpolation using `np.interp()` with periodic x-coordinates
by Saullo Castro Oct. 2, 2014

Oct. 2, 2014

>From the closed pull request PR #5109: https://github.com/numpy/numpy/pull/5109 it came out that the a good implementation would be adding a parameter `period`. I would like to know about the community's interest for this implementation. The modification are shown here: https://github.com/saullocastro/numpy/compare/interp_with_period?expand=1 Please, let me know about your feedback. Regards, Saullo

2 1