Mailman 3 December 2013 - NumPy-Discussion

numpy.dot causes segfault after ctypes call to cudaMalloc
by ebuchman 19 Dec '13

19 Dec '13

I'm learning cuda and decided to use python with ctypes to call all the cuda functions but I'm finding some memory issues. I've boiled it down to the simplest scenario. I use ctypes to call a cuda function which allocates memory on the device and then frees it. This works fine, but if I then try to use np.dot on a totally other array declared in python, I get a segmentation fault. Note this only happens if the numpy array is sufficiently large. If I change the cuda mallocs to simple c mallocs, all the problems go away, but thats not really helpful. Any ideas what's going on here? CUDA CODE (debug.cu): #include <stdio.h> #include <stdlib.h> extern "C" { void all_together( size_t N) { void*d; int size = N *sizeof(float); int err; err = cudaMalloc(&d, size); if (err != 0) printf("cuda malloc error: %d\n", err); err = cudaFree(d); if (err != 0) printf("cuda free error: %d\n", err); }} PYTHON CODE (master.py): import numpy as np import ctypes from ctypes import * dll = ctypes.CDLL('./cuda_lib.so', mode=ctypes.RTLD_GLOBAL) def build_all_together_f(dll): func = dll.all_together func.argtypes = [c_size_t] return func __pycu_all_together = build_all_together_f(dll) if __name__ == '__main__': N = 5001 # if this is less, the error doesn't show up a = np.random.randn(N).astype('float32') da = __pycu_all_together(N) # toggle this line on/off to get error #np.dot(a, a) print 'end of python' COMPILE: nvcc -Xcompiler -fPIC -shared -o cuda_lib.so debug.cu RUN: python master.py -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/numpy-dot-causes-segfault-after… Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 0

polyfit
by Alan G Isaac 19 Dec '13

19 Dec '13

For teaching it is certainly nice to have numpy.polynomial.polynomial.polyfit providing modern (vs. traditional) parameter order, but - it is rather buried - np.polyfit uses traditional order and has the same name I recall there was some controversy (?) over all of this, but might it not be appropriate to have a keyword argument to both specifying whether the parameter order is to be modern or traditional (in both polyfits and polyvals)? fwiw, Alan Isaac

2 2

repeat array in a fake dim without stride_tricks ?
by Pierre Haessig 18 Dec '13

18 Dec '13

Hello, In order to repeat rows or columns of an array as http://stackoverflow.com/questions/1550130/cloning-row-or-column-vectors I can use np.repeat as suggested by pv. However, looking at the flags of the resulting array, data seems to be copied and actually repeated in memory. This is not applicable if want a 1000x repetition. What are the other options for such a repeat ? On scipy lectures, there is a suggestion to use as_strided : http://scipy-lectures.github.io/advanced/advanced_numpy/#example-fake-dimen… Otherwise, I see broadcast_arrays : > N = 3 > data = np.arange(N) > np.broadcast_arrays(data[:,None], np.zeros((1,2)))[0] array([[0, 0], [1, 1], [2, 2]]) This works but it feels like invoking a magic formula. Did I miss a simpler function ? best, Pierre

2 3

[ANN] OpenOpt suite v 0.52
by Dmitrey 15 Dec '13

15 Dec '13

Hi all, I'm glad to inform you about new OpenOpt Suite release 0.52 (2013-Dec-15): Minor interalg speedup oofun expression MATLAB solvers fmincon and fsolve have been connected Several MATLAB ODE solvers have been connected New ODE solvers, parameters abstol and reltol New GLP solver: direct Some minor bugfixes and improvements Regards, D.

1 0

Does NumPy support indirect memory views?
by Alexander Belopolsky 15 Dec '13

15 Dec '13

PEP 3118 [1] allows exposing multi-dimensional data that is organized as array of pointers. It appears, however that NumPy cannot consume such memory views. Looking at _array_from_buffer_3118() function [2], I don't see any attempt to process suboffsets. The documentation [3] is also silent on this issue. What is the status of indirect memory views/buffers support in NumPy? [1] http://www.python.org/dev/peps/pep-3118/ [2] https://github.com/numpy/numpy/blob/4050ac73af79ae8cc513648ff02e9a22041501c… [3] http://docs.scipy.org/doc/numpy/reference/arrays.interface.html

2 2

ANN: Scipy 0.13.2 release
by Ralf Gommers 13 Dec '13

13 Dec '13

Hi, I'm happy to announce the availability of the scipy 0.13.2 release. This is a bugfix only release; it contains fixes for ndimage and optimize, and most importantly was compiled with Cython 0.19.2 to fix memory leaks in code using Cython fused types. Source tarballs, binaries and release notes can be found at http://sourceforge.net/projects/scipy/files/scipy/0.13.2/ Cheers, Ralf ========================== SciPy 0.13.2 Release Notes ========================== SciPy 0.13.2 is a bug-fix release with no new features compared to 0.13.1. Issues fixed ------------ - 3096: require Cython 0.19, earlier versions have memory leaks in fused types - 3079: ``ndimage.label`` fix swapped 64-bitness test - 3108: ``optimize.fmin_slsqp`` constraint violation

2 1

Using np.partition to extract n largest/smallest items from an array
by Jaime Fernández del Río 13 Dec '13

13 Dec '13

With the new np.partition functionality, there is a more efficient, but also less obvious, way of extracting the n largest (or smallest) elements from an array, i.e.: def smallest_n(a, n): return np.sort(np.partition(a, n)[:n]) def argsmallest_n(a, n): ret = np.argpartition(a, n)[:n] b = np.take(a, ret) return np.take(ret, np.argsort(b)) instead of the usual: np.sort(a)[:n] np.argsort(a)[:n] Are those 4 functions (smallest, argsmallest, largest, arglargest), with adequate axis support, worthy of including in numpy, or is the name space already too cluttered? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

1 0

ANN: HDF5 for Python 2.2.1
by Andrew Collette 10 Dec '13

10 Dec '13

Announcing HDF5 for Python (h5py) 2.2.1 ======================================= The h5py team is happy, in a sense, to announce the availability of h5py 2.2.1. This release fixes a critical bug reported by Jim Parker on December 7th, which affects code using HDF5 compound types. We recommend that all users of h5py 2.2.0 upgrade to avoid crashes or possible data corruption. About h5py, downloads, documentation: http://www.h5py.org Scope of bug ------------ The issue affects a feature introduced in h5py 2.2.0, in which HDF5 compound datasets may be updated in-place, by specifying a field name or names when writing to the dataset: >>> dataset['field_name'] = value Under certain conditions, h5py can supply uninitialized memory to the HDF5 conversion machinery, leading (in the case reported) to a segmentation fault. It is also possible for other fields of the type to be corrupted. This issue affects only code which updates a subset of the fields in the compound type. Programs reading from a compound type, writing all fields, or using other datatypes, are not affected; nor are versions of h5py prior to 2.2.0. More information ---------------- Github issue: https://github.com/h5py/h5py/issues/372 Original thread: https://groups.google.com/forum/#!topic/h5py/AbUOZ1MXf3U Thanks also to Christoph Gohlke for making Windows installers available on very short notice, after a glitch in the h5py build system.

1 0

OpenBLAS based numpy scipy test builds for WIN64
by Carl Kleffner 10 Dec '13

10 Dec '13

Hi list, I've uploaded on https://code.google.com/p/mingw-w64-static/ numpy/scipy binaries as wheel builds for testing. The binaries have been build with the help of a customized mingw-w64 toolchain and a recent (git) openBLAS. Regards Carl

1 0

wheels for OS X: request for testing
by Ralf Gommers 09 Dec '13

09 Dec '13

Hi all, At http://sourceforge.net/projects/numpy/files/wheels_to_test/ you can find a set of wheels, for numpy, scipy and ipython plus its dependencies. These are for the following configuration only: - OSX >= 10.6 - Python 2.7 32/64-bit from python.org It would be great if OS X users could test these. We're interested in whether these wheels work, and also if you run into any issues installing wheels. Since (a) I'm not really sure myself what the recommended and most robust way is to install wheels, and (b) it's interesting to see if the docs and usability of pip/wheel are ready for prime time, I'm not going to give the commands to execute but instead link to the relevant docs: http://www.pip-installer.org/en/latest/index.html http://python-packaging-user-guide.readthedocs.org/en/latest/ http://wheel.readthedocs.org/en/latest/index.html Please try this out and share your experience. Thanks, Ralf P.S.if you're wondering what on earth wheels are, they're the new binary install format for the standard python packaging and distribution tools (pip, setuptools).

2 2

NumPy-Discussion December 2013