NumPy-Discussion October 2012

numpy-discussion@python.org

74 participants
62 discussions

Blaze
by Damon McDougall 01 Nov '12

01 Nov '12

I just saw this on a news site I read pretty often and thought this community would be interested. It certainly looks useful! https://speakerdeck.com/sdiehl/blaze-next-generation-numpy Sorry if it's been posted already. Best wishes, Damon -- Damon McDougall http://www.damon-is-a-geek.com B2.39 Mathematics Institute University of Warwick Coventry West Midlands CV4 7AL United Kingdom

3 2

Simple question about scatter plot graph
by klo uo 01 Nov '12

01 Nov '12

I guess this is simple for many on this list I have 20 samples with ~420 points each, and scatter plot (lag_plot) for all samples is attached 16x16 grid pattern is easily visible, but I can't make the meaning Thanks [image: Inline image 1]

4 6

using numpy.argmax to index into another array
by Moroney, Catherine M (388D) 01 Nov '12

01 Nov '12

Hello Everybody, I have the following problem that I would be interested in finding an easy/elegant solution to. I've got it working, but my solution is exceedingly clunky and I'm sure that there must be a better way. Here is the (boiled-down) problem: I have 2 different 3-d array, shaped (4,4,4), "a" and "b" I can find the max values and location of those max values in "a" in the 0-th dimension using max and argmax resulting in a 4x4 matrix. So far, very easy. I then want to find the values in "b" that correspond to the maximum values in a. This is where I got stuck. Below find the sample code I used (pretty clumsy stuff ...) Can somebody show a better (less clumsy) way of finding bmax in the code excerpt below? >>> a = numpy.random.randint(0, high=15, size=(4,4,4)) >>> b = numpy.random.randint(0, high=15, size=(4,4,4)) >>> amax = a.argmax(axis=0) >>> idx2 = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3] >>> idx1 = [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3] >>> idx_all = (amax.flatten(), numpy.array(idx1), numpy.array(idx2)) >>> bmax = b[idx_all].reshape(4,4) >>> a array([[[ 7, 1, 7, 10], [11, 5, 9, 0], [13, 1, 13, 14], [ 4, 13, 4, 7]], [[ 2, 11, 4, 5], [ 6, 12, 14, 9], [ 0, 4, 5, 14], [13, 5, 3, 4]], [[ 1, 4, 5, 4], [ 7, 13, 1, 11], [ 9, 2, 10, 4], [ 3, 4, 10, 13]], [[14, 6, 7, 1], [ 8, 0, 8, 2], [ 6, 8, 7, 12], [ 4, 9, 0, 12]]]) >>> idx array([[3, 1, 0, 0], [0, 2, 1, 2], [0, 3, 0, 0], [1, 0, 2, 2]]) >>> b array([[[ 4, 2, 3, 2], [ 9, 6, 1, 4], [ 1, 10, 13, 11], [10, 8, 1, 10]], [[ 3, 13, 7, 7], [ 4, 7, 9, 7], [ 3, 13, 10, 10], [13, 8, 14, 6]], [[14, 10, 13, 6], [13, 2, 12, 12], [ 9, 6, 3, 8], [ 0, 7, 8, 11]], [[12, 10, 3, 0], [11, 7, 0, 4], [ 9, 7, 11, 12], [ 7, 12, 1, 1]]]) >>> bmax = b[idx_all].reshape(4,4) >>> bmax array([[12, 13, 3, 2], [ 9, 2, 9, 12], [ 1, 7, 13, 11], [13, 8, 8, 11]]) Catherine

3 3

numpy.savez(_compressed) in loop
by Radek Machulka 01 Nov '12

01 Nov '12

Hi, is there a way how to save more arrays into single npy (npz if possible) file in loop? Something like: fw = open('foo.bar', 'wb') while foo: arr = np.array(bar) np.savez_compressed(fw, arr) fw.close() Or some workaround maybe? I go through hundreds of thousands arrays and can not keep them in memory. Yes, I can save each array into single file, but I would better have them all in single one. Thanks Radek

5 4

nan result from np.linalg.lstsq()
by Larry Paltrow 29 Oct '12

29 Oct '12

I'm having some trouble using the linalg.lstsq() function with certain data and trying to understand why. It's always returning nans when I feed it this numpy array of float64 data: data = df.close.values #coming from a pandas dataframe type(data) >>> numpy.ndarray data.dtype >>> dtype('float64') data >>> array([ 1.31570348, 1.31565421, 1.3157375 , ..., 1.32175 , 1.32180441, 1.321775 ]) xi = np.arange(0,len(data)) A = np.vstack([xi, np.ones(len(xi))]).T A >>> array([[ 0.00000000e+00, 1.00000000e+00], [ 1.00000000e+00, 1.00000000e+00], [ 2.00000000e+00, 1.00000000e+00], ..., [ 2.87800000e+03, 1.00000000e+00], [ 2.87900000e+03, 1.00000000e+00], [ 2.88000000e+03, 1.00000000e+00]]) m, c = np.linalg.lstsq(A, data)[0] m, c >>> (nan, nan) oy, do not understand. Does anyone else know why?

3 6

NumPy to CPU+GPU compiler, looking for tests
by Rahul Garg 29 Oct '12

29 Oct '12

Hi. I am a PhD student at McGill University and I am developing a compiler for Python for CPUs and GPUs. For CPUs, I build upon LLVM. For GPUs, I generate OpenCL and I have also implemented some library functions on the GPU myself. The restriction that it is only for numerical code and intended for NumPy users. The compiler is aware of simple things in NumPy like matrix multiplication, slicing operators, strided layouts, some library functions (though limited at this time) and the negative indexing semantics etc. However, the compiler is not limited to vector code. Scalar code or manually written loops also work. However, only numerical datatypes are supported with no support for lists, dicts, classes etc. First class functions are not currently supported but are on the roadmap. You will have to add some type annotations to your functions. If you have a compatible GPU, you can also use the GPU by indicating which parts to run on the GPU. Otherwise you can just use it to run your code on the CPU. As an example, simple scalar code like fibonacci function works fine. Simple loops like those used in stencil-type computations are also working. Parallel-for loops are also provided and working. Simple vector oriented code is also working fine on both CPU and GPU. The system is being tested on Ubuntu 12.04 and tested with Python 2.7 (though I think should work with other Python 2.x variants). For GPUs, I am ensuring that the system works with AMD and Nvidia GPUs. The compiler is in early stages and I am looking for test cases. The project will be open-sourced in November under Apache 2 and thereafter will be developed in an open repo. If you have some simple code that I can use as a benchmark that I can use to test and evaluate the compiler, that will be very helpful. Some annotations will be required, which I can help you write. I will be VERY grateful to anyone who can provide test cases. In turn, it will help improve the compiler and everyone will benefit. Some of you may be wondering how it compares to Numba. Well it is essentially very similar in the idea. So why build a new compiler then? Actually the project I am building is not specific to Python. I am building a far more general compiler infrastructure for array languages, and Python frontend is just one small part of the project. For example, I am also working on a MATLAB frontend. (Some of you may remember me from an earlier compiler project which unfortunately went nowhere. This is a different project and this time I am determined to convert it into a usable system. I realize the proof is in the pudding, so I hope to convince people by releasing code soon.) thanks, Rahul

4 10

Regression in mpl: AttributeError: incompatible shape for a non-contiguous array
by Benjamin Root 29 Oct '12

29 Oct '12

This error started showing up in the test suite for mpl when using numpy master. AttributeError: incompatible shape for a non-contiguous array The tracebacks all point back to various code points where we are trying to set the shape of an array, e.g., offsets.shape = (-1, 2) Those lines haven't changed in a couple of years, and was intended to be done this way to raise an error when reshaping would result in a copy (since we needed to use the original in those places). I don't know how these arrays have become non-contiguous, so I am wondering if there was some sort of attribute that got screwed up somewhere (maybe with views?) Cheers! Ben Root

3 7

error of install numpy on linux redhat.
by Jack Bryan 27 Oct '12

27 Oct '12

Hi, All, I am trying to install numpy from http://www.scipy.org/Download . by git clone git://github.com/numpy/numpy.git numpy But, when I ran python setup.py install I got: SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel Where to get python-dev ? I tried: $ easy_install python-develSearching for python-develReading http://pypi.python.org/simple/python-devel/Couldn't find index page for 'python-devel' (maybe misspelled?)Scanning index of all packages (this may take a while)Reading http://pypi.python.org/simple/No local packages or download links found for python-develerror: Could not find suitable distribution for Requirement.parse('python-devel') and $ easy_install python-devSearching for python-devReading http://pypi.python.org/simple/python-dev/Couldn't find index page for 'python-dev' (maybe misspelled?)Scanning index of all packages (this may take a while)Reading http://pypi.python.org/simple/No local packages or download links found for python-deverror: Could not find suitable distribution for Requirement.parse('python-dev')

6 18

copy/deepcopy pull request, Travis build failure
by David Warde-Farley 27 Oct '12

27 Oct '12

I submitted a pull request and one of the Travis builds is failing: https://travis-ci.org/#!/numpy/numpy/jobs/2933551 Given my changes, https://github.com/dwf/numpy/commit/4c88fdafc003397d6879f81bf59f68adeeb59f2b I don't see how the masked array module (responsible for the failing test) could possibly be affected in this manner (RuntimeWarning raised by encountering an invalid value in power()). Moreover, I cannot reproduce it after setting NPY_SEPARATE_COMPILATION on my own machine (whatever that does -- I'm not quite clear). Does anyone have any insight into what's going on here? Can anyone else reproduce it outside of Travis? Thanks, David

5 10

MaskedArray, view, and fill_value
by Thomas Robitaille 26 Oct '12

26 Oct '12

Hi everyone, Could someone explain the following behavior? >>> from numpy import ma >>> x = ma.array([1,2,3]) >>> x.fill_value = 1 >>> x.view(ma.MaskedArray) masked_array(data = [1 2 3], mask = False, fill_value = 999999) i.e. the fill_value gets reset. It looks from ma/core.py like it is deliberate: # Make sure to reset the _fill_value if needed if getattr(output, '_fill_value', None) is not None: output._fill_value = None but this doesn't make any sense to me... Any ideas? I'm thinking maybe it should be: # Make sure to reset the _fill_value if needed if getattr(output, '_fill_value', None) is not None and dtype is not None: output._fill_value = None because then it would make sense to reset the fill_value if the dtype has changed. Any thoughts? Thanks, Tom

1 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion October 2012