Mailman 3 August 2010 - NumPy-Discussion

py3k execfile('setup.py')
by Ralf Gommers Aug. 20, 2010

Aug. 20, 2010

Hi, The execfile builtin has disappeared in python 3.x, so I'm trying to find another solution for the use of it in setupegg.py. So far I've tried if sys.version_info[0] >= 3: # 3.x doesn't have execfile anymore, so we define our own # The code below is syntactically valid 2.x, but 2.x thinks that a tuple # gets passed to the exec statement. (from Twisted) def execfile(filename, globals=None, locals=None): exec(compile(open(filename).read(), filename, 'exec'), globals, locals) and: if sys.version_info[0] >= 3: import imp setupfile = imp.load_source('setupfile', 'setup.py') setupfile.setup_package() Both of the above attempts hang at: $ paver dmg -p 3.1 ---> pavement.dmg ---> pavement.clean LDFLAGS='-undefined dynamic_lookup -bundle -arch i386 -arch ppc -Wl,-search_paths_first -Lbuild' /Library/Frameworks/Python.framework/Versions/3.1/bin/python3 setupegg.py bdist_mpkg Converting to Python3 via 2to3... Looks like the combination of setuptools, numpy.distutils and 2to3 is too much for me to wrap my head around. Anyone have an idea? Cheers, Ralf

3 4

index of the first element fulfilling a condition?
by Johann Bauer Aug. 20, 2010

Aug. 20, 2010

Hi, is there a good method to find the index of the first element in a 1D array fulfilling a condition? The following does the job >>> import numpy >>> a = numpy.array([1,5,78,3,12,4]) >>> numpy.where( a>10 )[0][0] 2 but it first compares the entire array and then selects the first index. Is there a way (especially for big arrays) to stop the comparison after the first hit? Thanks for any help, Johann

3 2

numpy.concatenate slower than slice copying
by Zbyszek Szmek Aug. 20, 2010

Aug. 20, 2010

Hi, this is a problem which came up when trying to replace a hand-written array concatenation with a call to numpy.vstack: for some array sizes, numpy.vstack(data) runs > 20% longer than a loop like alldata = numpy.empty((tlen, dim)) for x in data: step = x.shape[0] alldata[pos:pos+step] = x pos += step (example script attached) $ python del_cum3.py numpy 10000 10000 1 10 problem size: (10000x10000) x 1 = 10^8 0.816s <------------------------------- numpy.concatentate of 10 arrays 10000x10000 $ python del_cum3.py concat 10000 10000 1 10 problem size: (10000x10000) x 1 = 10^8 0.642s <------------------------------- slice manipulation giving the same result When the array size is reduced to 100x100 or so, the computation time goes to 0, so it seems that the dtype and dimension checking is negligible. Does numpy.concatenate do some extra work? Thanks for any pointers, Zbyszek PS. Architecture is amd64. python2.6, numpy 1.3.0 or python3.1, numpy 2.0.0.dev / trunk@8510 give the same result.

3 4

max value of an array of lists
by sm lkd Aug. 19, 2010

Aug. 19, 2010

Hello again, Here's a simple one. I have an array/matrix of lists (floats). array.max() or max(array) returns list (!) and not an individual value. Of course, it is possible to loop, but maybe someone knows a better way to do the same. Thanks,

2 3

vectorization of vectorization
by sm lkd Aug. 19, 2010

Aug. 19, 2010

Hello, Here's my problem: for each value t of an array (from 0 to 1e6) a smaller array is computed (size between 2-6). To compute the smaller array, I have a function (which can be easily vectorized if necessary) which takes t and an array of powers of t. The return is an array of modified Bessel function values, i.e.: def funct(order, t, power): return special.iv(order, t)**power Note that order and power are arrays after this vectorization: vec_func = sp.vectorization(func) Right this is how it's used: for i in range(1000000): y[i] = vec_func(orders, t, powers).prod() Incredibly slow. Of course, it is desirable to vectorize it it terms of t. I have tried different methods but still cannot make it work. Any suggestions or ointers? Thank you.

3 3

Re: [Numpy-discussion] summing over more than one axis
by Joe Harrington Aug. 19, 2010

Aug. 19, 2010

On Thu, 19 Aug 2010 09:06:32 -0500, G?khan Sever <gokhansever(a)gmail.com> wrote: >On Thu, Aug 19, 2010 at 9:01 AM, greg whittier <gregwh(a)gmail.com> wrote: > >> I frequently deal with 3D data and would like to sum (or find the >> mean, etc.) over the last two axes. I.e. sum a[i,j,k] over j and k. >> I find using .sum() really convenient for 2d arrays but end up >> reshaping 2d arrays to do this. I know there has to be a more >> convenient way. Here's what I'm doing >> >> a = np.arange(27).reshape(3,3,3) >> >> # sum over axis 1 and 2 >> result = a.reshape((a.shape[0], a.shape[1]*a.shape[2])).sum(axis=1) >> >> Is there a cleaner way to do this? I'm sure I'm missing something obvious. >> >> Thanks, >> Greg >> > >Using two sums > >np.sum(np.sum(a, axis=-2), axis=1) Be careful. This works for sums, but not for operations like median; the median of the row medians may not be the global median. So, you need to do the medians in one step. I'm not aware of a method cleaner than manually reshaping first. There may also be speed reasons to do things in one step. But, two steps may look cleaner in code. --jh--

5 5

loading database data into numpy arrays
by John Salvatier Aug. 19, 2010

Aug. 19, 2010

Hello, I am trying to load some time series data into numpy arrays from a MySQL database using pyodbc, but I am not sure what the standard way to do this is. I found the following: http://www.aidanfindlater.com/python-db-api-to-numpy but when I tried after doing a select (I made sure the cursor had rows). np.fromiter(cursor, dtype=[('a',float), ('b', float)]) I got Traceback (most recent call last): File "<console>", line 1, in <module> File "C:\Users\jsalvatier\workspace\rpxdata\src\rpxdata\mysql.py", line 45, in run_statistics_15 return np.fromiter(cursor, dtype=[('a',float), ('b', float)]) TypeError: expected a readable buffer object Does anyone have advice on this? John

2 3

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 47, Issue 47
by David Goldsmith Aug. 19, 2010

Aug. 19, 2010

Date: Wed, 18 Aug 2010 09:20:41 +0200 > From: Sturla Molden <sturla(a)molden.no> > Subject: Re: [Numpy-discussion] [SciPy-Dev] Good-bye, sort of (John > Hunter) > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <8C0B2317-2A22-4828-99E8-AC6C0F77824F(a)molden.no> > Content-Type: text/plain; charset=us-ascii; format=flowed; > delsp=yes > > Den 18. aug. 2010 kl. 08.19 skrev Martin Raspaud > <martin.raspaud(a)smhi.se>: > > > Once upon a time, when my boss wanted me to use matlab, I found myself > > implementing a python interpreter in matlab... > > > > There are just two sane solutions for Matlab: Either embed CPyton in a > MEX file, or use Matlab's JVM to run Jython ;) > > http://vader.cse.lehigh.edu/~perkins/pymex.html<http://vader.cse.lehigh.edu/%7Eperkins/pymex.html> > > Sturla > > Date: Wed, 18 Aug 2010 09:33:59 +0100 > From: Robin <robince(a)gmail.com> > Subject: Re: [Numpy-discussion] [SciPy-Dev] Good-bye, sort of (John > Hunter) > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <AANLkTi=cWWHiUG4g5P+oew7G1QAkysZYU57F9dWU62Cs(a)mail.gmail.com<cWWHiUG4g5P%2Boew7G1QAkysZYU57F9dWU62Cs(a)mail.gmail.com> > > > Content-Type: text/plain; charset=ISO-8859-1 > Just thought I'd mention another one since this came up: > > http://github.com/kw/pymex > This one works very nicely - it proxies any Python objects so you can > use, should you want to, the Matlab IDE as a python interpreter, > supports numpy arrays etc. Also cross-platform - I even got it to work > with 64 bit matlab/python on windows (in a fork on github). > > Thanks for the ideas; are any/all of these "solutions" freely/easily redistributable? DG

1 0

summing over more than one axis
by greg whittier Aug. 19, 2010

Aug. 19, 2010

I frequently deal with 3D data and would like to sum (or find the mean, etc.) over the last two axes. I.e. sum a[i,j,k] over j and k. I find using .sum() really convenient for 2d arrays but end up reshaping 2d arrays to do this. I know there has to be a more convenient way. Here's what I'm doing a = np.arange(27).reshape(3,3,3) # sum over axis 1 and 2 result = a.reshape((a.shape[0], a.shape[1]*a.shape[2])).sum(axis=1) Is there a cleaner way to do this? I'm sure I'm missing something obvious. Thanks, Greg

5 5

inconsistency in 10**(-2) or 10**array([-2])
by Mark Bakker Aug. 19, 2010

Aug. 19, 2010

Hello list, When I do 10**-2, Python nicely returns 0.02 But with numpy (version 1.3.0), I get zero: In [492]: 10**array([-2]) Out[492]: array([0]) Is this the intended behavior? Thanks, Mark

3 3