Mailman 3 July 2010 - NumPy-Discussion

match or vectorized in-type function.
by James Bullard 12 Jul '10

12 Jul '10

I have two vectors of integers of not necessarily the same length. Consider the hypothetical function match (or if you are familiar to R then consider that function). match(v1, v2) => returns a boolean array of length len(v1) indicating whether element i in v1 is in v2. I cannot find this function in numpy. I would assume a variant is there, but not being able to find it I wrote it myself. First, is there such a function? Second, if there is not, is this implementation reasonable? thanks, jim from numpy import * a2 = random.randint(1, 1000, 1000) a1 = random.randint(1, 10000000, 100000) def match(v1, v2): d = dict(zip(v2, range(0, len(v2)))) return array([ d.has_key(x) for x in v1])

2 1

3 dim array unpacking
by K.-Michael Aye 12 Jul '10

12 Jul '10

Dear numpy hackers, I can't find the syntax for unpacking the 3 dimensions of a rgb array. so i have a MxNx3 image array 'img' and would like to do: red, green, blue = img[magical_slicing] Which slicing magic do I need to apply? Thanks for your help! BR, Michael

2 1

Another reality check
by David Goldsmith 12 Jul '10

12 Jul '10

In numpy.fft we find the following: "Then A[1:n/2] contains the positive-frequency terms, and A[n/2+1:] contains the negative-frequency terms, in order of decreasingly negative frequency." Just want to confirm that "decreasingly negative frequency" means ..., A[n-2] = A_(-2), A[n-1] = A_(-1), as implied by our definition (attached). DG

3 4

Function returns nothing!
by allan oware 12 Jul '10

12 Jul '10

Hi All! def height_diffs(): h = [] i = 0 while True: x = raw_input("Enter height difference ") if x == 'q': break else: h.append(x) i = i + 1 m = asarray(h,dtype=float) return m why does return statement return nothing?

2 1

"Nyquist frequency" in numpy.fft docstring
by David Goldsmith 11 Jul '10

11 Jul '10

Hi! I'm a little confused: in the docstring for numpy.fft we find the following: "For an even number of input points, A[n/2] represents both positive and negative Nyquist frequency..." but according to http://en.wikipedia.org/wiki/Nyquist_frequency (I know, I know, I've bad mouthed Wikipedia in the past, but that's in a different context): "The *Nyquist frequency*...is half the sampling frequency<http://en.wikipedia.org/wiki/Sampling_frequency>of a discrete signal <http://en.wikipedia.org/wiki/Discrete_signal> processing system...The Nyquist frequency should not be confused with the *Nyquist rate<http://en.wikipedia.org/wiki/Nyquist_rate> *, which is the lower bound of the sampling frequency that satisfies the Nyquist sampling criterion for a given signal or family of signals...*Nyquist rate*, as commonly used with respect to sampling, is a property of a continuous-time signal <http://en.wikipedia.org/wiki/Continuous-time_signal>, not of a system, whereas *Nyquist frequency* is a property of a discrete-time system, not of a signal." Yet earlier in numpy.fft's docstring we find: "...the discretized input to the transform is customarily referred to as a * signal...*" Should we be using "Nyquist rate" instead of "Nyquist frequency," and if not, why not? DG

3 2

f2py problem with complex inout in subroutine
by Mark Bakker 11 Jul '10

11 Jul '10

Hello list. The following subroutine fails to compile with f2py. I use a complex variable with intent(inout). It works fine with two real variables, so I have a workaround, but it would be nicer with a complex variable. Any thoughts on what I am doing wrong? Thanks, Mark module bessel contains subroutine test( zxa ) complex(kind=8), intent(inout) :: zxa zxa = cmplx(1.d0,2.d0,8) return end subroutine test end module bessel

2 1

ANN: Numpy runs on Python 3
by Pauli Virtanen 10 Jul '10

10 Jul '10

Hi, As many of you probably already know, Numpy works fully on Python 3 and Python 2, with a *single code base*, since March. This work is scheduled to be included in the next releases 1.5 and 2.0. Porting Scipy to work on Python 3 has proved to be much less work, and will probably be finished "soon". (Ongoing work is here: http:// github.com/cournape/scipy3/commits/py3k , http://github.com/pv/scipy-work/ commits/py3k ) For those who are interested in already starting to port their stuff to Python 3, you can use Numpy's SVN trunk version. Grab it: svn clone http://svn.scipy.org/svn/numpy/trunk/ numpy cd numpy python3 setup.py build An important point is that supporting Python 3 and Python 2 in the same code base can be done, and it is not very difficult either. It is also much preferable from the maintenance POV to creating separate branches for Python 2 and 3. We attempted to log changes needed in Numpy at http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt which may be useful (although not completely up-to-date) information for people wanting to do make the same transition in their own code. (Announcement as recommended by our PR department @ EuroScipy :) -- Pauli Virtanen

3 2

numpy, matplotlib and masked arrays
by Peter Isaac 10 Jul '10

10 Jul '10

hello fellow numpyneers, I'm not sure if this is a problem with numpy, matplotlib, EPD or me but here goes. The general problem is that either matplotlib has stopped plotting masked arrays correctly or numpy has changed the way masked arrays are presented. Or there is something strange with the Enthought Python Distribution or me. Many would agree with the latter ... First up, the platform: Linux pisaac-laptop 2.6.32-23-generic #37-Ubuntu SMP Fri Jun 11 07:54:58 UTC 2010 i686 GNU/Linux Next the symptom: matplotlib stops plotting masked array values at the first occurrence of a True mask value. Then the script: from matplotlib import pyplot import numpy a = numpy.array(range(0,100)) a[50] = -9999.0 b = numpy.ma.masked_where(a==-9999,a) pyplot.subplot(211) pyplot.plot(a) pyplot.subplot(212) pyplot.plot(b) pyplot.show() Now the twists: The above script works correctly with the Python installation from the Ubuntu repositories (10.04 LTS). Details of the installation are: Python V2.6.5 (r265:79063, Apr 16 2010, 13:09:56) matplotlib '0.99.1.1' numpy '1.3.0' The above script does not work correctly in either EPD-6.1-1 or EPD-6.2-2. Their details are: EPD-6.1-1 Python 2.6.4 |EPD 6.1-1 (32-bit)| (r264:75706, Nov 4 2009, 15:27:32) matplotlib '0.99.1.1' numpy '1.4.0' EPD-6.2-2 Python 2.6.5 |EPD 6.2-2 (32-bit)| (r265:79063, Mar 22 2010, 17:31:34) matplotlib '0.99.3' numpy '1.4.0' Since the matplotlib version is the same for the repository Python installation and the EPD-6.1-1 installation and the script works in the first but not the second, it suggests the problem is not just matplotlib. That seems to leave the numpy version change (from 1.3 to 1.4) or something in the EPD. Or me. Note that EPD-6.2-2 works fine with this script on WinXP. Any suggestions welcome and happy to supply further details if required. Yours in masked confusion, Peter

4 4

TypeError when using double , longdouble in numpy.dot
by Christoph Gohlke 08 Jul '10

08 Jul '10

Dear NumPy developers, I am trying to solve some scipy.sparse TypeError failures reported in [1] and reduced them to the following example: >>> import numpy >>> a = numpy.array([[1]]) >>> numpy.dot(a.astype('single'), a.astype('longdouble')) array([[1.0]], dtype=float64) >>> numpy.dot(a.astype('double'), a.astype('longdouble')) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: array cannot be safely cast to required type Is this exception expected? Also I noticed this: >>> numpy.array([1]).astype('longdouble').dtype.num 13 >>> numpy.array([1.0]).astype('longdouble').dtype.num 12 I am using Python 2.6.5 for Windows and numpy 1.4.1 compiled with msvc9, where sizeof(longdouble) == sizeof(double). [1] http://aspn.activestate.com/ASPN/Mail/Message/scipy-user/3875416 -- Christoph

3 7

Memory usage of numpy-arrays
by Hannes Bretschneider 08 Jul '10

08 Jul '10

Dear NumPy developers, I have to process some big data files with high-frequency financial data. I am trying to load a delimited text file having ~700 MB with ~ 10 million lines using numpy.genfromtxt(). The machine is a Debian Lenny server 32bit with 3GB of memory. Since the file is just 700MB I am naively assuming that it should fit into memory in whole. However, when I attempt to load it, python fills the entire available memory and then fails with Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.6/site-packages/numpy/lib/io.py", line 1318, in genfromtxt errmsg = "\n".join(errmsg) MemoryError Is there a way to load this file without crashing? Thanks, Hannes

5 5