Mailman 3 July 2010 - NumPy-Discussion

match or vectorized in-type function.
by James Bullard July 12, 2010

July 12, 2010

I have two vectors of integers of not necessarily the same length. Consider the hypothetical function match (or if you are familiar to R then consider that function). match(v1, v2) => returns a boolean array of length len(v1) indicating whether element i in v1 is in v2. I cannot find this function in numpy. I would assume a variant is there, but not being able to find it I wrote it myself. First, is there such a function? Second, if there is not, is this implementation reasonable? thanks, jim from numpy import * a2 = random.randint(1, 1000, 1000) a1 = random.randint(1, 10000000, 100000) def match(v1, v2): d = dict(zip(v2, range(0, len(v2)))) return array([ d.has_key(x) for x in v1])

2 1

3 dim array unpacking
by K.-Michael Aye July 12, 2010

July 12, 2010

Dear numpy hackers, I can't find the syntax for unpacking the 3 dimensions of a rgb array. so i have a MxNx3 image array 'img' and would like to do: red, green, blue = img[magical_slicing] Which slicing magic do I need to apply? Thanks for your help! BR, Michael

2 1

Another reality check
by David Goldsmith July 12, 2010

July 12, 2010

In numpy.fft we find the following: "Then A[1:n/2] contains the positive-frequency terms, and A[n/2+1:] contains the negative-frequency terms, in order of decreasingly negative frequency." Just want to confirm that "decreasingly negative frequency" means ..., A[n-2] = A_(-2), A[n-1] = A_(-1), as implied by our definition (attached). DG

3 4

Function returns nothing!
by allan oware July 12, 2010

July 12, 2010

Hi All! def height_diffs(): h = [] i = 0 while True: x = raw_input("Enter height difference ") if x == 'q': break else: h.append(x) i = i + 1 m = asarray(h,dtype=float) return m why does return statement return nothing?

2 1

"Nyquist frequency" in numpy.fft docstring
by David Goldsmith July 11, 2010

July 11, 2010

Hi! I'm a little confused: in the docstring for numpy.fft we find the following: "For an even number of input points, A[n/2] represents both positive and negative Nyquist frequency..." but according to http://en.wikipedia.org/wiki/Nyquist_frequency (I know, I know, I've bad mouthed Wikipedia in the past, but that's in a different context): "The *Nyquist frequency*...is half the sampling frequency<http://en.wikipedia.org/wiki/Sampling_frequency>of a discrete signal <http://en.wikipedia.org/wiki/Discrete_signal> processing system...The Nyquist frequency should not be confused with the *Nyquist rate<http://en.wikipedia.org/wiki/Nyquist_rate> *, which is the lower bound of the sampling frequency that satisfies the Nyquist sampling criterion for a given signal or family of signals...*Nyquist rate*, as commonly used with respect to sampling, is a property of a continuous-time signal <http://en.wikipedia.org/wiki/Continuous-time_signal>, not of a system, whereas *Nyquist frequency* is a property of a discrete-time system, not of a signal." Yet earlier in numpy.fft's docstring we find: "...the discretized input to the transform is customarily referred to as a * signal...*" Should we be using "Nyquist rate" instead of "Nyquist frequency," and if not, why not? DG

3 2

f2py problem with complex inout in subroutine
by Mark Bakker July 11, 2010

July 11, 2010

Hello list. The following subroutine fails to compile with f2py. I use a complex variable with intent(inout). It works fine with two real variables, so I have a workaround, but it would be nicer with a complex variable. Any thoughts on what I am doing wrong? Thanks, Mark module bessel contains subroutine test( zxa ) complex(kind=8), intent(inout) :: zxa zxa = cmplx(1.d0,2.d0,8) return end subroutine test end module bessel

2 1

ANN: Numpy runs on Python 3
by Pauli Virtanen July 10, 2010

July 10, 2010

Hi, As many of you probably already know, Numpy works fully on Python 3 and Python 2, with a *single code base*, since March. This work is scheduled to be included in the next releases 1.5 and 2.0. Porting Scipy to work on Python 3 has proved to be much less work, and will probably be finished "soon". (Ongoing work is here: http:// github.com/cournape/scipy3/commits/py3k , http://github.com/pv/scipy-work/ commits/py3k ) For those who are interested in already starting to port their stuff to Python 3, you can use Numpy's SVN trunk version. Grab it: svn clone http://svn.scipy.org/svn/numpy/trunk/ numpy cd numpy python3 setup.py build An important point is that supporting Python 3 and Python 2 in the same code base can be done, and it is not very difficult either. It is also much preferable from the maintenance POV to creating separate branches for Python 2 and 3. We attempted to log changes needed in Numpy at http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt which may be useful (although not completely up-to-date) information for people wanting to do make the same transition in their own code. (Announcement as recommended by our PR department @ EuroScipy :) -- Pauli Virtanen

3 2

numpy, matplotlib and masked arrays
by Peter Isaac July 10, 2010

July 10, 2010

hello fellow numpyneers, I'm not sure if this is a problem with numpy, matplotlib, EPD or me but here goes. The general problem is that either matplotlib has stopped plotting masked arrays correctly or numpy has changed the way masked arrays are presented. Or there is something strange with the Enthought Python Distribution or me. Many would agree with the latter ... First up, the platform: Linux pisaac-laptop 2.6.32-23-generic #37-Ubuntu SMP Fri Jun 11 07:54:58 UTC 2010 i686 GNU/Linux Next the symptom: matplotlib stops plotting masked array values at the first occurrence of a True mask value. Then the script: from matplotlib import pyplot import numpy a = numpy.array(range(0,100)) a[50] = -9999.0 b = numpy.ma.masked_where(a==-9999,a) pyplot.subplot(211) pyplot.plot(a) pyplot.subplot(212) pyplot.plot(b) pyplot.show() Now the twists: The above script works correctly with the Python installation from the Ubuntu repositories (10.04 LTS). Details of the installation are: Python V2.6.5 (r265:79063, Apr 16 2010, 13:09:56) matplotlib '0.99.1.1' numpy '1.3.0' The above script does not work correctly in either EPD-6.1-1 or EPD-6.2-2. Their details are: EPD-6.1-1 Python 2.6.4 |EPD 6.1-1 (32-bit)| (r264:75706, Nov 4 2009, 15:27:32) matplotlib '0.99.1.1' numpy '1.4.0' EPD-6.2-2 Python 2.6.5 |EPD 6.2-2 (32-bit)| (r265:79063, Mar 22 2010, 17:31:34) matplotlib '0.99.3' numpy '1.4.0' Since the matplotlib version is the same for the repository Python installation and the EPD-6.1-1 installation and the script works in the first but not the second, it suggests the problem is not just matplotlib. That seems to leave the numpy version change (from 1.3 to 1.4) or something in the EPD. Or me. Note that EPD-6.2-2 works fine with this script on WinXP. Any suggestions welcome and happy to supply further details if required. Yours in masked confusion, Peter

4 4

TypeError when using double , longdouble in numpy.dot
by Christoph Gohlke July 8, 2010

July 8, 2010

Dear NumPy developers, I am trying to solve some scipy.sparse TypeError failures reported in [1] and reduced them to the following example: >>> import numpy >>> a = numpy.array([[1]]) >>> numpy.dot(a.astype('single'), a.astype('longdouble')) array([[1.0]], dtype=float64) >>> numpy.dot(a.astype('double'), a.astype('longdouble')) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: array cannot be safely cast to required type Is this exception expected? Also I noticed this: >>> numpy.array([1]).astype('longdouble').dtype.num 13 >>> numpy.array([1.0]).astype('longdouble').dtype.num 12 I am using Python 2.6.5 for Windows and numpy 1.4.1 compiled with msvc9, where sizeof(longdouble) == sizeof(double). [1] http://aspn.activestate.com/ASPN/Mail/Message/scipy-user/3875416 -- Christoph

3 7

Memory usage of numpy-arrays
by Hannes Bretschneider July 8, 2010

July 8, 2010

Dear NumPy developers, I have to process some big data files with high-frequency financial data. I am trying to load a delimited text file having ~700 MB with ~ 10 million lines using numpy.genfromtxt(). The machine is a Debian Lenny server 32bit with 3GB of memory. Since the file is just 700MB I am naively assuming that it should fit into memory in whole. However, when I attempt to load it, python fills the entire available memory and then fails with Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.6/site-packages/numpy/lib/io.py", line 1318, in genfromtxt errmsg = "\n".join(errmsg) MemoryError Is there a way to load this file without crashing? Thanks, Hannes

5 5