Mailman 3 March 2007 - NumPy-Discussion

New Trac feature: TracReSTMacro
by Jeff Strunk 20 Mar '07

20 Mar '07

Good afternoon, By request, I have installed the TracReSTMacro on the numpy, scipy, and scikits tracs. This plugin allows you to display ReST formatted text directly from svn. For example, http://projects.scipy.org/neuroimaging/ni/wiki/ReadMe in its entirety is: [[ReST(/ni/trunk/README)]] Thank you, Jeff Strunk IT Administrator Enthought, Inc.

2 3

Grouping & Collapsing multi-dimensional data
by Mike Biglan 20 Mar '07

20 Mar '07

I might be using the wrong terminology but I'm trying to take a 2d array where each row has a department object and then 36 floats after it, eg: [dept1, 3,6,7...] With SQL or R i know how to collapse a simple 2d data structure like this. For example in SQL: select dept, stddev(field1)... from tbl_x group by dept But I want to end up with either a collapsed table grouped on dept where each element is some summary statistic, or better yet where each value is a dictionary of name=summary statistic and value = value of that statistic. Just to sort and perform this by group has been problematic because it gives me an error when sorting by an object if that object has __cmp__ method. I can do an index-based sort after using a field in the object -- but then must figure out the groups of rows manually. There must be easier methods to collapse on a dimension, grouping by one or more elements and applying arbitrary functions. Can this be done with numpy? Should i work with scipy? thanks - mike biglan ps: my example here is 2d but i'm hoping that this functionality would work for any-d

1 0

Nonblocking Plots with Matplotlib
by Bill Baxter 20 Mar '07

20 Mar '07

Howdy Folks, I was missing the good ole days of using Matlab back at the Uni when I could debug my code, stop at breakpoints and plot various data without fear of blocking the interpreter process. Using "ipython -pylab" is what has been suggested to me in the past, but the problem is I don't do my debugging from ipython. I have a very nice IDE that works very well, and it has a lovely interactive debugging prompt that I can use to probe my code when stopped at a breakpoint. It's great except I can't really use matplotlib for debugging there because it causes things to freeze up. So I've come up with a decent (though not perfect) solution for quickie interactive plots which is to run matplotlib in a separate process. I call the result it 'ezplot'. The first alpha version of this is now available at the Cheeseshop. (I made an egg too, so if you have setuptools you can do "easy_install ezplot".) The basic usage is like so: In [1]: import ezplot In [2]: p = ezplot.Plotter() In [3]: p.plot([1,2,3],[1,4,9],marker='o') Connecting to server... waiting... connected to plotserver 0.1.0a1 on http://localhost:8397 Out[3]: True In [4]: from numpy import * In [5]: x = linspace(-5,5,20) In [13]: p.clf() Out[13]: True In [14]: p.plot(x, x*x*log(x*x+0.01)) (Imagine lovely plots popping up on your screen as these commands are typed.) The only return values you get back are True (success...probably) or False (failure...for sure). So no fancy plot object manipulation is possible. But you can do basic plots no problem. The nice part is that this (unlike ipython's built-in -pylab threading mojo) should work just as well from wherever you're using python. Whether it's ipython (no -pylab) or Idle, or a plain MS-DOS console, or WingIDE's debug probe, or SPE, or a PyCrust shell or whatever. It doesn't matter because all the client is doing is packing up data and shipping over a socket. All the GUI plotting mojo happens in a completely separate process. There are plenty of ways this could be made better, but for me, for now, this probably does pretty much all I need, so it's back to Real Work. But if anyone is interested in making improvements to this, let me know. Here's a short list of things that could be improved: * Right now I assume use of the wxAGG backend for matplotlib. Don't know how much work it would be to support other back ends (or how to go about it, really). wxAGG is what I always use. * Returning more error/exception info from the server would be nice * Returning full fledged proxy plot objects would be nice too, but I suspect that's a huge effort * SOAP may be better for this than xmlrpclib but I just couldn't get it to work (SOAPpy + Twisted). * A little more safety would be nice. Anyone know how to make a Twisted xmlrpc server not accept connections from anywhere except localhost? * There's a little glitch in that the spawned plot server dies with the parent that created it. Maybe there's a flag to subprocess.Popen to fix that? * Sometimes when you click on "Exit Server", if there are plot windows open it hangs while shutting down. Only tested on Win32 but there's nothing much platform specific in there. Give it a try and let me know what you think! --bb

5 9

Re: [Numpy-discussion] scalar recordarrays
by Matthew Koichi Grimes 19 Mar '07

19 Mar '07

Francesc Altet wrote: > with a > rank-0 'recarr', 'recarr.x' should return a rank-0 array (for > consistency), but it doesn't: > > In [74]:recarr=numpy.rec.array((1.0, 0, 3), dtype) > In [75]:recarr.x > Out[75]:1.0 > In [76]:type(recarr.x) > Out[76]:<type 'numpy.float64'> > > While I find this inconsistent, I'm not sure whether it should be be > fixed or not because the boundaries between rank-0 and scalar objects > are always a bit fuzzy. Travis can probably bring more light here. > Here's my case for "yes, it's a problem that should be fixed (please?)": In my particular project, this inconsistency becomes a pervasive problem when writing vectorized functions that are supposed to accept and return tensors of various ranks. To accomodate the inconsistency, one has to take what would have been a simple statement of the form: <snip> output.x[...] = blah </snip> and replace it with a bunch of boolean switches: <snip> if output.ndim == 0: output = output.reshape([1]) outputWas0dim = True else: outputWas0dim = False output.x[...] = blah if outputWas0dim: output = output.reshape([]) return output </snip> -- Matt

2 1

in place random generation
by Daniel Mahler 19 Mar '07

19 Mar '07

Is there an efficient way to fill an existing array with random numbers without allocating a new array? thanks Daniel

11 30

scalar recordarrays
by Matthew Koichi Grimes 19 Mar '07

19 Mar '07

I've found that if I have a record array of shape [] (i.e. a scalar recarray), I can't set its fields. >>> recarr recarray((0.0, 0.0, 0.0), dtype=[('x', '<f8'), ('dx', '<f8'), ('delta', '<f8')]) >>> recarr.x[...] = 1.0 TypeError: object does not support item assignment In the above, recarr.x returned a float "0.0", then attempted the assignment "0.0 = 1.0", which raised the TypeError. This is in contrast with the behavior of scalar ndarrays, where the above works as expected: >>> arr array(0.0) >>> arr[...] = 1.0 >>> arr array(1.0) I this a bug, or is there some reason for this behavior? The obvious workaround is reshape a scalar to a size-1 array before setting .x: >>> if recarr.ndim == 0: ....: recarr.reshape([1]).x[...] = 1.0 else: recarr.x[...] = 1.0 ....: else: ....: recarr.x[...] = 1.0 But this is cumbersome and easy to forget to do. -- Matt

2 1

Is the select function broken in numpy?
by Cory Davis 18 Mar '07

18 Mar '07

I have a few day old svn update of numpy where the following code produces an error, yet not in numpy-0.9.8 from numpy import * print select([array([1]),array([0])],[array([2]),array([3])]) File "/tmp/tmpY-7FJN/lib64/python/numpy/lib/function_base.py", line 426, in select File "/tmp/tmpY-7FJN/lib64/python/numpy/core/fromnumeric.py", line 151, in choose TypeError: array cannot be safely cast to required type [ Any clues?

1 2

import numpy segmentation fault
by Cory Davis 17 Mar '07

17 Mar '07

Hi there, I have just installed numpy-1.0.1 from source, which seemed to go fine. However when I try to "import numpy" I get a segmentation fault. A have a 64 bit machine running RedHat Enterprise Linux and Python 2.34 Any clues greatly appreciated. Cheers, Cory.

2 2

numpy.distutils, windows dll vs lib
by David Cournapeau 16 Mar '07

16 Mar '07

Hi, recently, I got some problems detecting a dynamic library (dll) with numpy.distutils. Basically, I have a package which uses a class derived from system_info from numpy.distutils to detect a dll I use through ctypes. If only the dll is present, my numpy.distutils.system_info derived class does not find the library; if the .lib is present too, then it is detected. Why is that ? Can I modify my class do detecting the dll is enough ? I don't know how windows dynamic linking and dynamically loaded libraries work, and I am kind of confused by this (I thought .lib was .a, and .dll was .so, and that symbols were not exported by default on windows contrary to Unix, but the difference seems more subtle than this....). cheers, David

3 5

zoom FFT with numpy?
by Ray S 16 Mar '07

16 Mar '07

We'd like to do what most call a "zoom FFT"; we only are interested in the frequencies of say, 6kHZ to 9kHz with a given N, and so the computations from DC to 6kHz are wasted CPU time. Can this be done without additional numpy pre-filtering computations? If explicit filtering is needed to "baseband" the data, is it worth it? It sounds like we need to generate cosine data once for each band and N, then multiple with the data for each FFT. Has anyone coded this up before? I couldn't find any references other than http://www.dsprelated.com/showmessage/38917/1.php and http://www.numerix-dsp.com/siglib/examples/test_zft.c (which uses Numerix). Ray

4 3