Mailman 3 April 2006 - NumPy-Discussion

Floating point exception with numpy and embedded python interpreter
by Arkaitz Bitorika 19 Apr '06

19 Apr '06

Hi, I'm embedding Python in a big C++ program (the NS network simulator) and I have problems when importing the numpy module, I get a Floating Point exception. The C code that causes the exception is: Py_Initialize(); PyObject* module = PyImport_ImportModule("numpy"); Py_DECREF(module); I'm running Ubuntu Breezy on a dual processor Dell machine, with the stock python and numpy 0.9.6. One strange thing is that I haven't been able to reproduce the crash by writing a minimal C program with the code above, it only crashes when added to my program. I've been embedding Python for ages on the same program and other modules work fine, only numpy fails. I've debugged the issue a bit and I've seen that the exception is thrown when the numpy __init__.py tries to import the core module. The GDB backtrace is pasted at the end. Any idea what may be going wrong? Thanks, Arkaitz 0xb7900fd2 in initumath () at build/src/numpy/core/src/umathmodule.c:10321 10321 pinf *= mul; (gdb) bt #0 0xb7900fd2 in initumath () at build/src/numpy/core/src/umathmodule.c:10321 #1 0xb7e4e310 in _PyImport_LoadDynamicModule () from /usr/lib/libpython2.4.so.1.0 #2 0xb7e4c450 in _PyImport_FindModule () from /usr/lib/libpython2.4.so.1.0 #3 0xb7e4cc01 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #4 0xb7e4ce26 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #5 0xb7e4d2c6 in PyImport_ImportModuleEx () from /usr/lib/libpython2.4.so.1.0 #6 0xb7e22d9e in _PyUnicodeUCS4_ToLowercase () from /usr/lib/libpython2.4.so.1.0 #7 0xb7df5923 in PyCFunction_Call () from /usr/lib/libpython2.4.so.1.0 #8 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #9 0xb7e2a92c in PyEval_CallObjectWithKeywords () from /usr/lib/libpython2.4.so.1.0 #10 0xb7e2e8f9 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0 #11 0xb7e31a2d in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0 #12 0xb7e31b76 in PyEval_EvalCode () from /usr/lib/libpython2.4.so.1.0 #13 0xb7e4a525 in PyImport_ExecCodeModuleEx () from /usr/lib/libpython2.4.so.1.0 #14 0xb7e4a8e9 in PyImport_ExecCodeModule () from /usr/lib/libpython2.4.so.1.0 #15 0xb7e4c73e in _PyImport_FindModule () from /usr/lib/libpython2.4.so.1.0 #16 0xb7e4cc01 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #17 0xb7e4ce26 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #18 0xb7e4d2c6 in PyImport_ImportModuleEx () from /usr/lib/libpython2.4.so.1.0 #19 0xb7e22d9e in _PyUnicodeUCS4_ToLowercase () from /usr/lib/libpython2.4.so.1.0 #20 0xb7df5923 in PyCFunction_Call () from /usr/lib/libpython2.4.so.1.0 #21 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #22 0xb7e2a92c in PyEval_CallObjectWithKeywords () from /usr/lib/libpython2.4.so.1.0 #23 0xb7e2e8f9 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0 #24 0xb7e31a2d in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0 #25 0xb7e31b76 in PyEval_EvalCode () from /usr/lib/libpython2.4.so.1.0 #26 0xb7e5667f in PyRun_String () from /usr/lib/libpython2.4.so.1.0 #27 0xb7e2fce6 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0 #28 0xb7e31a2d in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0 #29 0xb7e3011a in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0 #30 0xb7e31a2d in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0 #31 0xb7de31b6 in PyFunction_SetClosure () from /usr/lib/libpython2.4.so.1.0 #32 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #33 0xb7dd079b in PyMethod_New () from /usr/lib/libpython2.4.so.1.0 #34 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #35 0xb7dcfd7b in PyInstance_NewRaw () from /usr/lib/libpython2.4.so.1.0 #36 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #37 0xb7e2f5d2 in PyEval_EvalFrame () from /usr/lib/libpython2.4.so.1.0 #38 0xb7e31a2d in PyEval_EvalCodeEx () from /usr/lib/libpython2.4.so.1.0 #39 0xb7e31b76 in PyEval_EvalCode () from /usr/lib/libpython2.4.so.1.0 #40 0xb7e4a525 in PyImport_ExecCodeModuleEx () from /usr/lib/libpython2.4.so.1.0 #41 0xb7e4a8e9 in PyImport_ExecCodeModule () from /usr/lib/libpython2.4.so.1.0 #42 0xb7e4c73e in _PyImport_FindModule () from /usr/lib/libpython2.4.so.1.0 #43 0xb7e4cc01 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #44 0xb7e4ce26 in PyImport_ReloadModule () from /usr/lib/libpython2.4.so.1.0 #45 0xb7e4d2c6 in PyImport_ImportModuleEx () from /usr/lib/libpython2.4.so.1.0 #46 0xb7e22d9e in _PyUnicodeUCS4_ToLowercase () from /usr/lib/libpython2.4.so.1.0 #47 0xb7df5923 in PyCFunction_Call () from /usr/lib/libpython2.4.so.1.0 #48 0xb7dc8fdf in PyObject_Call () from /usr/lib/libpython2.4.so.1.0 #49 0xb7dcc6c0 in PyObject_CallFunction () from /usr/lib/libpython2.4.so.1.0 #50 0xb7e4d745 in PyImport_Import () from /usr/lib/libpython2.4.so.1.0 #51 0xb7e4d918 in PyImport_ImportModule () from /usr/lib/libpython2.4.so.1.0

2 1

using NaN, INT_MIN etc in ndarray instead of a masked array
by Michael Sorich 18 Apr '06

18 Apr '06

On 4/8/06, Sasha <ndarray(a)mac.com> wrote: > > ... > See above. For ndarray mask is always False unless an add-on module is > loaded that redefines arithmetic to recognize special bit-patterns > such as NaN or INT_MIN. > > Is it possible to implement masked values using these special bit patterns in the ndarray instead of using a separate MA class? If so has there been any thought as to whether this may be the better option. I think it would be preferable if the ability to handle masked data was available in the standard array class (ndarray), as this would increase the likelihood that functions built for numeric arrays will handle masked values well. It seems that ndarray already has decent support for nans (isnan() returns the equivalent of a boolean mask array), indicating that such an approach may be acceptable. How difficult is it to generalise the concept to other data types (int, string, bool)? Mike

6 6

Multiple inheritance from ndarray
by Robert Lupton 18 Apr '06

18 Apr '06

I have a swig extension that defines a class that inherits from both a personal C-coded image struct (actImage), and also from Numeric's UserArray. This works very nicely, but I thought that it was about time to upgrade to numpy. The code looks like: from UserArray import * class Image(UserArray, actImage): def __init__(self, *args): actImage.__init__(self, *args) UserArray.__init__(self, self.getArray(), 'd', copy=False, savespace=False) I can't figure out how to convert this to use ndarray, as ndarray doesn't seem to have an __init__ method, merely a __new__. So what's the approved numpy way to handle multiple inheritance? I've a nasty idea that this is a python question that I should know the answer to, but I'm afraid that I don't... R

4 8

Trac Wikis closed for anonymous edits until further notice
by Robert Kern 18 Apr '06

18 Apr '06

We've been hit badly by spammers, so I can only presume our Trac sites are now on the traded spam lists. I am going to turn off anonymous edits for now. Ticket creation will probably still be left open for now. Many thanks to David Cooke for quickly removing the spam. I am looking into ways to allow people to register themselves with the Trac sites so they can edit the Wikis and submit tickets without needing to be added by a project admin. -- Robert Kern robert.kern(a)gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

2 1

String (and unicode) comparisons and per-thread error handling fixed
by Travis Oliphant 18 Apr '06

18 Apr '06

String comparisons were added last week. Today, I added per-thread error handling to NumPy. There is 1 more enhancement (scalar math) prior to 0.9.8 release --- but it will probably take 1-2 weeks. The new error handling means that the three-scope system is gone. Now, there is only one per-Python-thread global scope for error handling. If you change the error handling it will affect all ufuncs. Because of this, the seterr function now returns an object with the old error-handling information. This object must be passed to umath.seterrobj() in order to restore the error handling. -Travis

2 2

Toward release 1.0 of NumPy
by Travis Oliphant 17 Apr '06

17 Apr '06

The next release of NumPy will be 0.9.8 Before this release is made, I want to make sure the following tickets are implemented http://projects.scipy.org/scipy/numpy/ticket/54 http://projects.scipy.org/scipy/numpy/ticket/55 http://projects.scipy.org/scipy/numpy/ticket/56 Once 0.9.8 is out, I'd like to name the next release NumPy 1.0 Release Candidate 1 and have a series of release candidates so that hopefully by SciPy 2006 conference, NumPy 1.0 is out. This also dove-tails nicely with the Python 2.5 release schedule so that NumPy 1.0 should work with Python 2.5 and be fully 64-bit capable for handling very-large arrays. The recent discussions and bug-reports have been very helpful. If you have found a bug, please report it on the Trac pages so that we don't lose sight of it. Report bugs by "submitting a ticket" here: http://projects.scipy.org/scipy/numpy/newticket -Travis

7 21

where
by Ryan Krauss 17 Apr '06

17 Apr '06

Can someone help me understand the proper use of where? I want to use it like this myvect=where(f>19.5 and phase>0, f, phase) but I seem to be getting or rather than and. Thanks, Ryan

4 11

Vectorize bug
by Stefan van der Walt 16 Apr '06

16 Apr '06

Hello all Vectorize segfaults for large arrays. I filed the bug at http://projects.scipy.org/scipy/numpy/ticket/52 The offending code is import numpy as N x = N.linspace(-3,2,10000) y = N.vectorize(lambda x: x) # Segfaults here y(x) Regards Stéfan

6 15

Re: [Numpy-discussion] Code Question
by Saqib bin Sohail 16 Apr '06

16 Apr '06

Thanks Guys for all your prompt responses. I have tried to use the provided solutions but I am had my share of issues mixed with my lack of knowledge to the point that I feel quite embarrassed to bother you guys. Issue 1 I am running FC 3 with native python-2.3 and then I installed python-2.4 in it. numarray-1.5.1 seems to have installed with success in python-2.3. I have tried to install numpy-0.9.6-1.i586.rpm but I don't have python-base and when I try to install python-base I get a long list of dependency lists which I need. I haven't further pursued down that line, unfortunately I haven't been able to use numarray, I don't know how to use it because ppl have repeatedly told me to use numpy but I can't seem to get that installed. Issue 2 To input the file, Ryan suggested to use scipy, I don't want to go down that path, if only there is a simple way to input the file, (i can clean up the file and format it in the right way in perl, I can do that in a heartbeat) Issue 3 I don't want to use gnuplot functionality, or mathplot, if only I am able to write the file then again I can use perl to format it and use gnuplot then, So if there is the simplest of ways in which I can just i) read the file (formatting will be done in perl) ii) get the fft iii) write the file or files (and then use perl to format for gnuplot) I am sure all of you will say why not use the existing functionalities, but after 3 days I haven't gotten anywhere. All I need to do is get FFT of some sound files so that I can verify the result of FFT's and compare them with my FFT code in VxWorks. An Pierre, I started reading diveintopython.pdf but got nowhere when I tried two of its examples, the attached image shows that when I tried to run one of the examples on python-2.3 and the output wasn't according to what the guide suggested. (no output to be precise) http://jobim.colorado.edu/~sohail/pythonExample.JPG Thanks again guys. Quoting Ryan Krauss <ryanlists(a)gmail.com>: > I guess it depends on how much you want to learn and what you want to do. > > I was able to load your data using > data=scipy.io.read_array('monkey.dat') > > I had to comment out the first line to make it work. I couldn't make > the fromfile method of numpy work because the data is actually fixed > width. > > If you don't want to install scipy, you would need to learn enough > Python to read the file and clean it up a little by hand. > > It seems like the first column is time and the second is the signal > you want to fft. I was able to fft it with: > myfft=numpy.fft(data[:,1]) > (I don't have the latest version of numpy and don't seem to have the > refft function Robert mentioned). > > t=data[:,0] > df=1/max(t) > df > maxf=8012 > fvect=arange(0,maxf+df,df) > > plot(fvect,abs(myfft)) > > I am plotting using matplotlib and the resulting figures are attached. > > If you really want to learn python for scientific and plotting > applications, I would highly recommend a few packages: > SciPy - some additional capabilities beyond Numpy (optimization, ode's , ...) > ipython - it is a really good interactive python shell > matplotlib - the best python 2d plotting package I am aware of > > Let me know if you have any additional questions. You can find out > about each package by googling it. They are all closely related to > Numpy and all have good mailing lists to help you. > > Ryan > > On 4/15/06, Saqib bin Sohail <Saqib.Sohail(a)colorado.edu> wrote: > > Do let me know if you get somewhere. > > > > Thanks > > > > > > Quoting Ryan Krauss <ryanlists(a)gmail.com>: > > > > > email me the dat file and I could play with it a bit. If I can read > > > your input file, the rest should be easy. > > > > > > Ryan > > > > > > On 4/15/06, Saqib bin Sohail <Saqib.Sohail(a)colorado.edu> wrote: > > > > Hi guys > > > > > > > > I have never used python, but I wanted to compute FFT of audio files, I > > > came > > > > upon a page which had python code, so I installed Numpy but after > beating > > > the > > > > bush for a few days, I have finally come in here to ask. After taking > the > > > FFT I > > > > want to output it to a file and the use gnuplot to plot it. > > > > > > > > When I instaled NumPy, and ran the tests, it seemed that all passed > without > > > a > > > > problem. My input is a .dat file converted from .wav file by sox. > > > > > > > > Here is the code which obviously doesn't work because it seems that > changes > > > > have occured since this code was written. (not my code, just from some > > > website > > > > where a guy had written on how to do things which i require) > > > > > > > > import Numeric > > > > import FFT > > > > out_array=Numeric.array(out) > > > > out_fft=FFT.fft(out) > > > > > > > > offt=open('outfile_fft.dat','w') > > > > for x in range(len(out_fft)/2): > > > > offt.write('%f %f\n'%(1.0*x/wtime,abs(out_fft[x].real))) > > > > > > > > > > > > I do the following at the python prompt > > > > > > > > import numarray > > > > myFile = open('test.dat', 'r') > > > > my_array = numarray.arra(myFile) > > > > > > > > /* at this stage I wanted to see if it was correctly read */ > > > > > > > > print myArray > > > > [1632837691 1701605485 1952535072 ..., 538976288 538976288 > 168632368] > > > > > > > > it seems that these values do not correspond to the values in the file > (but > > > I > > > > guess the array is considering these as ints when infact these are > floats) > > > > > > > > anyway the problem starts when i try to do fft, because I can't seem to > > > find > > > > module or how to invoke it, > > > > > > > > the second problem is writing to the file, that code obviously doesn't > > > work, > > > > and in my search through various documentations, i found arrayrange() > but > > > > couldn't make it to work, call me stupid, but despite going through > several > > > > examples, i haven't been able to make the for loop worked in any case, > > > > > > > > it would be very kind of someone if he could at least tell me what i am > > > doing > > > > wrong and reply a simple example so that I can modify my code or at > least > > > be > > > > able to understand . > > > > > > > > Thanks > > > > > > > > > > > > > > > > -- > > > > Saqib bin Sohail > > > > PhD ECE > > > > University of Colorado at Boulder > > > > Res: (303) 786 0636 > > > > http://ucsu.colorado.edu/~sohail/index.html > > > > > > > > > > > > ------------------------------------------------------- > > > > This SF.Net email is sponsored by xPML, a groundbreaking scripting > language > > > > that extends applications into web and mobile media. Attend the live > > > webcast > > > > and join the prime developer group breaking into this new coding > territory! > > > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 > > > > _______________________________________________ > > > > Numpy-discussion mailing list > > > > Numpy-discussion(a)lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > > > > > > > > > > -- > > Saqib bin Sohail > > PhD ECE > > University of Colorado at Boulder > > Res: (303) 786 0636 > > http://ucsu.colorado.edu/~sohail/index.html > > > > > -- Saqib bin Sohail PhD ECE University of Colorado at Boulder Res: (303) 786 0636 http://ucsu.colorado.edu/~sohail/index.html

3 3

where() has started returning a tuple!?
by Andrew Jaffe 16 Apr '06

16 Apr '06

I think the following behavior is (only recently) wrong: In [7]: numpy.__version__ Out[7]: '0.9.7.2360' In [8]: numpy.nonzero([True, False, True]) Out[8]: array([0, 2]) In [9]: numpy.where([True, False, True]) Out[9]: (array([0, 2]),) Note the tuple output to where(), which should be the same as nonzero. Andrew

2 2