Mailman 3 January 2010 - SciPy-Dev

Is this a bugfix for scipy.hilbert?
by Ariel Rokem 19 Jan '10

19 Jan '10

Hi everyone, I have been trying to use scipy.signal.hilbert and I got the following puzzling result: In [22]: import scipy In [23]: scipy.__version__ #I have r6182 Out[23]: '0.8.0.dev' In [24]: import scipy.signal as signal In [25]: a = np.random.rand(100,100) In [26]: np.abs(signal.hilbert(a[-1])) Out[26]: array([ 0.57567681, 0.25918624, 0.50207097, 0.51834052, 0.24293389, 0.5779464 , 0.6515758 , 0.89973173, 1.00275444, 0.37352935, 0.62332717, 0.93599749, 0.40651376, 0.65088756, 0.8332281 , 0.5770101 , 0.9288512 , 0.46671906, 0.41536055, 0.71418068, 0.81250913, 0.07652627, 0.72939072, 0.26755626, 0.36396146, 0.59725999, 1.02264694, 0.41227986, 0.98122853, 0.71906675, 0.58582611, 0.77288117, 0.3217015 , 0.65261394, 0.11947618, 0.75632703, 0.43432935, 0.52182485, 1.0277177 , 1.01104986, 0.3023265 , 0.6024772 , 0.69257548, 0.55418735, 0.46259052, 0.25832231, 0.38278355, 0.45508532, 0.26215872, 0.34207947, 0.80704729, 0.80755477, 0.95317178, 0.97458885, 0.58762294, 0.82540618, 0.62005585, 0.82494646, 1.04221293, 0.14983027, 1.01571579, 0.99381328, 0.24158714, 0.84256569, 0.53418924, 0.24067628, 0.90489883, 1.02217747, 0.34988034, 0.5310065 , 0.48135002, 1.03020269, 0.6013679 , 0.46062485, 0.3918485 , 0.21554545, 0.31704519, 0.04868385, 0.1787766 , 0.37361852, 0.21977912, 0.7649772 , 0.77867281, 0.37684278, 0.64432638, 0.77494951, 0.87106309, 0.77611484, 0.52666801, 0.88683667, 0.69164967, 0.98618191, 0.84811375, 0.35934198, 0.32650478, 0.1752677 , 0.60574454, 0.5109132 , 0.52332287, 0.99777805]) In [27]: np.abs(signal.hilbert(a))[-1] Out[27]: array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) ---------------------------------------------------------------------- I was expecting both of these to have the same values - am I missing something? I think that the following solves this issue, but now I am not that sure whether it does what it is supposed to do and I couldn't find a test for this in test_signaltools.py. Does anyone know of a good test-case for the analytic signal, that I could create for this? Index: scipy/signal/signaltools.py =================================================================== --- scipy/signal/signaltools.py (revision 6182) +++ scipy/signal/signaltools.py (working copy) @@ -1062,13 +1062,13 @@ """ x = asarray(x) if N is None: - N = len(x) + N = x.shape[-1] if N <=0: raise ValueError, "N must be positive." if iscomplexobj(x): print "Warning: imaginary part of x ignored." x = real(x) - Xf = fft(x,N,axis=0) + Xf = fft(x,N,axis=-1) h = zeros(N) if N % 2 == 0: h[0] = h[N/2] = 1 @@ -1078,7 +1078,7 @@ h[1:(N+1)/2] = 2 if len(x.shape) > 1: - h = h[:, newaxis] + h = h[newaxis,:] x = ifft(Xf*h) return x Cheers, Ariel -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel

2 13

RFC: initial support for Harwell-Boeing files
by David Cournapeau 18 Jan '10

18 Jan '10

Hi, I have added basic support for Harwell-Boeing file format support, for sparse matrices. I would be happy to receive to get some reviews on the API before pushing this into the trunk, as well as bug reports from people who have matrices in such a format: http://github.com/cournape/scipy3/tree/hb_io The code is in scipy.sparse.io. Two API are supported: - a simple, function-based one: read_hb(filename) and write_hb(filename, m) - a more complete API based on HBFile and HBInfo, to control the exact format, get the metadata when reading. The simple API is just sugar on top on this one For now, only read/write is supported for Real, Unsymmetric, Assembled matrices, as that's what I need. There is also some code to deal with fortran-formatted field: maybe this can be useful outside HB IO, although I only parse a limited subset of fortran format for now (exponential format and integer format). cheers, David

1 0

Ticket 1070
by Nils Wagner 16 Jan '10

16 Jan '10

Hi all, Ticket http://projects.scipy.org/scipy/ticket/1070 can be closed. Nils

1 0

Python coders for Haiti disaster relief
by Peter Clarke 16 Jan '10

16 Jan '10

Apologies for off topic posting but I think this in an important project. Python programmers are required immediately for assistance in coding a disaster management framework for the Earthquake in Haiti. >From http://wiki.python.org/moin/VolunteerOpportunities: ----------------- URGENT REQUEST, Sahana Disaster Management System, Haiti Earthquake *Job Description*:This is an urgent call for experienced Python programmers to help in the Sahana Disaster Management System immediately - knowledge of Web2Py <http://wiki.python.org/moin/Web2Py> platform would be best. The Sahana Disaster Management System is used to coordinate relief efforts. Please recruit any available programmers for the Haiti effort as quickly as possible and have them contact me immediately so that I can put them in touch with the correct people. Thank you kindly and I do hope that we can quickly identify some contributors for this monumental effort - they are needed ASAP. http://sahanapy.org/ is the developer site and the demo is http://demo.sahanapy.org/ - *Contact*: Connie White, PhD, Institute for Emergency Preparedness, Jacksonville State University - *E-mail contact*: connie.m.white(a)gmail.com - *Web*: http://sahanapy.org/ ----------------------------- Please help if you can. -Peter Clarke

1 0

Please stress-test SVN version pf matlab reader
by Matthew Brett 14 Jan '10

14 Jan '10

Hi, For those of you interested in the matlab reader in scipy - a bout of 'flu left me with only enough energy to try and work out the last undocumented bits of the matlab file format. I _believe_ that the current SVN version of the reader: import scipy.io as sio a = sio.loadmat('your_matfile_here.mat') should successfully read any non-HDF matlab-written matfile - and this is just to ask if those of you, with mat files lying around, could try out the latest SVN version, and let me know that I am wrong, most usefully with some way of me being able to reproduce the problem. I also have the hope / belief that the scipy reader should be performing at round about the same speed as matlab to read the same file - please let me know if this is way off too. Thanks a lot, Matthew

3 5

build system for scikit
by Nathaniel Smith 14 Jan '10

14 Jan '10

So I've had some reports of problems with the scikits.sparse build system; I'm sure the general thrust will be familiar to many people here: -- Even on Linux, different distributions put the CHOLMOD headers in different places (argh wft people) -- Apparently numpy headers are not always placed in the Python include path? This seems broken to me, but okay... -- In principle, it might be nice to be able to build just some of the scikit, e.g. if the person has the right libraries installed for scikits.sparse.cholmod but not scikits.sparse.umfpack. I know that Python building is completely broken in many ways, but there must be some kind of standard solutions (or at least standard hacks!) for these. I did check numpy.distutils, but the docs I can find are all about how to piece together a build for a giant project with many pieces, which is not very relevant, and I couldn't figure out how to use it and Cython together (they both want to replace build_ext). Help! -- Nathaniel

4 3

Is "Creative Commons: Attribution" an acceptable license for datasets included in scipy ?
by David Cournapeau 13 Jan '10

13 Jan '10

Hi, Everything is in the title - I have some new IO code for scipy.sparse I would like to include in scipy, and the tests include some dataset under this license. Should I remove them before inclusion ? cheers, David

5 7

bug in scipy.stats package
by Leon Adams 12 Jan '10

12 Jan '10

Hi all, I am currently doing some work involving distribution fitting and it looks like a came across a strange bug in the ppf function of the gamma distribution. Can some one else verify and if so suggest possible resolution. Below is a sample of my python script: import scipy.stats as st import numpy as np nbins = 12 binProb = np.zeros(nbins) + 1.0/nbins binSumProb = np.add.accumulate(binProb) print binSumProb print st.gamma.ppf(binSumProb,0.6379,loc=1.6,scale=39.555) With the following output: [ 0.08333333 0.16666667 0.25 0.33333333 0.41666667 0.5 0.58333333 0.66666667 0.75 0.83333333 0.91666667 1. ] [ 2.28713122 3.68072087 1.6 8.20183111 11.42062087 15.44977132 20.5371578 27.11627185 36.01838095 49.13633397 72.58240868 Inf] The problem occurs at an input value of 0.25, the ppf returns the location parameter (in this case 1.6) Thanks in advance -- Leon Adams

2 2

Re: [SciPy-dev] Please stress-test SVN version pf matlab reader
by Ariel Rokem 10 Jan '10

10 Jan '10

Hi Matthew, further - if I set both squeeze_me and struct_as_record to True (does that make sense?), I get into some rather nasty situations, where I can't use the variables in the matfile: For example: In [40]: m = sio.loadmat('C-RA_Adapt.mat',squeeze_me=True,struct_as_record=True) In [41]: h = m['history'] In [42]: h1 = h[0] In [43]: q = h1['q'] Yes - I know it's a bit convoluted, but that's how it is. Now - I can't do anything with 'q': In [44]: q[0] --------------------------------------------------------------------------- IndexError Traceback (most recent call last) /Users/arokem/Projects/SchizoSpread/OrientationTuningBehavior/DATA/<ipython console> in <module>() IndexError: 0-d arrays can't be indexed In [45]: q.dtype Out[45]: dtype('object') There's supposed to be an array called 'x' in there: In [46]: q['x'] --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /Users/arokem/Projects/SchizoSpread/OrientationTuningBehavior/DATA/<ipython console> in <module>() ValueError: field named x not found. Am I doing something unreasonable? Cheers, Ariel On Thu, Jan 7, 2010 at 7:26 PM, Ariel Rokem <arokem(a)berkeley.edu> wrote: > Hi Matthew, > > I updated my scipy from the repo and I am still getting the following > behavior (also reported a couple of days ago on scipy-user), when dealing > with the attached file: > > In [6]: m = > sio.loadmat('/Users/arokem/Projects/SchizoSpread/OrientationTuningBehavior/DATA/C-RA_Adapt.mat') > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio.py:99: > FutureWarning: Using struct_as_record default value (False) This will change > to True in future versions > return MatFile5Reader(byte_stream, **kwargs) > > So far, so good - this is to be expected. But, for obvious reasons, I want > to set the kwarg squeeze_me to True: > > In [7]: m = > sio.loadmat('/Users/arokem/Projects/SchizoSpread/OrientationTuningBehavior/DATA/C-RA_Adapt.mat',squeeze_me=True) > --------------------------------------------------------------------------- > TypeError Traceback (most recent call last) > > /Users/arokem/<ipython console> in <module>() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio.pyc > in loadmat(file_name, mdict, appendmat, **kwargs) > 132 ''' > 133 MR = mat_reader_factory(file_name, appendmat, **kwargs) > --> 134 matfile_dict = MR.get_variables() > 135 if mdict is not None: > 136 mdict.update(matfile_dict) > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5.pyc > in get_variables(self, variable_names) > 411 continue > 412 try: > --> 413 res = self.read_var_array(hdr, process) > 414 except MatReadError, err: > 415 warnings.warn( > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5.pyc > in read_var_array(self, header, process) > 380 `process`. > 381 ''' > --> 382 return self._matrix_reader.array_from_header(header, > process) > 383 > 384 def get_variables(self, variable_names=None): > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so > in scipy.io.matlab.mio5_utils.VarReader5.array_from_header > (scipy/io/matlab/mio5_utils.c:4718)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so > in scipy.io.matlab.mio5_utils.VarReader5.array_from_header > (scipy/io/matlab/mio5_utils.c:4367)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so > in scipy.io.matlab.mio5_utils.VarReader5.read_struct > (scipy/io/matlab/mio5_utils.c:6771)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so > in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix > (scipy/io/matlab/mio5_utils.c:3995)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio5_utils.so > in scipy.io.matlab.mio5_utils.VarReader5.array_from_header > (scipy/io/matlab/mio5_utils.c:4610)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio_utils.so > in scipy.io.matlab.mio_utils.process_element > (scipy/io/matlab/mio_utils.c:1272)() > > /Library/Frameworks/Python.framework/Versions/6.0.0/lib/python2.6/site-packages/scipy/io/matlab/mio_utils.so > in scipy.io.matlab.mio_utils.process_element > (scipy/io/matlab/mio_utils.c:1149)() > > TypeError: Cannot convert mat_struct to numpy.ndarray > > > This doesn't happen when I set struct_as_record to True as well, but then I > need to deal with all these record arrays that I get. And yes - it does seem > to work faster, when it works, though I haven't done any thorough testing of > that. > > Any ideas? > > Cheers, > > Ariel > > > > > -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel

3 3

tiny issue in numpy online doc
by Yaroslav Halchenko 07 Jan '10

07 Jan '10

probably this is a wrong list... but since at scipy.org: it seems that merge wasn't successful ;) I don't see those conflict lines in git repo though -- so may be it was built from some local svn checkout or smth like that: http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#mine now it looks like .... <<<<<<< .mineP: +---------------------------------------------------------------------------+ | Expected type | Default | |------------------------------------+--------------------------------------| | bool | False | |------------------------------------+--------------------------------------| | int | -1 | |------------------------------------+--------------------------------------| | float | np.nan | |------------------------------------+--------------------------------------| | complex | np.nan+0j | |------------------------------------+--------------------------------------| | string | '???' | +---------------------------------------------------------------------------+ >>>>>>> .r7751 We can get a finer control on the conversion of missing values with the filling_values optional argument. Like missing_values, this argument accepts different kind of values: .... -- .-. =------------------------------ /v\ ----------------------------= Keep in touch // \\ (yoh(a)|www.)onerussian.com Yaroslav Halchenko /( )\ ICQ#: 60653192 Linux User ^^-^^ [175555]

2 2