Mailman 3 April 2012 - SciPy-User

ANN: pandas 0.7.3 release, upcoming time series work
by Wes McKinney April 12, 2012

April 12, 2012

hi all, I'm very pleased to announce the pandas 0.7.3 release. This is primarily a bug fix release from 0.7.2, but also includes a number of new features. The most noticeable ones are a number of new plotting types and options. What's new: http://pandas.pydata.org/pandas-docs/stable/whatsnew.html pandas 0.8.0 will be the largest pandas release in some time. We have been working intensely to expand its time series capabilities and to incorporate code and features from scikits.timeseries, thus enabling users to migrate their code to pandas. From 0.8.0 onward, pandas will use NumPy's datetime64 dtype and thus require at least NumPy version 1.6.1. I am extremely excited about this step forward, as pandas will be one of the most featureful and highest performance tools for time series data in any language. If this is of interest to you, tune in to the timeseries branch on GitHub, which will soon be merged into master. Targeting a mid-May release for 0.8.0, so the more user feedback between now and then the better. - Wes What is it ========== pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with relational, time series, or any other kind of labeled data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Links ===== Release Notes: http://github.com/pydata/pandas/blob/master/RELEASE.rst Documentation: http://pandas.pydata.org Installers: http://pypi.python.org/pypi/pandas Code Repository: http://github.com/pydata/pandas Mailing List: http://groups.google.com/group/pydata Blogs: http://blog.wesmckinney.com and http://blog.lambdafoundry.com

1 0

Re: [SciPy-User] Strange behavior of sph_jn
by ZIEGLER Yann (ETU EOT) April 12, 2012

April 12, 2012

> Thanks for this excellent feedback. You're welcome, thanks to all for your reactivity and comments. Even if I eventually found the solution elsewhere, it helped me to clarify the origin of my problem. > It would be very helpful if you > could file a bug at > > http://projects.scipy.org/scipy > > That way we we don't lose track of the problem you ran in to. > The algorithm should be adjusted to handle these values. > > Best regards, > > -Travis In fact, I'm the author of the last ticket: http://projects.scipy.org/scipy/ticket/1640 But in a previous mail I spoke about this one: http://projects.scipy.org/scipy/ticket/1114 I didn't realize immediately that its author exactly described the problem I have reported here and gave the same trick to solve it :-S ... but 'in my defence' its ticket dates back to 2010! -Yann

1 0

scipy installation error on Mac OS X (10.6.8) (with pip)
by Peng Yu April 11, 2012

April 11, 2012

Hi, I get the following error. When I try to instal scipy (pip install scipy). Does anybody know how to fix the problem? .... error: Command "/opt/local/bin/g95 -shared -shared build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/_fftpackmodule.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/drfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zrfft.o build/temp.macosx-10.6-intel-2.7/scipy/fftpack/src/zfftnd.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/src/dct.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/scipy/fftpack/src/dst.o build/temp.macosx-10.6-intel-2.7/build/src.macosx-10.6-intel-2.7/fortranobject.o -Lbuild/temp.macosx-10.6-intel-2.7 -ldfftpack -lfftpack -o build/lib.macosx-10.6-intel-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1 -- Regards, Peng

2 1

Strange behavior of sph_jn
by ZIEGLER Yann (ETU EOT) April 11, 2012

April 11, 2012

Hi, It seems that the bug I have with sph_jn for some values is due to scipy itself. I finded someone having the same kind of trouble on projects.scipy.org/scipy (the website seems to have some technical problems at this time). If anyone is interested in it, here is the solution I have finded to circumvent this bug : Using the definition of Spherical Bessel function involving Bessel function (http://functions.wolfram.com/Bessel-TypeFunctions/SphericalBesselJ/02/), I wrote -- nothing extraordinary, this is a 2-lines function -- my own SphericalBessel function (naive but seemingly correct) for positive or negative order n and complex argument z : import numpy as np from scipy.special import jn def SphericalBessel(n,z): zsqrt = np.sqrt(np.abs(z)) * np.exp(np.angle(z)/2) return np.sqrt(np.pi/2) / zsqrt * jn(n+0.5, z) Yann

4 4

CSR sorting/duplication of entries
by Chris Laumann April 11, 2012

April 11, 2012

Hi all- Does anybody know whether it is safe (or rather, what breaks) if you use a CSR matrix whose rows are not sorted and may even include duplicate entries? Are duplicates treated additively? The main scipy.sparse documentation page notes that CSR rows are not necessarily sorted but doesn't address duplication. It also simply cryptically notes that you may need to sort the CSR rows when talking to other libraries, but doesn't specifically address whether they are always safe within scipy submodules.. Best, Chris

1 0

(no subject)
by L. Barrott April 11, 2012

April 11, 2012

>On Tue, Apr 3, 2012 at 11:29 AM, L. Barrott <lb489(a)cam.ac.uk> wrote: > >> > On Tue, Apr 3, 2012 at 5:54 AM, L. Barrott <lb489(a)cam.ac.uk> wrote: >> > >> >> Hello, >> >> >> >> I have been trying to get scipy to solve a set of coupled odes and >> >> in particular I want to use the dopri 45 method as I want to compare >> >> the results to the ode45 method in MATLAB. The code runs along the >> >> lines of: >> >> >> >> def func (t, Y, params): >> >> ... >> >> return (Ydot) >> >> >> >> with Y a vector. The other ode methods (except dop853 of course) solve >> >> this fine but even if I use the example code on the documentation page >> >> the dopri method returns the following error >> >> >> >> create_cb_arglist: Failed to build argument list (siz) with enough >> >> arguments (tot-opt) required by user-supplied function >> >> (siz,tot,opt=2,3,0). ...(traceback stuff) _dop.error: failed in >> >> processing argument list for call-back fcn. >> >> >> >> Any ideas where I am going wrong? >> >> >> >> Many thanks >> >> LB >> >> >> > >> > >> > It would help to see more of your code. Could you include a complete, >> > self-contained script that demonstrates the error? >> > >> > Warren >> >> Even something as simple as; >> >> from scipy.integrate import ode >> y0, t0 = [0, 1], 0 >> def func (t, y, x): >> return [x, y[0]] >> r = ode(func).set_integrator ('dopri5') >> r.set_initial_value(y0, t0).set_f_params(1) >> t1 = 10 >> dt = 0.1 >> while r.successful() and r.t < t1: >> r.integrate(r.t+dt) >> >> Will fail and this is lifted straight from the documentation as far as I >> can see. The full error message is >> >> create_cb_arglist: Failed to build argument list (siz) with enough >> arguments (tot-opt) required by user-supplied function >> (siz,tot,opt=2,3,0). Traceback (most recent call last): >> File "<stdin>", line 2, in <module> >> File "/usr/lib/python2.7/dist-packages/scipy/integrate/ode.py", line >> 326, in integrate >> self.f_params,self.jac_params) >> File "/usr/lib/python2.7/dist-packages/scipy/integrate/ode.py", line >> 745, in run >> x,y,iwork,idid = self.runner(*((f,t0,y0,t1) + tuple(self.call_args))) >> _dop.error: failed in processing argument list for call-back fcn. >> >> LB >> > > >I suspect you are using version 0.9 (or earlier) of scipy. This looks like >a bug that was fixed in 0.10 > > http://projects.scipy.org/scipy/ticket/1392 > > >Warren Sorry for being slow about responding, that works now. Many thanks. LB

1 0

scipy.stats convolution of two distributions
by nicky van foreest April 10, 2012

April 10, 2012

Hi, In one of my projects I built some code that depends in a nice and generic way on the methods of rv_continuous in scipy.stats. Now it turns out that I shot myself in the foot because I need to run a test with the sum (convolution) of three such distributions. As far as I can see there is no standard way to achieve this in scipy.stats. Does anybody know of a good (generic) way to do this? If not, would it actually be useful to add such functionality to scipy.stats, if possible at all? After some struggling I wrote the code below, but this is a first attempt. I am very interested in obtaining feedback to turn this into something that is useful for a larger population than just me. Thanks in advance Nicky import numpy as np import scipy.stats from scipy.stats import poisson, uniform, expon import pylab as pl # I need a grid since np.convolve requires two arrays. # choose the grid such that it covers the numerically relevant support # of the distributions grid = np.arange(0., 5., 0.001) # I need P(D1+D2+D3 <= x) D1 = expon(scale = 1./2) D2 = expon(scale = 1./3) D3 = expon(scale = 1./6) class convolved_gen(scipy.stats.rv_continuous): def __init__(self, D1, D2, grid): self.D1 = D1 self.D2 = D2 delta = grid[1]-grid[0] p1 = self.D1.pdf(grid) p2 = self.D2.pdf(grid)*delta self.conv = np.convolve(p1, p2) super(convolved_gen, self).__init__(name = "convolved") def _cdf(self, grid): cdf = np.cumsum(self.conv) return cdf/cdf[-1] # ensure that cdf[-1] = 1 def _pdf(self, grid): return self.conv[:len(grid)] def _stats(self): m = self.D1.stats("m") + self.D2.stats("m") v = self.D1.stats("v") + self.D2.stats("v") return m, v, 0., 0. convolved = convolved_gen(D1, D2, grid) conv = convolved() convolved2 = convolved_gen(conv, D3, grid) conv2 = convolved2() pl.plot(grid,D1.cdf(grid)) pl.plot(grid,conv.cdf(grid)) pl.plot(grid,conv2.cdf(grid)) pl.show()

2 6

[SciPy-user] ValueError: The truth value of an array with more than one element is ambiguous.
by surfcast23 April 8, 2012

April 8, 2012

Hi, I have an if statement and what I want it to do is go through arrays and find the common elements in all three arrays. When I try the code below I get this error ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() Can some one explain the error to me and how I might be able to fix it. Thanks in advance. if min <= Xa <=max & min <= Ya <=max & min <= Za <=max: print("in range") else: print("Not in range") -- View this message in context: http://old.nabble.com/ValueError%3A-The-truth-value-of-an-array-with-more-t… Sent from the Scipy-User mailing list archive at Nabble.com.

4 11

Help with clustering : Memory Error with large dataset
by Abhishek Pratap April 7, 2012

April 7, 2012

Hey Guys I am re-posting a message I had sent to numpy mailing list earlier. In summary I need help with clustering. My input dataset is about 1-2 million x,y coordinates which I would like to cluster together for ex using DBSCAN algo. I tried it on a small data set and it works fine. When I increase my input size it crashes. Can I be more efficient ? More details copied below. Thanks! -Abhi ===message from numpy mailing list==== I am new to both python and more so to numpy. I am trying to cluster close to a 900K points using DBSCAN algo. My input is a list of ~900k tuples each having two points (x,y) coordinates. I am converting them to numpy array and passing them to pdist method of scipy.spatial.distance for calculating distance between each point. Here is some size info on my numpy array shape of input array : (828575, 2) Size : 6872000 bytes I think the error has something to do with the default double dtype of numpy array of pdist function. I would appreciate if you could help me debug this. I am sure I overlooking some naive thing here See the traceback below. MemoryError Traceback (most recent call last) /house/homedirs/a/apratap/Dropbox/dev/ipython/<ipython-input-83-ee29361b7276> in <module>() 36 37 print cleaned_senseBam ---> 38 cluster_pet_points_per_chromosome(sense_bamFile) /house/homedirs/a/apratap/Dropbox/dev/ipython/<ipython-input-83-ee29361b7276> in cluster_pet_points_per_chromosome(bamFile) 30 print 'Size of list points is %d' % sys.getsizeof(points) 31 print 'Size of numpy array is %d' % sys.getsizeof(points_array) ---> 32 cluster_points_DBSCAN(points_array) 33 #print points_array 34 /house/homedirs/a/apratap/Dropbox/dev/ipython/<ipython-input-72-77005d7cd900> in cluster_points_DBSCAN(data_numpy_array) 9 def cluster_points_DBSCAN(data_numpy_array): 10 #eucledian distance calculation ---> 11 D = distance.pdist(data_numpy_array) 12 S = distance.squareform(D) 13 H = 1 - S/np.max(S) /house/homedirs/a/apratap/playground/software/epd-7.2-2-rh5-x86_64/lib/python2.7/site-packages/scipy/spatial/distance.pyc in pdist(X, metric, p, w, V, VI) 1155 1156 m, n = s -> 1157 dm = np.zeros((m * (m - 1) / 2,), dtype=np.double) 1158 1159 wmink_names = ['wminkowski', 'wmi', 'wm', 'wpnorm']

3 2

AUTO: Christopher Peters is out of the office (returning 04/09/2012)
by Christopher Peters April 6, 2012

April 6, 2012

I am out of the office until 04/09/2012. I am out of the office. Please email urgent requests to Mike McDonald. Note: This is an automated response to your message "Re: [SciPy-User] [SciPy-user] ValueError: The truth value of an array with more than one element is ambiguous." sent on 4/6/2012 12:54:40 AM. This is the only notification you will receive while this person is away.

1 0