Mailman 3 January 2012 - SciPy-Dev

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 99, Issue 31
by stuart 14 Jan '12

14 Jan '12

http://i.imgur.com/bHLym.jpg On Sat, 2012-01-14 at 12:00 -0600, scipy-dev-request(a)scipy.org wrote: > Send SciPy-Dev mailing list submissions to > scipy-dev(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)scipy.org > > You can reach the person managing the list at > scipy-dev-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Re: Summarizing the scikit-signal discussion (Jaidev Deshpande) > 2. Re: Summarizing the scikit-signal discussion (Travis Oliphant) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 14 Jan 2012 01:11:08 +0530 > From: Jaidev Deshpande <deshpande.jaidev(a)gmail.com> > Subject: Re: [SciPy-Dev] Summarizing the scikit-signal discussion > To: SciPy Developers List <scipy-dev(a)scipy.org> > Message-ID: > <CAB=suEnTQwfzHjJxO-nxEnvyP4D9sqU15G25VKGoUOB-98Vufw(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Hi > > We've jumped to a separate scikit project only because it will be > easier to develop and maintain. > > Once it gets off the ground, the community can choose to put the > project wherever they like. > > Thanks, > > Jaidev > > > ------------------------------ > > Message: 2 > Date: Fri, 13 Jan 2012 16:17:05 -0600 > From: Travis Oliphant <travis(a)continuum.io> > Subject: Re: [SciPy-Dev] Summarizing the scikit-signal discussion > To: SciPy Developers List <scipy-dev(a)scipy.org> > Message-ID: <4A1F92B9-B76F-4B5D-ABA5-E4CBE6826DB9(a)continuum.io> > Content-Type: text/plain; charset=us-ascii > > I wish you all well. I think this is unfortunate, however. > > I think it would be better to make a branch of scipy.signal and develop all you want there. There is a need for active development in scipy.signal. > > But, I can understand wanting a faster development cycle and wanting less interference with design decisions based on a larger project goals until you are farther along. > > Ultimately, I'm excited people will be writing more signal processing code for Python. > > -Travis > > > On Jan 13, 2012, at 1:41 PM, Jaidev Deshpande wrote: > > > Hi > > > > We've jumped to a separate scikit project only because it will be > > easier to develop and maintain. > > > > Once it gets off the ground, the community can choose to put the > > project wherever they like. > > > > Thanks, > > > > Jaidev > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > ------------------------------ > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev(a)scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > End of SciPy-Dev Digest, Vol 99, Issue 31 > *****************************************

1 0

Summarizing the scikit-signal discussion
by Jaidev Deshpande 13 Jan '12

13 Jan '12

Hi list, I have attempted to summarize the recent discussion we had about the idea of creating a scikit for signal processing in this blog post: http://brocabrain.blogspot.com/2012/01/scikit-signal-python-for-signal.html The purpose of the post is to highlight the key arguments made in our discussions about scipy.signal - it's limitations and scope - and the additional signal processing abilities that we would like to see in Python. I intend this post to be the flag-off for the scikit-signal development. Please let me know if I've missed anything. Thanks

5 6

NASA code on github
by Charles R Harris 13 Jan '12

13 Jan '12

NASA has put several projects up on github <https://github.com/nasa> that might be of interest to some here. Chuck

1 0

Please switch to arpack-ng
by Sylvestre Ledru 13 Jan '12

13 Jan '12

Hello, As suggested here [1], I am contacting the scipy dev team to suggest to switch to arpack-ng. It is a join project between Scilab, Octave and Debian. We also merged patches from Gentoo. This version is already packaged into Debian, Ubuntu, Fedora, Fink, Macports, etc. It would avoid you to bundle sources here: https://github.com/scipy/scipy/tree/master/scipy/sparse/linalg/eigen/arpack I am available to apply any patches may have wrote. Thanks [1] http://projects.scipy.org/scipy/ticket/1578

2 1

scipy.optimize.cobyla not consistant in Windows
by Casper Skovby 12 Jan '12

12 Jan '12

> > Hello. I have realized that scipy.optimize.cobyla.fmin_cobyla does not always give the same result even though the input is exactly the same when running in Windows. When running in Linux I do not experience this. Below I have added an example. In this case as you can see in the example I am trying to represent a curve with a cubic spline. As input to the spline I have 12 points – two end points and 10 inner points. I am trying to optimize these 10 inner points (both in x and y direction) in order to represent the curve as good as possible with this spline. But as said it does not always ends up with the same result. Sometimes you can run the code several times with the same result but suddenly the result differ. I realized this bug when I used OpenOpt (openopt.org) because it gave similar problems. I have a conversation with the Developer of OpenOpt Dmitrey (see http://forum.openopt.org/viewtopic.php?id=499). When using OpenOpt in Linux some of the solvers do also give inconsistant results. Dmitreys conclusion is: Since scipy_cobyla works different in Windows and Linux, probably something with f2py is wrong (or something in libraries it involves). He suggested me to contact this group. Have you any ideas where the bug can be located? Kind Regards, Casper from matplotlib.pylab import * import numpy as np from numpy import linspace from scipy import interpolate from scipy.optimize import cobyla def residual(p, r, y): tck = interpolate.splrep(np.concatenate(([r[0]], p[0:len(p)/2], [r[-1]])), np.concatenate(([y[0]], p[len(p)/2::], [y[-1]]))) yFit = interpolate.splev(r, tck) resid = sum((y-yFit)**2) return resid def constraint(x): return 1 def FitDistribution(r, y): rP = linspace(r[0], r[-1], 12) yP = np.interp(rP, r, y) p0 = np.concatenate((rP[1:-1], yP[1:-1])) # form box-bound constraints lb <= x <= ub lb = np.concatenate((np.zeros(len(rP)-2), np.ones(len(rP)-2)*(-np.inf))) # lower bound ub = np.concatenate((np.ones(len(rP)-2)*r[-1], np.ones(len(rP)-2)*(np.inf))) # upper bound ftol = 10e-5 f = lambda p: residual(p, r, y) pvec = cobyla.fmin_cobyla(f, p0, constraint, iprint=1, maxfun=100000) #def objective(x): # return x[0]*x[1] # #def constr1(x): # return 1 - (x[0]**2 + x[1]**2) # #def constr2(x): # return x[1] # #x=cobyla.fmin_cobyla(objective, [0.0, 0.1], [constr1, constr2], rhoend=1e-7, iprint=0) tck = interpolate.splrep(np.concatenate(([r[0]], pvec[0:len(pvec)/2], [r[-1]])), np.concatenate(([y[0]], pvec[len(pvec)/2::], [y[-1]]))) yFit = interpolate.splev(r, tck) return yFit r = linspace(1,100) y = r**2 yFit = FitDistribution(r, y) plot(r, y, label = 'func') plot(r, yFit, label = 'fit') legend(loc=0) grid() show()

1 1

Announce: scikit-learn 0.10
by Fabian Pedregosa 11 Jan '12

11 Jan '12

Dear all, I am pleased to announce the availability of scikits.learn 0.10 scikit-learn 0.10 was released on January 2012, four months after the 0.9 release. This release includes the new modules for decision trees, ensemble methods, kernel approximation, multiclass and multilabel algorithms. It also contains new algorithms added to existing modules such as graph lasso, extension of existing algorithms to work with sparse matrices and a handful of performance and documentation improvements. All this and more can be found in the changelog: http://scikit-learn.org/whats_new.html As usual, sources and windows binaries can be found on pypi (http://pypi.python.org/pypi/scikit-learn/0.10) or installed though pip: pip install -U scikit-learn Best regards, Fabian.

1 0

Distance Metrics
by Jacob VanderPlas 10 Jan '12

10 Jan '12

Hello, I've been working on a little project lately centered around distance metrics ( https://github.com/jakevdp/pyDistances ). The idea was to create a set of cython distance metrics that can be called as normal from python with numpy arrays, but which also expose low-level C function pointers so that the same metrics can be called directly on memory buffers from within cythonized tree-based KNN searches (KD Tree, Ball Tree, etc.), without any python overhead. I initially had in mind developing this for scikit-learn in order to extend the capability of Ball Tree, but it occurred to me that this might be nice to have in scipy as well. The speed of computing a distance matrix is comparable to that of pdist/cdist in scipy.spatial.distance (a few metrics are slightly faster, a few are slightly slower). The primary advantage to this approach is the exposure of underlying C functions which can be easily imported and called from other cython scripts. I think there are several other advantages over the current scipy implementation. Because the new code is pure cython, it would likely be easier to maintain and to add metrics than the current scipy setup, which relies on C routines wrapped by-hand using the numpy C-API. Because all distance functions rely on the same set of underlying cython routines, there are fewer places for error (for instance, currently the scipy.spatial.distance boolean routines return different results depending on whether you call the metrics directly or use cdist/pdist) I'm curious what people think: could a framework like this replace the current scipy.spatial.distances implementation? Are there any disadvantages that I'm not noticing? Thanks Jake

1 0

suggestion: transform keyword in RBF ?
by josef.pktd＠gmail.com 10 Jan '12

10 Jan '12

The standard norm in RBF treats all dimensions on the same scale. IIRC, someone (Pauli) mentioned on the mailing list before that the data should be standardized. I was just reading the same recommendation in Elements of Statistical Learning. A keyword that would handle standardization (zscore or orthonormalization) internally would be convenient for users. I think it would be possible to use the norm keyword to define a distance measure that is not symmetric in the dimension (mahalanobis), but that requires thinking. just a thought. Josef

1 0

usage of inspect.getargspec ?
by josef.pktd＠gmail.com 07 Jan '12

07 Jan '12

triggered by the recent commit, I started to look a bit more at inspect.getargspec https://github.com/scipy/scipy/commit/c9c2d66701583984589c88c08c478e2fc6d9f… Until now I have seen it mainly in convenience code were it makes it easier for users but that I usually don't use, in this case it is part of essential code. The main problem with inspect.getargspec is that it gets easily confused by `self` if it is an instance method and by keywords and flexible arguments. recent fix for curve_fit https://github.com/scipy/scipy/pull/92 my only other experience is with numpy.vectorize that has some tricky features and IIRC has some cases of flexible arguments that don't work. I don't think the changes to fmin_ncg break anything in statsmodels since we have identical simple signatures for the main functions, hessians and gradients, but I think it would break if we add a keyword argument to the Hessians. It also would be possible to work around any limitations by writing a wrapper or a lambda function. What experience do others have with inspect? My experience is mostly unpleasant but I never looked much at the details of inspect, just avoided it as much as possible. Is it ok to use it in basic library code like the optimizers? Just asking for general comments and since the pull request is closed. Josef

10 21

pull requests and code review
by josef.pktd＠gmail.com 06 Jan '12

06 Jan '12

Can we keep pull requests open for more than 3 hours, so we actually have time to look at them. looking at https://github.com/scipy/scipy/commit/c9c2d66701583984589c88c08c478e2fc6d9f… my first guess is that the use of inspect.getargspec breaks when the hessian is a method attached to a class, as we have almost all our code in statsmodels. We just fixed a similar case in scipy. There should be at least time to check whether this kind of suspicions are justified or not. Josef

7 17