Mailman 3 September 2012 - SciPy-Dev

Bootstrap confidence limits code
by Constantine Evans Oct. 4, 2014

Oct. 4, 2014

Hello everyone, A few years ago I implemented a scikit for bootstrap confidence limits (https://github.com/cgevans/scikits-bootstrap). I didn’t think much about it after that until recently, when I realized that some people are actually using it, and that there’s apparently been some talk about implementing this functionality in either scipy.stats or statsmodels (I should thank Randal Olson for discussing this and bringing it to my attention). As such I’ve rewritten most of the code, and written up some docstrings. The current code can do confidence intervals with basic percentile interval, bias-corrected accelerated, and approximate bootstrap confidence methods, and can also provide bootstrap and jackknife indexes. Most of it is implemented from the descriptions in Efron and Tibshirani’s Introduction to the Bootstrap, but the ABC code at the moment is a port from the modified-BSD-licensed bootstrap package for R (not the boot package) as I’m not entirely confident in my understanding of the method. And so, I have a few questions for everyone: * Is there any interest in including this sort of code in either scipy.stats or statsmodels? If so, where do people think would be the better place? The code is relatively small; at the moment it is less than 200 lines, with docstrings probably making up 100 of those lines. * Also, if so, what would need to be changed, added, and improved beyond what is mentioned in the Contributing to Scipy part of the reference guide? I’m never a fan of my own code, and imagine quite a bit would need to be fixed; I know tests will need to be added too. In addition, I have a few questions about what would be better practice for the API, and I haven’t really found a guide on best practices for Scipy: * When I started writing the code, I wrote a single function ci for confidence intervals, with a method argument to choose the method. This is easy for users, especially so that they don’t have to look through documentation to realize that BCA is the most generally useful method (at least from everything I’ve read) and that there really isn’t any reason to use many of the simpler methods. However, ABC takes different paramenters, and needs a statistic function that takes weights, which makes this single-function organization trickier. At the moment, I have a separate function for ABC. Would it be better to split up all the methods to their own functions? * ABC requires a statistic function that takes weights. I’ve noticed that things like np.average takes a weights= argument. Would it be better to require input of a stat(data,weights) function, or input of a stat(data,weights=) with weights as a named argument? The latter would be nice in terms of allowing the same function to be used for all methods, but would make it impossible to use a lambda for the function. Is there some other method of doing this entirely? * Are there any missing features that anyone thinks should be added? I apologize if much of this is answered elsewhere, I just haven’t found any of it; I also apologize if this is far too long-winded and confusing! Regards, Constantine Evans

4 3

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Scipy.org is down + question about "topical software"
by Michael Clerx Jan. 2, 2013

Jan. 2, 2013

Dear all, First of all, scipy.org appears to be down and has been for some time today. Yesterday, when it was still up, I registered an account and tried adding a link to the topical software section but got the error "too many links". Does anybody know how/if I can fix this? kind regards, Michael

7 12

Scipy docstrings
by Cera, Tim Oct. 9, 2012

Oct. 9, 2012

A whole bunch of conflicts and merges showed up on the docstring editor at http://docs.scipy.org/scipy/merge/ Can't 'Accept Merges' because all of the merges that I looked at did not follow the docstring standard. There isn't a way to 'Reject Merges', so what I have done in the past is accept, then revert. Could this be handled in some other way? Kindest regards, Tim

4 5

stats.distributions.py documentation
by nicky van foreest Oct. 7, 2012

Oct. 7, 2012

Hi, Below are two proposals to handle the documentation of the scipy distributions. The first is to add a set of examples to each distribution, see the list at the end of the mail as an example. However, I actually wonder whether it wouldn't be better to put this stuff in the stats tutorial. (I recently updated this, but given the list below, it is still not complete.) The list below is a bit long... too long perhaps. I actualy get the feeling that, given the enormous diversity of the distributions, it may not be possible to automatically generate a set of simple examples that work for each and every distributions. Such examples then would involve the usage of x.dist.b, and so on, and this is not particularly revealing to first (and second) time users. A possible resolution is to include just one or two generic examples in the example doc string (e.g., dist.rvs(size = (2,3)) ), and refer to the tutorial for the rest. The tutorial then should show extensive examples for each method of the norm distribution. I assume that then any user of other distributions can figure out how to proceed for his/her own distribution. The second possibility would be to follow Josef's suggestion: --snip snip Splitting up the distributions pdf docs in tutorial into separate pages for individual distributions, make them nicer with code and graphs and link them from the docstring of the distribution. This would keep the docstring itself from blowing up, but we could get the full html reference if we need to. --snip snip This idea offers a lot of opportunities. In a previous mail I mentioned that I don't quite like that the documentation is spread over multiple documents. There are doc strings in distributions.py (leading to a bloated file), and there is continuous.rst. Part of the implementation can be understood from the doc-string, typically, the density function, but not the rest; this requires continuous.rst. Besides this, in case some specific distribution requires extra explanation/examples, this will have to put in the doc-string, making distributions.py longer still. Thus, to take up Josef's suggestion, what about a documentation file organised like this: # some tag to tell that these are the docs for the norm distribution # eg. # norm_gen Normal Distribution ---------------------------- Notes ^^^^^^^ # should be used by the interpreter The probability density function for `norm` is:: norm.pdf(x) = exp(-x**2/2)/sqrt(2*pi) Simple Examples ^^^^^^^^^^^^^^^^^^^^ # used for by interpreter >>> norm.rvs( size = (2,3) ) Extensive Examples ^^^^^^^^^^^^^^^^^^^^^^^^ # Not used by the interpreter, but certainly by a html viewer, containing graphs, hard/specific examples. Mathematical Details ^^^^^^^^^^^^^^^^^^^^^^ Stuff from continuous.rst # dist2_gen Distribution number 2 ----------------------------------------- etc It shouldn't be too hard to parse such a document, and couple each piece of documentation to a distribution in distributions.py (or am I mistaken?) as we use the class name as the tag in the documentation file. The doc-string for a distribution in distributions.py can then be removed, Nicky Example for the examples section of the docstring of norm. Notes ----- The probability density function for `norm` is:: norm.pdf(x) = exp(-x**2/2)/sqrt(2*pi) #%(example)s Examples -------- Setting the mean and standard deviation: >>> from scipy.stats import norm >>> norm.cdf(0.0) >>> norm.cdf(0., 1) # set mu = loc = 1 >>> norm.cdf(0., 1, 2) # mu = loc = 1, scale = sigma = 2 >>> norm.cdf(0., loc = 1, scale = 2) # mu = loc = 1, scale = sigma = 2 Frozen rvs >>> norm(1., 2.).cdf(0) >>> x = norm(scale = 2.) >>> x.cdf(0.0) Moments >>> norm(loc = 2).stats() >>> norm.mean() >>> norm.moment(2, scale = 3.) >>> x.std() >>> x.var() Random number generation >>> norm.rvs(3, 1, size = (2,3)) # loc = 3, scale =1, array of shape (2,3) >>> norm.rvs(3, 1, size = [2,3]) >>> x.rvs(3) # array with 3 random deviates >>> x.rvs([3,4]) # array of shape (3,4) with deviates Expectations >>> norm.expect(lambda x: x, loc = 1) # 1.00000 >>> norm.expect(lambda x: x**2, loc = 1., scale = 2.) # second moment Support of the distribution >>> norm.a # left limit, -np.inf here >>> norm.b # right limit, np.inf here Plot of the cdf >>> import numpy as np >>> x = np.linspace(0, 3) >>> P = norm.cdf(x) >>> plt.plot(x,P) >>> plt.show()

2 2

Make orthogonal eval_* ufuncs?
by Eric Moore Oct. 7, 2012

Oct. 7, 2012

Hi list, There are some issues with the eval_* routines for orthogonal polynomials. 1. They appear to support out parameters, but they don't. 2. They choke when handed lists rather than arrays. (This is ticket #1435) 3. special.binom is exported but doesn't appear in the docs on scipy.org. I started to fix these issues this morning by turning them into ufuncs and calling hyp2f1, Gamma and lgam from cephes.h and hyp1f1_wrap from specfun_wrappers.h directly. However the various error handling routines defined in _cephesmodule.c (line 1185+) are actually called in cephes/mtherr.c. This means that I can't actually do it that way, unless I move some things around. I'd propose moving all of the eval_* functions defined in orthogonal_eval.pyx to _cephesmodule.c and turning them all into ufuncs. Thoughts? Eric #1435: Problems when calling special.legendre function over a list for high order http://projects.scipy.org/scipy/ticket/1435

2 1

Docstrings for the optimize functions
by The Helmbolds Oct. 1, 2012

Oct. 1, 2012

Problem: As we update the scipy.optimize docstrings, how should we deal with the difference between the "new style" or 'minimize' calling sequence and its return dictionary, and the "old style" or 'pre-minimize' calling sequence and return values? Background and Discussion: The docstring for the over-arching 'minimize' function is OK as far as it goes. However, while it mentions that the different methods differ in the number and/or kind of parameters in both their calling sequences and in their 'Results' dictionaries, it (quite properly) does not go into the nitty-gritty details of these items for each 'method'. Yet, those details should be described someplace, especially for new users. Suggested Approach: I propose the following approach to documenting the individual 'methods' while we are in this "transition" period: 1. In the Summary (or Extended Summary) of the method, include words to the effect that: Although this documentation for the most part describes the "old style" calling sequence and return values, it is strongly recommended that new code invoking this method use the "new style" or 'minimize' calling sequence and return values, and that serious consideration be given to updating existing code to the "new style". To ease the transition, one or more examples illustrating both the "old style" and the "new style" calling sequences and return values are provided in the Examples section. 2. Leave the docstring's sections on Parameters, Returns, Other Parameters, Raises, Notes, and References, discuss as though the "old style" calling sequence is being used. Add to these sections remarks on the "new style" only when clarity requires it. For example, it may be helpful to add a sentence to the Parameters and Returns sections stting that: The Examples section shows how to adapt this to the "new style" calling sequence and return values. 3. In the examples, first illustrate how to use the "old style" version. Then add an example illustrating how exactly the same problem would be handled when the "new style's" 'minimize' calling sequence is used. In that example discuss just so much of the details of the method's "options dictionary", "constraints dictionary sequence", and "Results dictionary" as seems necessary to guide a new user. In other words, adhere to the priniciple that "All documentation should be as brief as possible -- but no briefer !!!" Bob H

2 1

Scipy.spatial.Delaunay fail to produce the tetrahedron for all particles
by Ilaudy Sept. 30, 2012

Sept. 30, 2012

Dear developer, I am using Scipy.spatial.Delaunay to do some analysis. I found that the Delaunay model can not produce the tetrahedron for my simulated 3D particles which is at the center of a halo. Lots of it are not included in the tri.vertices. For example, I select about ~2300 particles in the center, and do the triangulation. But the tri.vertices only have a shape of (484, 4)! I think this should be a problem of the Hull's accuracy setting. Can I know where I can tune this? Or any suggestions are welcome. I have report this in here, http://projects.scipy.org/scipy/ticket/1728. You can find the 3D particles' position in the attached .txt file Thanks. ilaudy

2 1

ANN: SciPy 0.11.0 release
by Ralf Gommers Sept. 26, 2012

Sept. 26, 2012

Hi, I am pleased to announce the availability of SciPy 0.11.0. For this release many new features have been added, and over 120 tickets and pull requests have been closed. Also noteworthy is that the number of contributors for this release has risen to over 50. We hope to see this number continuing to increase! The highlights of this release are: - A new module, sparse.csgraph, has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. Sources and binaries can be found at http://sourceforge.net/projects/scipy/files/scipy/0.11.0/, release notes are copied below. Thanks to everyone who contributed to this release, Ralf ========================== SciPy 0.11.0 Release Notes ========================== .. contents:: SciPy 0.11.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. Highlights of this release are: - A new module has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Our development attention will now shift to bug-fix releases on the 0.11.x branch, and on adding new features on the master branch. This release requires Python 2.4-2.7 or 3.1-3.2 and NumPy 1.5.1 or greater. New features ============ Sparse Graph Submodule ---------------------- The new submodule :mod:`scipy.sparse.csgraph` implements a number of efficient graph algorithms for graphs stored as sparse adjacency matrices. Available routines are: - :func:`connected_components` - determine connected components of a graph - :func:`laplacian` - compute the laplacian of a graph - :func:`shortest_path` - compute the shortest path between points on a positive graph - :func:`dijkstra` - use Dijkstra's algorithm for shortest path - :func:`floyd_warshall` - use the Floyd-Warshall algorithm for shortest path - :func:`breadth_first_order` - compute a breadth-first order of nodes - :func:`depth_first_order` - compute a depth-first order of nodes - :func:`breadth_first_tree` - construct the breadth-first tree from a given node - :func:`depth_first_tree` - construct a depth-first tree from a given node - :func:`minimum_spanning_tree` - construct the minimum spanning tree of a graph ``scipy.optimize`` improvements ------------------------------- The optimize module has received a lot of attention this release. In addition to added tests, documentation improvements, bug fixes and code clean-up, the following improvements were made: - A unified interface to minimizers of univariate and multivariate functions has been added. - A unified interface to root finding algorithms for multivariate functions has been added. - The L-BFGS-B algorithm has been updated to version 3.0. Unified interfaces to minimizers ```````````````````````````````` Two new functions ``scipy.optimize.minimize`` and ``scipy.optimize.minimize_scalar`` were added to provide a common interface to minimizers of multivariate and univariate functions respectively. For multivariate functions, ``scipy.optimize.minimize`` provides an interface to methods for unconstrained optimization (`fmin`, `fmin_powell`, `fmin_cg`, `fmin_ncg`, `fmin_bfgs` and `anneal`) or constrained optimization (`fmin_l_bfgs_b`, `fmin_tnc`, `fmin_cobyla` and `fmin_slsqp`). For univariate functions, ``scipy.optimize.minimize_scalar`` provides an interface to methods for unconstrained and bounded optimization (`brent`, `golden`, `fminbound`). This allows for easier comparing and switching between solvers. Unified interface to root finding algorithms ```````````````````````````````````````````` The new function ``scipy.optimize.root`` provides a common interface to root finding algorithms for multivariate functions, embeding `fsolve`, `leastsq` and `nonlin` solvers. ``scipy.linalg`` improvements ----------------------------- New matrix equation solvers ``````````````````````````` Solvers for the Sylvester equation (``scipy.linalg.solve_sylvester``, discrete and continuous Lyapunov equations (``scipy.linalg.solve_lyapunov``, ``scipy.linalg.solve_discrete_lyapunov``) and discrete and continuous algebraic Riccati equations (``scipy.linalg.solve_continuous_are``, ``scipy.linalg.solve_discrete_are``) have been added to ``scipy.linalg``. These solvers are often used in the field of linear control theory. QZ and QR Decomposition ```````````````````````` It is now possible to calculate the QZ, or Generalized Schur, decomposition using ``scipy.linalg.qz``. This function wraps the LAPACK routines sgges, dgges, cgges, and zgges. The function ``scipy.linalg.qr_multiply``, which allows efficient computation of the matrix product of Q (from a QR decompostion) and a vector, has been added. Pascal matrices ``````````````` A function for creating Pascal matrices, ``scipy.linalg.pascal``, was added. Sparse matrix construction and operations ----------------------------------------- Two new functions, ``scipy.sparse.diags`` and ``scipy.sparse.block_diag``, were added to easily construct diagonal and block-diagonal sparse matrices respectively. ``scipy.sparse.csc_matrix`` and ``csr_matrix`` now support the operations ``sin``, ``tan``, ``arcsin``, ``arctan``, ``sinh``, ``tanh``, ``arcsinh``, ``arctanh``, ``rint``, ``sign``, ``expm1``, ``log1p``, ``deg2rad``, ``rad2deg``, ``floor``, ``ceil`` and ``trunc``. Previously, these operations had to be performed by operating on the matrices' ``data`` attribute. LSMR iterative solver --------------------- LSMR, an iterative method for solving (sparse) linear and linear least-squares systems, was added as ``scipy.sparse.linalg.lsmr``. Discrete Sine Transform ----------------------- Bindings for the discrete sine transform functions have been added to ``scipy.fftpack``. ``scipy.interpolate`` improvements ---------------------------------- For interpolation in spherical coordinates, the three classes ``scipy.interpolate.SmoothSphereBivariateSpline``, ``scipy.interpolate.LSQSphereBivariateSpline``, and ``scipy.interpolate.RectSphereBivariateSpline`` have been added. Binned statistics (``scipy.stats``) ----------------------------------- The stats module has gained functions to do binned statistics, which are a generalization of histograms, in 1-D, 2-D and multiple dimensions: ``scipy.stats.binned_statistic``, ``scipy.stats.binned_statistic_2d`` and ``scipy.stats.binned_statistic_dd``. Deprecated features =================== ``scipy.sparse.cs_graph_components`` has been made a part of the sparse graph submodule, and renamed to ``scipy.sparse.csgraph.connected_components``. Calling the former routine will result in a deprecation warning. ``scipy.misc.radon`` has been deprecated. A more full-featured radon transform can be found in scikits-image. ``scipy.io.save_as_module`` has been deprecated. A better way to save multiple Numpy arrays is the ``numpy.savez`` function. The `xa` and `xb` parameters for all distributions in ``scipy.stats.distributions`` already weren't used; they have now been deprecated. Backwards incompatible changes ============================== Removal of ``scipy.maxentropy`` ------------------------------- The ``scipy.maxentropy`` module, which was deprecated in the 0.10.0 release, has been removed. Logistic regression in scikits.learn is a good and modern alternative for this functionality. Minor change in behavior of ``splev`` ------------------------------------- The spline evaluation function now behaves similarly to ``interp1d`` for size-1 arrays. Previous behavior:: >>> from scipy.interpolate import splev, splrep, interp1d >>> x = [1,2,3,4,5] >>> y = [4,5,6,7,8] >>> tck = splrep(x, y) >>> splev([1], tck) 4. >>> splev(1, tck) 4. Corrected behavior:: >>> splev([1], tck) array([ 4.]) >>> splev(1, tck) array(4.) This affects also the ``UnivariateSpline`` classes. Behavior of ``scipy.integrate.complex_ode`` ------------------------------------------- The behavior of the ``y`` attribute of ``complex_ode`` is changed. Previously, it expressed the complex-valued solution in the form:: z = ode.y[::2] + 1j * ode.y[1::2] Now, it is directly the complex-valued solution:: z = ode.y Minor change in behavior of T-tests ----------------------------------- The T-tests ``scipy.stats.ttest_ind``, ``scipy.stats.ttest_rel`` and ``scipy.stats.ttest_1samp`` have been changed so that 0 / 0 now returns NaN instead of 1. Other changes ============= The SuperLU sources in ``scipy.sparse.linalg`` have been updated to version 4.3 from upstream. The function ``scipy.signal.bode``, which calculates magnitude and phase data for a continuous-time system, has been added. The two-sample T-test ``scipy.stats.ttest_ind`` gained an option to compare samples with unequal variances, i.e. Welch's T-test. ``scipy.misc.logsumexp`` now takes an optional ``axis`` keyword argument. Authors ======= This release contains work by the following people (contributed at least one patch to this release, names in alphabetical order): * Jeff Armstrong * Chad Baker * Brandon Beacher + * behrisch + * borishim + * Matthew Brett * Lars Buitinck * Luis Pedro Coelho + * Johann Cohen-Tanugi * David Cournapeau * dougal + * Ali Ebrahim + * endolith + * Bjørn Forsman + * Robert Gantner + * Sebastian Gassner + * Christoph Gohlke * Ralf Gommers * Yaroslav Halchenko * Charles Harris * Jonathan Helmus + * Andreas Hilboll + * Marc Honnorat + * Jonathan Hunt + * Maxim Ivanov + * Thouis (Ray) Jones * Christopher Kuster + * Josh Lawrence + * Denis Laxalde + * Travis Oliphant * Joonas Paalasmaa + * Fabian Pedregosa * Josef Perktold * Gavin Price + * Jim Radford + * Andrew Schein + * Skipper Seabold * Jacob Silterra + * Scott Sinclair * Alexis Tabary + * Martin Teichmann * Matt Terry + * Nicky van Foreest + * Jacob Vanderplas * Patrick Varilly + * Pauli Virtanen * Nils Wagner + * Darryl Wally + * Stefan van der Walt * Liming Wang + * David Warde-Farley + * Warren Weckesser * Sebastian Werk + * Mike Wimmer + * Tony S Yu + A total of 55 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

1 0

Fwd: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion
by Orion Poplawski Sept. 24, 2012

Sept. 24, 2012

This is a plea for some help. We've been having trouble getting scipy to pass all of the tests in the Fedora 18 build with python 3.3 (although it seems to build okay in Fedora 19). Below are the logs of the build. There appears to be some kind of memory corruption that manifests itself a little differently on 32-bit vs. 64-bit. I really have no idea myself how to pursue debugging this, though I'm happy to provide any more needed information. Thanks, Orion -------- Original Message -------- Subject: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion Date: Thu, 20 Sep 2012 21:01:54 +0000 (UTC) From: Fedora Koji Build System <buildsys(a)fedoraproject.org> To: ausil(a)fedoraproject.org, jspaleta(a)fedoraproject.org, voronov(a)fedoraproject.org, torwangjl(a)fedoraproject.org, alagunambi(a)fedoraproject.org, urkle(a)fedoraproject.org, orion(a)fedoraproject.org Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion ID: 350761 Started: Thu, 20 Sep 2012 20:39:45 UTC Finished: Thu, 20 Sep 2012 21:01:32 UTC scipy-0.11.0-0.1.rc2.fc18 (350761) failed on buildvm-33.phx2.fedoraproject.org (x86_64), buildvm-35.phx2.fedoraproject.org (i386), buildvm-34.phx2.fedoraproject.org (noarch): BuildError: error building package (arch x86_64), mock exited with status 1; see build.log for more information SRPMS: scipy-0.11.0-0.1.rc2.fc18.src.rpm Failed tasks: ------------- Task 4509076 on buildvm-33.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, x86_64) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=state.log Task 4509077 on buildvm-35.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, i686) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=state.log Task 4509063 on buildvm-34.phx2.fedoraproject.org Task Type: build (f18-candidate, /scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) Closed tasks: ------------- Task 4509064 on buildvm-30.phx2.fedoraproject.org Task Type: buildSRPMFromSCM (/scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=checkout.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=mock_output.… http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=state.log Task Info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4509063 Build Info: http://koji.fedoraproject.org/koji/buildinfo?buildID=350761

2 2

tomorrow 0.11.0 final
by Ralf Gommers Sept. 23, 2012

Sept. 23, 2012

Hi all, The 0.11.0 release cycle has taken quite a bit longer than previous cycles unfortunately, but I think now the 0.11.x branch is ready. Final commit marking the test TestEig.test_singular that was failing on Windows for some people as knownfail in 0.11.x is at https://github.com/scipy/scipy/pull/323. The sort keyword to linalg.qz, which has been giving problems on several platforms, has been disabled in both master and 0.11.x. I consider the issues with arpack and single precision linalg routines on OS X 10.8 out of scope for 0.11.0. It would be great if someone who's on that OS could investigate, but it shouldn't block this release. The few changes since RC2 don't require a new RC imo, so I plan to tag the release tomorrow. If there's any commits I missed that absolutely need to be backported, please let me know. Cheers, Ralf

1 0

TestNorm.test_stable failure [was ANN: SciPy 0.11.0 release candidate 2]
by Ralf Gommers Sept. 22, 2012

Sept. 22, 2012

On Sat, Sep 8, 2012 at 9:38 PM, <josef.pktd(a)gmail.com> wrote: > > ------------------------- > If I just unzip the superpack instead of installing the sse3, I get > only one failure > > ====================================================================== > FAIL: test_basic.TestNorm.test_stable > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "E:\Josef\testing\tox\py27b\lib\site-packages\nose-1.1.2-py2.7.egg\nose\case.py", > line 197, in runTest > self.test(*self.arg) > File > "E:\Josef\testing\tox\py27b\lib\site-packages\scipy\linalg\tests\test_basic.py", > line 592, in > test_stable > assert_almost_equal(norm(a) - 1e4, 0.0, err_msg=msg) > File > "E:\Josef\testing\tox\py27b\lib\site-packages\numpy-1.6.2-py2.7-win32.egg\numpy\testing\utils.py", > line 468, in assert_almost_equal > raise AssertionError(msg) > AssertionError: > Arrays are not almost equal to 7 decimals > : Result should equal either 0.0 or 0.5 (depending on implementation of > snrm2). > ACTUAL: 0.4990234375 > DESIRED: 0.0 > > maybe some sse incompatibilities. > This one keeps on coming back. We fixed the test for allowing 0.0 or 0.5, but it can also just break like this. It does so at least for some people on OS X 10.8 and on Windows. Since the problem seems to be in ATLAS / Accelerate, should we leave it as is or not? Ralf

2 4

Re: [SciPy-Dev] [SciPy-User] ANN: SciPy 0.11.0 release candidate 2
by Ralf Gommers Sept. 22, 2012

Sept. 22, 2012

On Tue, Aug 14, 2012 at 3:35 PM, John Hassler <hasslerjc(a)comcast.net> wrote: > > On 8/14/2012 7:21 AM, Pauli Virtanen wrote: > > Ralf Gommers <ralf.gommers <at> gmail.com> writes: > > [clip] > >> Does anyone have an idea about that test_singular failure? > > That's very likely some problem with the underlying LAPACK library. > > I think the problem solved is close to a numerical instability. > > > > The failing comparison compares eigenvalues computed by > > > > eig(A, B) > > eig(A, B, left=False, right=False) > > > > which differ solely in passing 'N' vs. 'V' to DGGEV. The eigenvalue > > property of the former is also checked and seems to pass. Interestingly, > > the result obtained from the two seems to differ (therefore, the latter > > is probably wrong), which appears to point to a LAPACK issue. > > > > Here, it would be interesting to know if the problem occurs with > > the official Scipy binaries, or something else. > > > > I installed rc2 on Python 2.7.3. Same problem. I get the test_singular > error on some, but not all, of the runs. Both are win32-superpack from > http://sourceforge.net/projects/scipy/files/scipy/0.11.0rc2/. > > The error occurs on less than half but more than 1/3 (based on a very > small sample) of the runs on both 2.7 and 3.2. > > I've been working on computers for more than 50 years. Somehow, I had > developed the delusion that they were deterministic ..... > john > What are we going to do about this one? I'm tempted to open a ticket for it and mark it as knownfail on Windows for now, since it's a corner case. Ralf > ------------- Python 2.7 -------------------- > >>> import scipy > >>> scipy.test() > Running unit tests for scipy > NumPy version 1.6.2 > NumPy is installed in C:\Python27\lib\site-packages\numpy > SciPy version 0.11.0rc2 > SciPy is installed in C:\Python27\lib\site-packages\scipy > Python version 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit > (Intel)] > nose version 0.11.2 > > ====================================================================== > FAIL: test_decomp.TestEig.test_singular > Test singular pair > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "C:\Python27\lib\site-packages\nose-0.11.2-py2.7.egg\nose\case.py", line > 186, in runTest > self.test(*self.arg) > File > "C:\Python27\lib\site-packages\scipy\linalg\tests\test_decomp.py", line > 201, in test_singular > self._check_gen_eig(A, B) > File > "C:\Python27\lib\site-packages\scipy\linalg\tests\test_decomp.py", line > 188, in _check_gen_eig > err_msg=msg) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line > 800, in assert_array_almost_equal > header=('Arrays are not almost equal to %d decimals' % decimal)) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line > 636, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Arrays are not almost equal to 6 decimals > > array([[22, 34, 31, 31, 17], > [45, 45, 42, 19, 29], > [39, 47, 49, 26, 34], > [27, 31, 26, 21, 15], > [38, 44, 44, 24, 30]]) > array([[13, 26, 25, 17, 24], > [31, 46, 40, 26, 37], > [26, 40, 19, 25, 25], > [16, 25, 27, 14, 23], > [24, 35, 18, 21, 22]]) > (mismatch 25.0%) > x: array([ -2.45037885e-01 +0.00000000e+00j, > 5.17637463e-16 -4.01120590e-08j, > 5.17637463e-16 +4.01120590e-08j, 2.00000000e+00 > +0.00000000e+00j]) > y: array([ -3.74550285e-01 +0.00000000e+00j, > -5.17716907e-17 -1.15230800e-08j, > -5.17716907e-17 +1.15230800e-08j, 2.00000000e+00 > +0.00000000e+00j]) > > ---------------------------------------------------------------------- > Ran 5490 tests in 103.250s > > FAILED (KNOWNFAIL=14, SKIP=36, failures=1) > <nose.result.TextTestResult run=5490 errors=0 failures=1> > >>> > > -------------- Python 3.2 -------------- > >>> scipy.test() > Running unit tests for scipy > NumPy version 1.6.2 > NumPy is installed in C:\Python32\lib\site-packages\numpy > SciPy version 0.11.0rc2 > SciPy is installed in C:\Python32\lib\site-packages\scipy > Python version 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit > (Intel)] > nose version 1.0.0 > > ====================================================================== > FAIL: test_decomp.TestEig.test_singular > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "C:\Python32\lib\site-packages\nose-1.0.0-py3.2.egg\nose\case.py", line > 188, in runTest > self.test(*self.arg) > File > "C:\Python32\lib\site-packages\scipy\linalg\tests\test_decomp.py", line > 201, in test_singular > self._check_gen_eig(A, B) > File > "C:\Python32\lib\site-packages\scipy\linalg\tests\test_decomp.py", line > 188, in _check_gen_eig > err_msg=msg) > File "C:\Python32\lib\site-packages\numpy\testing\utils.py", line > 800, in assert_array_almost_equal > header=('Arrays are not almost equal to %d decimals' % decimal)) > File "C:\Python32\lib\site-packages\numpy\testing\utils.py", line > 636, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Arrays are not almost equal to 6 decimals > > array([[22, 34, 31, 31, 17], > [45, 45, 42, 19, 29], > [39, 47, 49, 26, 34], > [27, 31, 26, 21, 15], > [38, 44, 44, 24, 30]]) > array([[13, 26, 25, 17, 24], > [31, 46, 40, 26, 37], > [26, 40, 19, 25, 25], > [16, 25, 27, 14, 23], > [24, 35, 18, 21, 22]]) > (mismatch 25.0%) > x: array([ -2.450e-01 +0.000e+00j, 5.176e-16 -4.011e-08j, 5.176e-16 > +4.011e-08j, > 2.000e+00 +0.000e+00j]) > y: array([ -3.746e-01 +0.000e+00j, -5.177e-17 -1.152e-08j, -5.177e-17 > +1.152e-08j, > 2.000e+00 +0.000e+00j]) > >

3 4

ANN: SciPy 0.11.0 release candidate 2
by Ralf Gommers Sept. 22, 2012

Sept. 22, 2012

Hi, I am pleased to announce the availability of the second release candidate of SciPy 0.11.0. For this release many new features have been added, and over 120 tickets and pull requests have been closed. Also noteworthy is that the number of contributors for this release has risen to over 50. Some of the highlights are: - A new module, sparse.csgraph, has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. Sources and binaries can be found at http://sourceforge.net/projects/scipy/files/scipy/0.11.0rc2/, release notes are copied below. For this release candidate all known issues (with the exception of one Qhull issue on Debian, s390x platform) have been solved. In the meantime also OS X 10.8 was released, this RC contains a few build fixes for that platform. If no more serious issues are reported, the final release will be in one week. Cheers, Ralf ========================== SciPy 0.11.0 Release Notes ========================== .. note:: Scipy 0.11.0 is not released yet! .. contents:: SciPy 0.11.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. Highlights of this release are: - A new module has been added which provides a number of common sparse graph algorithms. - New unified interfaces to the existing optimization and root finding functions have been added. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Our development attention will now shift to bug-fix releases on the 0.11.x branch, and on adding new features on the master branch. This release requires Python 2.4-2.7 or 3.1-3.2 and NumPy 1.5.1 or greater. New features ============ Sparse Graph Submodule ---------------------- The new submodule :mod:`scipy.sparse.csgraph` implements a number of efficient graph algorithms for graphs stored as sparse adjacency matrices. Available routines are: - :func:`connected_components` - determine connected components of a graph - :func:`laplacian` - compute the laplacian of a graph - :func:`shortest_path` - compute the shortest path between points on a positive graph - :func:`dijkstra` - use Dijkstra's algorithm for shortest path - :func:`floyd_warshall` - use the Floyd-Warshall algorithm for shortest path - :func:`breadth_first_order` - compute a breadth-first order of nodes - :func:`depth_first_order` - compute a depth-first order of nodes - :func:`breadth_first_tree` - construct the breadth-first tree from a given node - :func:`depth_first_tree` - construct a depth-first tree from a given node - :func:`minimum_spanning_tree` - construct the minimum spanning tree of a graph ``scipy.optimize`` improvements ------------------------------- The optimize module has received a lot of attention this release. In addition to added tests, documentation improvements, bug fixes and code clean-up, the following improvements were made: - A unified interface to minimizers of univariate and multivariate functions has been added. - A unified interface to root finding algorithms for multivariate functions has been added. - The L-BFGS-B algorithm has been updated to version 3.0. Unified interfaces to minimizers ```````````````````````````````` Two new functions ``scipy.optimize.minimize`` and ``scipy.optimize.minimize_scalar`` were added to provide a common interface to minimizers of multivariate and univariate functions respectively. For multivariate functions, ``scipy.optimize.minimize`` provides an interface to methods for unconstrained optimization (`fmin`, `fmin_powell`, `fmin_cg`, `fmin_ncg`, `fmin_bfgs` and `anneal`) or constrained optimization (`fmin_l_bfgs_b`, `fmin_tnc`, `fmin_cobyla` and `fmin_slsqp`). For univariate functions, ``scipy.optimize.minimize_scalar`` provides an interface to methods for unconstrained and bounded optimization (`brent`, `golden`, `fminbound`). This allows for easier comparing and switching between solvers. Unified interface to root finding algorithms ```````````````````````````````````````````` The new function ``scipy.optimize.root`` provides a common interface to root finding algorithms for multivariate functions, embeding `fsolve`, `leastsq` and `nonlin` solvers. ``scipy.linalg`` improvements ----------------------------- New matrix equation solvers ``````````````````````````` Solvers for the Sylvester equation (``scipy.linalg.solve_sylvester``, discrete and continuous Lyapunov equations (``scipy.linalg.solve_lyapunov``, ``scipy.linalg.solve_discrete_lyapunov``) and discrete and continuous algebraic Riccati equations (``scipy.linalg.solve_continuous_are``, ``scipy.linalg.solve_discrete_are``) have been added to ``scipy.linalg``. These solvers are often used in the field of linear control theory. QZ and QR Decomposition ```````````````````````` It is now possible to calculate the QZ, or Generalized Schur, decomposition using ``scipy.linalg.qz``. This function wraps the LAPACK routines sgges, dgges, cgges, and zgges. The function ``scipy.linalg.qr_multiply``, which allows efficient computation of the matrix product of Q (from a QR decompostion) and a vector, has been added. Pascal matrices ``````````````` A function for creating Pascal matrices, ``scipy.linalg.pascal``, was added. Sparse matrix construction and operations ----------------------------------------- Two new functions, ``scipy.sparse.diags`` and ``scipy.sparse.block_diag``, were added to easily construct diagonal and block-diagonal sparse matrices respectively. ``scipy.sparse.csc_matrix`` and ``csr_matrix`` now support the operations ``sin``, ``tan``, ``arcsin``, ``arctan``, ``sinh``, ``tanh``, ``arcsinh``, ``arctanh``, ``rint``, ``sign``, ``expm1``, ``log1p``, ``deg2rad``, ``rad2deg``, ``floor``, ``ceil`` and ``trunc``. Previously, these operations had to be performed by operating on the matrices' ``data`` attribute. LSMR iterative solver --------------------- LSMR, an iterative method for solving (sparse) linear and linear least-squares systems, was added as ``scipy.sparse.linalg.lsmr``. Discrete Sine Transform ----------------------- Bindings for the discrete sine transform functions have been added to ``scipy.fftpack``. ``scipy.interpolate`` improvements ---------------------------------- For interpolation in spherical coordinates, the three classes ``scipy.interpolate.SmoothSphereBivariateSpline``, ``scipy.interpolate.LSQSphereBivariateSpline``, and ``scipy.interpolate.RectSphereBivariateSpline`` have been added. Binned statistics (``scipy.stats``) ----------------------------------- The stats module has gained functions to do binned statistics, which are a generalization of histograms, in 1-D, 2-D and multiple dimensions: ``scipy.stats.binned_statistic``, ``scipy.stats.binned_statistic_2d`` and ``scipy.stats.binned_statistic_dd``. Deprecated features =================== ``scipy.sparse.cs_graph_components`` has been made a part of the sparse graph submodule, and renamed to ``scipy.sparse.csgraph.connected_components``. Calling the former routine will result in a deprecation warning. ``scipy.misc.radon`` has been deprecated. A more full-featured radon transform can be found in scikits-image. ``scipy.io.save_as_module`` has been deprecated. A better way to save multiple Numpy arrays is the ``numpy.savez`` function. The `xa` and `xb` parameters for all distributions in ``scipy.stats.distributions`` already weren't used; they have now been deprecated. Backwards incompatible changes ============================== Removal of ``scipy.maxentropy`` ------------------------------- The ``scipy.maxentropy`` module, which was deprecated in the 0.10.0 release, has been removed. Logistic regression in scikits.learn is a good and modern alternative for this functionality. Minor change in behavior of ``splev`` ------------------------------------- The spline evaluation function now behaves similarly to ``interp1d`` for size-1 arrays. Previous behavior:: >>> from scipy.interpolate import splev, splrep, interp1d >>> x = [1,2,3,4,5] >>> y = [4,5,6,7,8] >>> tck = splrep(x, y) >>> splev([1], tck) 4. >>> splev(1, tck) 4. Corrected behavior:: >>> splev([1], tck) array([ 4.]) >>> splev(1, tck) array(4.) This affects also the ``UnivariateSpline`` classes. Behavior of ``scipy.integrate.complex_ode`` ------------------------------------------- The behavior of the ``y`` attribute of ``complex_ode`` is changed. Previously, it expressed the complex-valued solution in the form:: z = ode.y[::2] + 1j * ode.y[1::2] Now, it is directly the complex-valued solution:: z = ode.y Minor change in behavior of T-tests ----------------------------------- The T-tests ``scipy.stats.ttest_ind``, ``scipy.stats.ttest_rel`` and ``scipy.stats.ttest_1samp`` have been changed so that 0 / 0 now returns NaN instead of 1. Other changes ============= The SuperLU sources in ``scipy.sparse.linalg`` have been updated to version 4.3 from upstream. The function ``scipy.signal.bode``, which calculates magnitude and phase data for a continuous-time system, has been added. The two-sample T-test ``scipy.stats.ttest_ind`` gained an option to compare samples with unequal variances, i.e. Welch's T-test. ``scipy.misc.logsumexp`` now takes an optional ``axis`` keyword argument. Authors ======= This release contains work by the following people (contributed at least one patch to this release, names in alphabetical order): * Jeff Armstrong * Chad Baker * Brandon Beacher + * behrisch + * borishim + * Matthew Brett * Lars Buitinck * Luis Pedro Coelho + * Johann Cohen-Tanugi * David Cournapeau * dougal + * Ali Ebrahim + * endolith + * Bjørn Forsman + * Robert Gantner + * Sebastian Gassner + * Christoph Gohlke * Ralf Gommers * Yaroslav Halchenko * Charles Harris * Jonathan Helmus + * Andreas Hilboll + * Marc Honnorat + * Jonathan Hunt + * Maxim Ivanov + * Thouis (Ray) Jones * Christopher Kuster + * Josh Lawrence + * Denis Laxalde + * Travis Oliphant * Joonas Paalasmaa + * Fabian Pedregosa * Josef Perktold * Gavin Price + * Jim Radford + * Andrew Schein + * Skipper Seabold * Jacob Silterra + * Scott Sinclair * Alexis Tabary + * Martin Teichmann * Matt Terry + * Nicky van Foreest + * Jacob Vanderplas * Patrick Varilly + * Pauli Virtanen * Nils Wagner + * Darryl Wally + * Stefan van der Walt * Liming Wang + * David Warde-Farley + * Warren Weckesser * Sebastian Werk + * Mike Wimmer + * Tony S Yu + A total of 55 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

2 3

Re: [SciPy-Dev] Adding solvers to scipy.integrate [Was: A step toward merging odeint and ode]
by Ralf Gommers Sept. 18, 2012

Sept. 18, 2012

<moved to scipy-dev> On Tue, Aug 21, 2012 at 10:22 AM, Fabrice Silva <silva(a)lma.cnrs-mrs.fr>wrote: > Le lundi 20 août 2012 à 22:04 +0200, Ralf Gommers a écrit : > > > https://github.com/FabricioS/scipy/commit/f867f2b8133d3f6ea47d449bd760a77a7… > > > This is probably not worth the cost for existing users imho. It is a > > backwards compatibility break that doesn't really add anything except > > for some consistency (right?). > > Hi Ralf, > Ok concerning this point. > > Hi Fabrice, sorry for the slow reply. > > In addition, I have been looking to suggest additional solvers, > essentially simpler scheme, that would thus allow to easily switch > between "complex" (lsode, vode, cvode) and basic schemes (Euler, > Nicholson, etc...) > Sounds like a good idea. I think though that for each solver added there should be a gain in performance for some problems, whether that's speed, accuracy, robustness or something else. Adding a solver for educational purposes or completeness only wouldn't be all that useful. > I came across some code on the Montana Univ.'s Computer Science dpt: > http://wiki.cs.umt.edu/classes/cs477/index.php/Creating_ODE_Solver_Objects > and asked Jesse Johnson (the responsible for that class) what is the > license for that code. Here is his answer : > > Any thing that you find on those pages, you may use. However, > I'm not sure how to go about officially giving the code a > particular license. Can I add a license to the wiki, stating > that it applies to all the code therein? > > PS It is fantastic you're doing this. I've often thought that > scipy.ode could use some improvements. > > He is cc'ed of this mail, could anyone concerned about scipy license > requirements and more generally in code licensing answer him ? > Hi Jesse, thanks a lot for contributing your code. As for the licensing question, I think the best would be to either add a license at the bottom of each page, or in the code itself. Adding a page somewhere on your wiki saying "license X applies to all code in this wiki" is in principle enough, but it may be hard to find for readers. And if they can't find a license, they have to assume they can't reuse your code. As for the license itself, it would be good to be explicit ("public domain" or "BSD" for example). Disclaimer: I'm not a lawyer. Best, Ralf

1 0

Two speed/enhancement PRs
by Jacob VanderPlas Sept. 17, 2012

Sept. 17, 2012

Hello, I currently have two small open PRs which are basically ready for merge, and have been sitting inactive for a while. I'd love a few people to read-through so we can merge: https://github.com/scipy/scipy/pull/289 This PR speeds up the pseudoinverse calculation, adds a fast hermitian pseudoinverse, and adds a `return_rank` option on all pseudoinverse methods https://github.com/scipy/scipy/pull/311 This PR speeds up single-row access in CSR matrices, and allows for fast generic slicing of CSR matrix rows. Thanks! Jake

1 0

distributions.py
by nicky van foreest Sept. 16, 2012

Sept. 16, 2012

Hi, Now that I understand github (Thanks to Ralf for his explanations in Dutch) and got some simple stuff out of the way in distributions.py I would like to tackle a somewhat harder issue. The function argsreduce is, as far as I can see, too generic. I did some tests to see whether its most generic output, as described by its docstring, is actually swallowed by the callers of argsreduce, but this appears not to be the case. My motivation to simplify the code in distributions.py (and clean it up) is partly based on making it simpler to understand for myself, but also to others. The fact that github makes code browsing a much nicer experience, perhaps more people will take a look at what's under the hood. But then the code should also be accessible and clean. Are there any reasons not to pursue this path, and focus on more important problems of the stats library? Nicky

6 18

distributions.py expect
by nicky van foreest Sept. 14, 2012

Sept. 14, 2012

Hi, I am trying to implement the expect method in rv_frozen. To understand the normal working of the expect method I tried the following: from scipy.stats import geom, norm, gamma print norm.expect(loc = 3,scale =5) print gamma.expect(None, 4.5) print gamma.expect(lambda x: x, 4.5) print geom.expect(lambda x: x, 1./3) This is the result: 3.0 4.5 4.5 Traceback (most recent call last): File "expecttest.py", line 6, in <module> print geom.expect(lambda x: x, 1./3) File "/home/nicky/prog/scipy/scipy/stats/distributions.py", line 6375, in expect self._argcheck(*args) # (re)generate scalar self.a and self.b TypeError: _argcheck() argument after * must be a sequence, not float So the first examples work, but the rv_discrete example doesn't. One thing is that the _argcheck is not called in rv_continuous while it is in rv_discrete. Removing this line results in another error: 3.0 4.5 4.5 Traceback (most recent call last): File "expecttest.py", line 6, in <module> print geom.expect(lambda x: x, 1./3) File "/home/nicky/prog/scipy/scipy/stats/distributions.py", line 6394, in expect low, upp = self._ppf(0.001, *args), self._ppf(0.999, *args) TypeError: _ppf() argument after * must be a sequence, not float What would be actually the right way to call the expect method for rv_discrete? Or perhaps the other way around, should this method be changed so that my example with geom works? ---- I extended rv_frozen with this method, and this appears to work well for continuous rvs, but also fails for discrete rvs. def expect(self, *args, **kwds): args += self.args kwds.update(self.kwds) return self.dist.expect(*args, **kwds) Nicky

2 1

stats some further clean up
by nicky van foreest Sept. 13, 2012

Sept. 13, 2012

Hi, distributions.py contains some #hashed documentation from lines https://github.com/nokfi/scipy/blob/master/scipy/stats/distributions.py#L509 to https://github.com/nokfi/scipy/blob/master/scipy/stats/distributions.py#L598 I would prefer to remove it as some parts should better fit in the documentation at http://docs.scipy.org/doc/scipy/reference/tutorial/stats/continuous.html#co… while a lot is already covered in the standard doc strings of rv_continuous and rv_discrete. Should I just go ahead, or first make a somewhat more specified list of what I intend to do? nicky

2 2

continuous distributions documentation
by nicky van foreest Sept. 13, 2012

Sept. 13, 2012

Hi, I updated my git version of https://github.com/scipy/scipy/blob/master/doc/source/tutorial/stats/contin… somewhat. I converted it to latex to convert it, with pandoc, to rst. However, the result is dramatic. Does anybody know how the lyx file was converted to rst so as to get a decent result? BTW, the latex file looks ok, so the problem resides in the conversion to rst. thanks Nicky

1 1

Savitzky-Golay for scipy.signal
by thomash Sept. 13, 2012

Sept. 13, 2012

In my line of work, the Savitzky-Golay filter is one of the most useful filters, and I think it would be worth including it into scipy.signal. "rgommer" has already mentioned that he would support that notion, and I was wondering if it would be ok with the other developers if I go ahead with it? thomash

3 5

ANN: SfePy 2012.3
by Robert Cimrman Sept. 12, 2012

Sept. 12, 2012

I am pleased to announce release 2012.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Home page: http://sfepy.org Downloads, mailing list, wiki: http://code.google.com/p/sfepy/ Git (source) repository, issue tracker: http://github.com/sfepy Highlights of this release -------------------------- - several new terms - material parameters can be defined per region using region names - base function values can be defined per element - support for global options For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Alec Kalinin, Vladimír Lukeš

1 0

Re: [SciPy-Dev] scipy domains
by Travis Vaught Sept. 11, 2012

Sept. 11, 2012

On Sep 10, 2012, at 12:06 PM, Travis Vaught wrote: > Forwarding to scipy-user, since I've not had any thoughts out of scipy-dev on this. > > Maybe this is relevant to the naming discussion. > > Anyone? > > > Begin forwarded message: > >> From: Travis Vaught <travis(a)vaught.net> >> Subject: scipy domains >> Date: September 5, 2012 9:31:49 AM CDT >> To: "Scipy-Dev@Scipy. Org" <scipy-dev(a)scipy.org> >> >> All, >> >> In a fit of nostalgia, I began to renew the domains scipy.com and scipy.net this morning. >> >> I've no idea whether they have any real use other than brand protection, but I'm more than willing to fund the renewals for as long as I have the means to do so. (Note: even though I'm the listed administrator of these domains, I consider Enthought -- with which I'm no longer formally affiliated -- to be their capable caretaker). >> >> In the renewal process, the company from which I register the domains 'suggested' that there are other domains I might be interested in registering. Namely, scipy.info, scipy.us, and scipy.biz. Is there any use for these? I'm happy to add them to the bill if someone can make a case for their use/protection. >> >> Any thoughts are appreciated. >> >> Best, >> >> Travis > Alright, I'm feeling a bit neglected on this, so I'll just renew the scipy.net and scipy.com domains and leave it at that. If anyone needs to admin the domains going forward, let me know. Best, Travis

1 0

Re: [SciPy-Dev] SciPy.org Website Move and Redesign
by Kyle Mandli Sept. 10, 2012

Sept. 10, 2012

I kind of dropped the ball on this as I have been traveling for the past month. I think we are ready to move forward on this and at least get the pieces in place. I am still not clear exactly what the current status of the scipy.org-new and scipy.github.com repos on github are. Can anyone speak to this? On a related topic, I have been searching out someone who may be able to do some redesign work on the site but was not successful (at least a free option). Any suggestions for someone with some design experience that could update the look of the site a bit please let us know. Kyle On Friday, August 24, 2012 at 10:00 AM, scipy-dev-request(a)scipy.org wrote: > On Fri, Jul 27, 2012 at 11:35 AM, Fernando Perez <fperez.net(a)gmail.com (mailto:fperez.net@gmail.com)> wrote: > > And precisely having the site moved over to a system where: > > > > - it's trivial to delegate website work to a separate team who can > > collaborate with the regular github flow (branches, PRs, etc) to > > manage the web content. > > > > - form and content are separated, with templates and CSS for all formal display > > Is this transition still on track? We should push it through, even if > it means living with a slightly outdated website for a week or two. > (I'm sure we'll all feel much more comfortable updating the web site > using PRs than whatever the current mechanism might be.) > > St?fan

6 6

stats cleanup, attempt
by nicky van foreest Sept. 9, 2012

Sept. 9, 2012

Hi, Slowly but surely I am making some progress with respect to understanding the stats library. Hence I feel can start with making some contributions too. As a first attempt I would like to propose some clean up first. There is some code from https://github.com/scipy/scipy/blob/master/scipy/stats/distributions.py#L415 to https://github.com/scipy/scipy/blob/master/scipy/stats/distributions.py#L430 This is not used in distributions.py, and if I recall from a previous mail from Josef it not used at all anymore. Can I remove it? BTW: should I post a mail about proposals like this, or just remove it, and send a pull request with some explanation? Nicky

2 1