Mailman 3 August 2010 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

scipy error compiling csr_wrap under Python 2.7
by Bruce Southey Sept. 27, 2010

Sept. 27, 2010

Hi, I failed to get the scipy 0.8 rc1 and SVN to build under Python2.7 with numpy '2.0.0.dev8469' and gcc version 4.4.4 20100503 (Red Hat 4.4.4-2) (GCC) . But scipy does compile with Python 2.6. building 'scipy.sparse.sparsetools._csr' extension compiling C++ sources C compiler: g++ -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC compile options: '-I/usr/local/lib/python2.7/site-packages/numpy/core/include -I/usr/local/include/python2.7 -c' g++: scipy/sparse/sparsetools/csr_wrap.cxx scipy/sparse/sparsetools/csr_wrap.cxx: In function ‘int require_size(PyArrayObject*, npy_intp*, int)’: scipy/sparse/sparsetools/csr_wrap.cxx:2910: error: expected ‘)’ before ‘PRIdPTR’ scipy/sparse/sparsetools/csr_wrap.cxx:2910: warning: spurious trailing ‘%’ in format scipy/sparse/sparsetools/csr_wrap.cxx:2910: warning: too many arguments for format scipy/sparse/sparsetools/csr_wrap.cxx:2917: error: expected ‘)’ before ‘PRIdPTR’ scipy/sparse/sparsetools/csr_wrap.cxx:2917: warning: spurious trailing ‘%’ in format scipy/sparse/sparsetools/csr_wrap.cxx:2917: warning: too many arguments for format scipy/sparse/sparsetools/csr_wrap.cxx: In function ‘int require_size(PyArrayObject*, npy_intp*, int)’: scipy/sparse/sparsetools/csr_wrap.cxx:2910: error: expected ‘)’ before ‘PRIdPTR’ scipy/sparse/sparsetools/csr_wrap.cxx:2910: warning: spurious trailing ‘%’ in format scipy/sparse/sparsetools/csr_wrap.cxx:2910: warning: too many arguments for format scipy/sparse/sparsetools/csr_wrap.cxx:2917: error: expected ‘)’ before ‘PRIdPTR’ scipy/sparse/sparsetools/csr_wrap.cxx:2917: warning: spurious trailing ‘%’ in format scipy/sparse/sparsetools/csr_wrap.cxx:2917: warning: too many arguments for format error: Command "g++ -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python2.7/site-packages/numpy/core/include -I/usr/local/include/python2.7 -c scipy/sparse/sparsetools/csr_wrap.cxx -o build/temp.linux-x86_64-2.7/scipy/sparse/sparsetools/csr_wrap.o" failed with exit status 1 The error is possibly related the variable 'NPY_INTP_FMT' (defined in numpy/core/include/numpy/ndarraytypes.h). A grep shows these files: scipy/sparse/sparsetools/coo_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/coo_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); scipy/sparse/sparsetools/bsr_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/bsr_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); scipy/sparse/sparsetools/numpy.i: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/numpy.i: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); scipy/sparse/sparsetools/csr_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/csr_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); scipy/sparse/sparsetools/csc_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/csc_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); scipy/sparse/sparsetools/dia_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", size[i]); scipy/sparse/sparsetools/dia_wrap.cxx: sprintf(s,"%" NPY_INTP_FMT ",", array_size(ary,i)); Note that the scipy/sparse/sparsetools/numpy.i contains tabs in some places instead of spaces. The ndarraytypes.h is present in /usr/local/lib/python2.7/site-packages/numpy/core/include/numpy/ Bruce

5 10

N-dimensional interpolation
by Pauli Virtanen Sept. 6, 2010

Sept. 6, 2010

Hi all, I took the Qhull by the horns, and wrote a straightforward `griddata` implementation for working in N-D: http://github.com/pv/scipy-work/tree/qhull http://github.com/pv/scipy-work/blob/qhull/scipy/interpolate/qhull.pyx Sample (4-D): ------------------------------------------------------------- import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import griddata def f(x, y, z, u): return np.cos(x + np.cos(y + np.cos(z + np.cos(u)))) points = np.random.rand(2000, 4) values = f(*points.T) p = np.linspace(0, 1, 1500) z = griddata(points, values, np.c_[p, p, p, p], method='linear') plt.plot(p, z, '-', p, f(p, p, p, p), '-') plt.show() ------------------------------------------------------------- It performs the N-D Delaunay tesselation with Qhull, and after that does linear barycentric interpolation on the simplex containing each point. (The nearest-neighbor "interpolation" is, on the other hand, implemented using scipy.spatial.KDTree.) I ended up writing some custom simplex-walking code for locating the simplex containing the points -- this is much faster than a brute-force search. However, it is slightly different from what `qh_findbestfacet` does. [If you're a computational geometry expert, see below...] Speed-wise, Qhull appears to lose in 2-D to `scikits.delaunay` by a factor of about 10 in triangulation in my tests; this, however, includes the time taken by computing the barycentric coordinate transforms. Interpolation, however, seems to be faster. *** I think this would be a nice addition to `scipy.optimize`. I'd like to get it in for 0.9. Later on, we can specialize the `griddata` function to perform better in low dimensions (ndim=2, 3) and add more interpolation methods there; for example natural neighbors, interpolating splines, etc. In ndim > 3, the `griddata` is however already feature-equivalent to Octave and MATLAB. Comments? Pauli PS. A question for computational geometry experts: `qh_findbestfacet` finds the facet on the lower convex hull whose oriented hyperplane distance to the point lifted on the paraboloid is maximal --- however, this does not always give the simplex containing the point in the projected Delaunay tesselation. There's a counterexample in qhull.pyx:_find_simplex docstring. On the other hand, Qhull documentation claims that this is the way to locate the Delaunay simplex containing a point. What gives? It's clear, however, that if a simplex contains the point, then the hyperplane distance is positive. So currently, I just walk around positive-distance simplices checking the inside condition for each and hoping for the best. If nothing is found, I just fall back to brute force search. This seems to work well in practice. However, if a point is outside the triangulation (but inside the bounding box), I have to fall back to a brute-force search. Is there a better option here? *** Another sample (2-D, object-oriented interface): ------------------------------------------------------------- import time import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import LinearNDInterpolator # some random data x = np.array([(0,0), (0,1), (1, 1), (1, 0)], dtype=np.double) np.random.seed(0) xr = np.random.rand(200*200, 2).astype(np.double) x = np.r_[xr, x] y = np.arange(x.shape[0], dtype=np.double) # evaluate on a grid nn = 500 xx = np.linspace(-0.1, 1.1, nn) yy = np.linspace(-0.1, 1.1, nn) xx, yy = np.broadcast_arrays(xx[:,None], yy[None,:]) xi = np.array([xx, yy]).transpose(1,2,0).copy() start = time.time() ip = LinearNDInterpolator(x, y) print "Triangulation", time.time() - start start = time.time() zi = ip(xi) print "Interpolation", time.time() - start from scikits.delaunay import Triangulation start = time.time() tri2 = Triangulation(x[:,0], x[:,1]) print "scikits.delaunay triangulation", time.time() - start start = time.time() ip2 = tri2.linear_interpolator(y) zi2 = ip2[-0.1:1.1:500j,-0.1:1.1:500j].T print "scikits.delaunay interpolation", time.time() - start print "rel-difference", np.nanmax(abs(zi - zi2))/np.nanmax(np.abs(zi)) plt.figure() plt.imshow(zi) plt.clim(y.min(), y.max()) plt.colorbar() plt.figure() plt.imshow(zi2) plt.clim(y.min(), y.max()) plt.colorbar() plt.show() ------------------------------------------------------------- -- Pauli Virtanen

5 8

Travis: test_continuous_basic.py
by josef.pktd＠gmail.com Aug. 30, 2010

Aug. 30, 2010

Travis, Is there a reason why you disabled the kolmogorov-smirnov tests ? http://projects.scipy.org/scipy/changeset/6472/trunk/scipy/stats/tests/test… 180 yield check_distribution_rvs, dist, args, alpha, rvs 181 184 # yield check_distribution_rvs, dist, args, alpha, rvs Josef

1 0

SciPy 0.9.0.dev6651 test failures (segfault)
by Gökhan Sever Aug. 27, 2010

Aug. 27, 2010

Hello, Again on Fedora 13 Virtualbox setup: Linux a 2.6.33.6-147.2.4.fc13.i686 #1 SMP Fri Jul 23 17:27:40 UTC 2010 i686 i686 i386 GNU/Linux Python 2.6.4 (r264:75706, Jun 4 2010, 18:20:16) [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy >>> scipy.test('full') Running unit tests for scipy NumPy version 2.0.0.dev8671 NumPy is installed in /usr/lib/python2.6/site-packages/numpy SciPy version 0.9.0.dev6651 SciPy is installed in /usr/lib/python2.6/site-packages/scipy Python version 2.6.4 (r264:75706, Jun 4 2010, 18:20:16) [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] nose version 0.11.3 ................................................................................................................................................................................................................................................................................/usr/lib/python2.6/site-packages/scipy/interpolate/fitpack2.py:670: UserWarning: The coefficients of the spline returned have been computed as the minimal norm least-squares solution of a (numerically) rank deficient system (deficiency=7). If deficiency is large, the results may be inaccurate. Deficiency may strongly depend on the value of eps. warnings.warn(message) ....../usr/lib/python2.6/site-packages/scipy/interpolate/fitpack2.py:601: UserWarning: The required storage space exceeds the available storage space: nxest or nyest too small, or s too small. The weighted least-squares spline corresponds to the current set of knots. warnings.warn(message) .............................................K..K............................................................Warning: divide by zero encountered in log Warning: invalid value encountered in multiply Warning: divide by zero encountered in log Warning: invalid value encountered in multiply Warning: divide by zero encountered in log Warning: invalid value encountered in multiply .Warning: divide by zero encountered in log Warning: invalid value encountered in multiply Warning: divide by zero encountered in log Warning: invalid value encountered in multiply .Warning: divide by zero encountered in log Warning: invalid value encountered in multiply Warning: divide by zero encountered in log Warning: invalid value encountered in multiply .........Warning: divide by zero encountered in log Warning: invalid value encountered in multiply Warning: divide by zero encountered in log Warning: invalid value encountered in multiply ...................................................................................................................................................................................................................................................................................................................................................../usr/lib/python2.6/site-packages/scipy/io/recaster.py:328: ComplexWarning: Casting complex values to real discards the imaginary part test_arr = arr.astype(T) ../usr/lib/python2.6/site-packages/scipy/io/recaster.py:375: ComplexWarning: Casting complex values to real discards the imaginary part return arr.astype(idt) ../usr/lib/python2.6/site-packages/scipy/io/wavfile.py:30: WavFileWarning: Unfamiliar format bytes warnings.warn("Unfamiliar format bytes", WavFileWarning) /usr/lib/python2.6/site-packages/scipy/io/wavfile.py:120: WavFileWarning: chunk not understood warnings.warn("chunk not understood", WavFileWarning) .......................................................................................F............................................/usr/lib/python2.6/site-packages/scipy/lib/blas/tests/test_fblas.py:86: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) ....../usr/lib/python2.6/site-packages/scipy/lib/blas/tests/test_fblas.py:196: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) .................../usr/lib/python2.6/site-packages/scipy/lib/blas/tests/test_fblas.py:279: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) ..................................................................SSSSSS......SSSSSS......SSSS.....................................................FF........F..Warning: invalid value encountered in divide .....Warning: invalid value encountered in divide Warning: invalid value encountered in divide .......................................................................................................................................................................................................K............................................./usr/lib/python2.6/site-packages/scipy/linalg/tests/test_fblas.py:89: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) ....../usr/lib/python2.6/site-packages/scipy/linalg/tests/test_fblas.py:199: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) .................../usr/lib/python2.6/site-packages/scipy/linalg/tests/test_fblas.py:282: ComplexWarning: Casting complex values to real discards the imaginary part self.blas_func(x,y,n=3,incy=5) ....................................................................../usr/lib/python2.6/site-packages/scipy/linalg/matfuncs.py:94: ComplexWarning: Casting complex values to real discards the imaginary part return dot(dot(vr,diag(exp(s))),vri).astype(t) .................................................................................................................................................................................................................................................../usr/lib/python2.6/site-packages/scipy/ndimage/tests/test_ndimage.py:56: ComplexWarning: Casting complex values to real discards the imaginary part a = a.astype(numpy.float64) /usr/lib/python2.6/site-packages/scipy/ndimage/tests/test_ndimage.py:58: ComplexWarning: Casting complex values to real discards the imaginary part b = b.astype(numpy.float64) ......................................................................................................Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide .....................................................................Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide ........................Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide Warning: invalid value encountered in divide ..............................................9 7 ....F...............Warning: invalid value encountered in divide ................./usr/lib/python2.6/site-packages/scipy/signal/filter_design.py:247: BadCoefficients: Badly conditioned filter coefficients (numerator): the results may be meaningless "results may be meaningless", BadCoefficients) ..............................................................................................................................................................................................................................................................................................SSSSSSSSSSS..........Segmentation fault (core dumped) -- Gökhan

3 4

NumPy 2.0.0.dev8671 test failure
by Gökhan Sever Aug. 27, 2010

Aug. 27, 2010

Hello, On a Fedora 13 VirtualBox setup Linux a 2.6.33.6-147.2.4.fc13.i686 #1 SMP Fri Jul 23 17:27:40 UTC 2010 i686 i686 i386 GNU/Linux python -c 'import numpy; numpy.test()' Running unit tests for numpy NumPy version 2.0.0.dev8671 NumPy is installed in /usr/lib/python2.6/site-packages/numpy Python version 2.6.4 (r264:75706, Jun 4 2010, 18:20:16) [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)] nose version 0.11.3 ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K..................................................................................................................................................................................................................................K............................................................................................K......................K.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F.....................................................................................................................................................................................................................................................................................................................................................................................................................................Warning: divide by zero encountered in log ....................................................................................................................................................................................................................................................................................... ====================================================================== FAIL: test_lapack (test_build.TestF77Mismatch) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/numpy/testing/decorators.py", line 146, in skipper_func return f(*args, **kwargs) File "/usr/lib/python2.6/site-packages/numpy/linalg/tests/test_build.py", line 50, in test_lapack information.""") AssertionError: Both g77 and gfortran runtimes linked in lapack_lite ! This is likely to cause random crashes and wrong results. See numpy INSTALL.txt for more information. "Fail the test if the expression is true." >> if True: raise self.failureException, 'Both g77 and gfortran runtimes linked in lapack_lite ! This is likely to\ncause random crashes and wrong results. See numpy INSTALL.txt for more\ninformation.' ---------------------------------------------------------------------- Ran 3024 tests in 21.928s FAILED (KNOWNFAIL=4, failures=1) Any idea how to resolve this one? I use package manager to install requirements. It seems g77 and gfortran are mixed for lapack, but not sure how to fix it. When I try to uninstall gfortran it tries to remove lapack/blas/atlas all. -- Gökhan

2 3

scipy.stats.kde
by Sam Birch Aug. 27, 2010

Aug. 27, 2010

Hi all, I was thinking of renovating the kernel density estimation package (although no promises; I'm leaving for college tomorrow morning!). I was wondering: a) whether anyone had started code in that direction b) what people want in it I was thinking (as an ideal, not necessarily goal): - Support for more than Gaussian kernels (e.g. custom, uniform, Epanechnikov, triangular, quartic, cosine, etc.) - More options for bandwidth selection (custom bandwidth matrices, AMISE optimization, cross-validation, etc.) - Assorted conveniences: automatically generate the mesh, limit the kernel's support for speed So, thoughts anyone? I figure it's better to over-specify and then under-produce, so don't hold back. Thanks, Sam

4 10

splitting an ordered list as evenly as possilbe
by John Hunter Aug. 25, 2010

Aug. 25, 2010

Suppose I have an ordered list/array of numbers, and I want to split them into N chunks, such that the intersection of any chunk with each other is empty and the data is split as evenly as possible (eg the std dev of the lengths of the chunks is minimized or some other such criterion). Context: I am trying to do a quintile analysis on some data, and np.percentile doesn't behave like I want because more than 20% of my data equals 1, so 1 is in the first and second quintiles. I want to avoid this -- I'd rather have uneven counts in my quintiles than have the same value show up in multiple quintiles, but I'd like the counts to be as even as possible.. Here is some sample code that illustrates my problem: In [178]: run ~/test tile i=1 range=[1.00, 1.00), count=0 tile i=2 range=[1.00, 3.00), count=79 tile i=3 range=[3.00, 4.60), count=42 tile i=4 range=[4.60, 11.00), count=39 tile i=5 range=[11.00, 43.00), count=41 import numpy as np x = np.array([ 2., 3., 4., 5., 1., 2., 1., 1., 1., 2., 3., 1., 2., 3., 1., 2., 3., 1., 2., 3., 4., 1., 1., 2., 3., 2., 2., 3., 4., 5., 1., 2., 3., 4., 5., 6., 7., 1., 1., 2., 3., 4., 5., 6., 7., 1., 2., 3., 1., 2., 1., 2., 3., 1., 2., 4., 1., 2., 1., 2., 3., 4., 5., 6., 1., 2., 3., 1., 1., 1., 1., 1., 1., 2., 1., 2., 3., 1., 2., 3., 1., 1., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 1., 1., 2., 3., 1., 2., 3., 4., 5., 6., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31., 32., 33., 34., 35., 36., 37., 38., 39., 40., 41., 42., 43., 1., 2., 3., 4., 5., 6., 7., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 1., 2., 1., 2., 3., 4., 5., 6., 1., 2., 3., 4., 5., 6., 1., 2., 3., 4., 1., 2., 3., 4., 5., 6., 1., 2., 3., 1., 2., 1., 2.]) tiles = np.percentile(x, (0, 20, 40, 60, 80, 100)) print for i in range(1, len(tiles)): xmin, xmax = tiles[i-1], tiles[i] print 'tile i=%d range=[%.2f, %.2f), count=%d'%(i, xmin, xmax, ((x>=xmin) & (x<xmax)).sum())

6 8

Getting Scipy's weave to work reliably on Windows
by Chris Ball Aug. 24, 2010

Aug. 24, 2010

Hi, While testing Scipy's weave on several different Windows installations, I came across some problems with spaces in paths that often prevent weave from working. I can see a change that could probably get weave working on most Windows installations, but it is a quick hack. Someone knowledgeable about distutils (and numpy.distutils?) might be able to help me fix this properly. Below I describe three common problems with weave on Windows, in the hope that this information helps others, or allows someone to suggest how to fix the spaces-in- paths problem properly. I think there are three common problems that stop weave from working on Windows. The first is not having a C compiler. Both Python(x,y) and EPD provide a C compiler that seems to work fine, which is great! The second problem is that if weave is installed to a location with a space in the path, linking fails. There is already a scipy bug report about this (http://projects.scipy.org/scipy/ticket/809). I've just commented on that report, saying the problem appears to be with distutils, and there is already a Python bug report about it (http://bugs.python.org/issue4508). Maybe someone could close this scipy bug, or link it to the Python one somehow? In any case, when using Python(x,y) or EPD, this bug will not show up if the default installation locations are accepted. So, that's also good news! The third problem is that if the Windows user name has a space in it (which in my experience is quite common), compilation fails. Weave uses the user name to create a path for its "intermediate" and "compiled" files. When the compilation command is issued, the path with the space in it is also not quoted. Presumably that is another error in distutils (or numpy.distutils)? Unfortunately I wasn't able to pinpoint what function is failing to quote strings properly, because I couldn't figure out the chain that leads to the compiler being called. However, I can avoid the problem by removing spaces from the user name in weave itself (catalog.py): def whoami(): """return a string identifying the user.""" return (os.environ.get("USER") or os.environ.get("USERNAME") or "unknown").replace(" ","") (where I have added .replace(" ","") to the existing code). I realize this isn't the right solution, so if someone could help to guide me to the point where quoting should occur, that would be very helpful. Otherwise, is there any chance of applying a hack like this so weave can work reliably on Windows? Thanks, Chris

2 1

ltisys.py
by Roberto Bucher Aug. 23, 2010

Aug. 23, 2010

I'm working on a control system toolbox for python. In particular I've modified some functions of the control system toolbox developed by Richard Murray and I added some new functions. The first problem is related to the modul ltisys.py that I modified for handling: - MIMO systems (state-space only) - Sampling Time for discrete time system I've done other modifications to different modul of Richard, who is in CC: - matlab.py - statesp.py - xferfcn.py in order to implement the following new functions in my yottalab.py modul: - c2d (zoh+bilinear) - d2c (zoh+bilinear) - dare - discrete riccati solution - care - continous riccati solution - dlqr - discrete linear quadratic regulator - ctrb - controllability matrix - acker - ackerman pole placement - minreal - minimal state space representation - dcgain steady state gain for both continous and discrete time systems - tf (casting function) - ss (casting function) The modified files can be downloaded from my homepage in the python section, but I want to see how I can contribute to put the modifications in the main Scipy distribution. Best regards Roberto -- ----------------------------------------------------------------------------- Coltivate Linux! Tanto Windows si pianta da solo... ----------------------------------------------------------------------------- University of Applied Sciences of Southern Switzerland Dept. Innovative Technologies CH-6928 Lugano-Manno http://web.dti.supsi.ch/~bucher

4 4

question to scikits.appspot.com editors
by Dmitrey Aug. 23, 2010

Aug. 23, 2010

hi all, who is responsible for scikits.appspot.com editing? I see some scikits (e.g. learn) has mentioned their personal website ( http://scikit-learn.sourceforge.net), while openopt entry still points to deprecated location that is out of maintanance for several years; same for svn root mentioned there. Could you either fix it to modern locations (http://openopt.org, svn://openopt.org/PythonPackages/OpenOpt) or, at least, remove it at all to prevent misleading of users? Regards, D.

3 2

could you consider creating "build/install issues" mail list?
by Dmitrey Aug. 21, 2010

Aug. 21, 2010

Hi all, could you consider creating special mail list "build/install numpy/scipy issues" or something like that? I guess lots of people (as well as I) are not interested in reading all those huge amounts of messages related to these issues. Regards, D.

3 5

status of http://ask.scipy.org and http://advice.mechanicalkern.com/
by Jarrod Millman Aug. 18, 2010

Aug. 18, 2010

Hello, This email was prompted by a blog post from William Stein: http://sagemath.blogspot.com/2010/08/overflow.html Is http://ask.scipy.org the official site at this point? What is the plan for http://advice.mechanicalkern.com/? Thanks, Jarrod

3 3

Contingency Table Model
by Anthony Scopatz Aug. 11, 2010

Aug. 11, 2010

Hello All, I have just opened a ticket (http://projects.scipy.org/scipy/ticket/1258) that adds a general contingency table class to the the stats package. This class includes methods to slice and collapse the table as well a calculate metrics such as chi-squared and entropy. This implementation came out of Warren Weckesser and me working on this over the SciPy 2010 statistics sprint. Please take a look! Comments and suggestions are always welcome. Be Well, Anthony

3 7

First contact
by Roberto Bucher Aug. 11, 2010

Aug. 11, 2010

Hi all I'm new in the mailing list and I'd like to contact people responsible of the signal/ltisys module. I'm working on a porting of all my control functions from Scicoslab under Python and I found some errors(?) in the signal/ltisys module. In addition I added some code to the module for handling state space MIMO systems. The first problem was related with the function "ss2tf" which gives as return numerator a 2 dimensional array, not more usable by the print function of the class. I added these lines at the end of the function: # Avoid leading zeros in num num=num[0] while num[0]==0: num=num[1:] but I don't want to get problems with other modules... Best regards Roberto -- ----------------------------------------------------------------------------- Coltivate Linux! Tanto Windows si pianta da solo... ----------------------------------------------------------------------------- University of Applied Sciences of Southern Switzerland Dept. Innovative Technologies CH-6928 Lugano-Manno http://web.dti.supsi.ch/~bucher

2 2

ANN: SfePy 2010.3
by Robert Cimrman Aug. 6, 2010

Aug. 6, 2010

I am pleased to announce release 2010.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method. The code is based on NumPy and SciPy packages. It is distributed under the new BSD license. Development, mailing lists, issue tracking: http://sfepy.org Documentation: http://docs.sfepy.org/doc Git repository: http://github.com/sfepy Project page: http://sfepy.kme.zcu.cz Highlights of this release -------------------------- - significantly rewritten code for better interactive use - cleaner and simpler high level interface - new tutorial section: - Interactive Example: Linear Elasticity [1] [1] http://docs.sfepy.org/doc/tutorial.html#interactive-example-linear-elastici… Major improvements ------------------ Apart from many bug-fixes, let us mention: - new examples: - demonstration of the high level interface - new tests: - tests of the new high level interface - simplified but more powerful homogenization engine For more information on this release, see http://sfepy.googlecode.com/svn/web/releases/2010.3_RELEASE_NOTES.txt (full release notes, rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Vladimír Lukeš, Osman, Andre Smit, Logan Sorenson

1 0

scipy.org down?
by David Goldsmith Aug. 6, 2010

Aug. 6, 2010

I'm getting "Network Timeout" failures trying to visit www.scipy.org and mail.scipy.org... DG

3 3

Status of scipy.* docstrings
by David Goldsmith Aug. 5, 2010

Aug. 5, 2010

Hi, folks! Except for scipy.stats, the docstrings of all sub-packages immediately "below" scipy have now had autosummary directives for all (with one exception) their objects added to them, with either the existing description if there already was one, or the description pulled in by the autosummary (perhaps paraphrased if necessary to satisfy the 75 character restriction) if said pulling worked, or "TODO" if neither of those conditions were met; scipy.stats was already partially under autosummary "control" when I checked it, so I've left it alone pending further info as to whether this incompleteness is intentional or not. I will continue to work my way down the namespace tree - at a reduced pace - but I just wanted to announce that this *ad hoc* (i.e., "unofficial") milestone has been met, and point out that some of the holes alluded to above can serve as pointers to places that need work, e.g., places where the autosummary directive either pulled nothing, "failed to parse" the summary, or pulled an excessively long description (indicating, IIUC, a docstring w/ an excessively long Brief Summary). Thanks for all you do (and thanks for all the kudos that have come in post my resignation announcement). DG -- Mathematician: noun, someone who disavows certainty when their uncertainty set is non-empty, even if that set has measure zero.

4 16

Would it be acceptable to give up subpackage build ?
by David Cournapeau July 31, 2010

July 31, 2010

Hi there, I have made enough progress on bento recently to start tackling the meat of the issue: building numpy and scipy with it. Numpy goes nicely, but scipy causes issues because of the following constraing: in scipy, each package with a setup.py is a self-contained package, i.e. you can just do python setup.py build in scipy/lib/blas, and it will run. Is it acceptable to give up on this in the long run ? I don't see much advantages in that feature, and giving it up has several nice advantages: - one can have a global configure stage (i.e. check blas/lapack only once instead of many times as done currently, although the way numpy.distutils has near 0 cost since it only stat files) - I would expect the build to be faster and more accurate - it makes my life easier for bento Note that this has no incidence on the existing distutils-based infrastructure, cheers, David

3 2