Mailman 3 June 2009 - SciPy-Dev

SLSQP Constrained Optimizer Status
by Rob Falck Nov. 24, 2013

Nov. 24, 2013

I'm currently implementing the Sequential Least Squares Quadratic Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy. The Fortran code being wrapped with F2PY is here: http://www.netlib.org/toms/733 (its use within Scipy has been cleared) SLSQP provides for bounds on the independent variables, as well as equality and inequality constraint functions, which is a capability that doesn't exist in scipy.optimize. Currently the code works, although the constraint normals are being generated by approximation. I'm working on a way to pass these in. I think the most elegant way will be a single function that returns the matrix of constraint normals. For a demonstration of what the code can do, here is an optimization of f(x,y) = 2xy + 2x - x**2 - 2y**2 Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0. The unbounded optimization tests find this solution. As expected, its faster when derivatives are provided rather than approximated. Unbounded optimization. Derivatives approximated. Elapsed time: 1.45792961121 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] Unbounded optimization. Derivatives provided. Elapsed time: 1.03211402893 ms Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4, 0, 'Optimization terminated successfully.'] The following example uses an equality constraint to find the optimal when x=y. Bound optimization. Derivatives approximated. Elapsed time: 1.384973526 ms Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889, 4, 0, 'Optimization terminated successfully.'] I've tried to conform to the syntax used by the other optimizers in scipy.optimize. The function definition and doc string are below. If anyone is interested in testing it out, let me know. def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[], f_ieqcons=None, bounds = [], fprime = None, fprime_cons=None,args = (), iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0, epsilon = _epsilon ): """ Minimize a function using Sequential Least SQuares Programming Description: Python interface function for the SLSQP Optimization subroutine originally implemented by Dieter Kraft. Inputs: func - Objective function (in the form func(x, *args)) x0 - Initial guess for the independent variable(s). eqcons - A list of functions of length n such that eqcons[j](x0,*args) == 0.0 in a successfully optimized problem f_eqcons - A function of the form f_eqcons(x, *args) that returns an array in which each element must equal 0.0 in a successfully optimized problem. If f_eqcons is specified, eqcons is ignored. ieqcons - A list of functions of length n such that ieqcons[j](x0,*args) >= 0.0 in a successfully optimized problem f_ieqcons - A function of the form f_ieqcons(x0, *args) that returns an array in which each element must be greater or equal to 0.0 in a successfully optimized problem. If f_ieqcons is specified, ieqcons is ignored. bounds - A list of tuples specifying the lower and upper bound for each independent variable [ (xl0, xu0), (xl1, xu1), ...] fprime - A function that evaluates the partial derivatives of func fprime_cons - A function of the form f(x, *args) that returns the m by n array of constraint normals. If not provided, the normals will be approximated. Equality constraint normals precede inequality constraint normals. args - A sequence of additional arguments passed to func and fprime iter - The maximum number of iterations (int) acc - Requested accuracy (float) iprint - The verbosity of fmin_slsqp. iprint <= 0 : Silent operation iprint == 1 : Print summary upon completion (default) iprint >= 2 : Print status of each iterate and summary full_output - If 0, return only the minimizer of func (default). Otherwise, output final objective function and summary information. epsilon - The step size for finite-difference derivative estimates. Outputs: ( x, { fx, gnorm, its, imode, smode }) x - The final minimizer of func. fx - The final value of the objective function. its - The number of iterations. imode - The exit mode from the optimizer, as an integer. smode - A string describing the exit mode from the optimizer. Exit modes are defined as follows: -1 : Gradient evaluation required (g & a) 0 : Optimization terminated successfully. 1 : Function evaluation required (f & c) 2 : Number of equality constraints larger than number of independent variables 3 : More than 3*n iterations in LSQ subproblem 4 : Inequality constraints incompatible 5 : Singular matrix E in LSQ subproblem 6 : Singular matrix C in LSQ subproblem 7 : Rank-deficient equality constraint subproblem HFTI 8 : Positive directional derivative for linesearch 9 : Iteration limit exceeded -- - Rob Falck

4 16

Continuous Wavelet Transform for SciPy
by Sean Arms Nov. 12, 2009

Nov. 12, 2009

Greetings! My name is Sean Arms and I'm a graduate student at the University of Oklahoma in the School of Meteorology. As part of my PhD research, I'm studying coherent structures in atmospheric boundary layer turbulence, primarily using in-situ observations and, secondarily, Direct Numerical Simulation (DNS) output. One common approach for detecting coherent structures in observational datasets relies on the use of the global wavelet power spectrum as estimated from a continuous wavelet transform (CWT). I know SciPy has a DWT impementation, and I've already been in contact with Filip. He recommeded that I post my code in hopes that it would add some momentum to the python-cwt development and create some feedback (I'm currently looking for a good place to post my code). I've implemented the CWT using pure python (that is, I did not write any C extension code myself - nothing to build), along with one mother wavelet (second derivitive of a Gaussian, or the Mexican Hat) - I'll be adding more Mother wavelets as I go along. I've made it a point to (try to) design my MotherWavelet class to be easily extendable. I'm working on documentation and a few tests at the moment, but so far my code compares well with other wavelet routines. The point of this email is to introduce myself and let the SciPy dev community know that I am willing to help develop CWT support for SciPy - I'll already be doing the work for my research, so I might as well put in the extra effort to make is usable by the larger community! Cheers! Sean Arms Graduate Research Assistant School of Meteorology University of Oklahoma

4 7

Porting SciPy to Py3k GSOC project
by ross smith July 19, 2009

July 19, 2009

Hello everyone, I am interested in porting SciPy/NumPy to Py3k. I've been working this past school year to port an existing code base to py3k for a research group on campus. A lot of the code relies on SciPy and NumPy but the scope of my project didn't let me work on porting either project, to my dismay. I'd love the opportunity to port a project I use heavily in my own code and gain a better understanding of how it works. We are supposed to contact the group we would be working with, to flesh out the details of our application. I've looked at the application and the only thing I know I'll need significant help with is the Milestones portion. Of course, Any and all suggestions are welcome! thank you, Ross Smith (Gaurdro on Freenode)

2 3

rgb_to_hsv in scipy.misc ? (was: [Numpy-discussion] Optimizing speed for large-array inter-element algorithms (specifically, color space conversion))
by Nicolas Pinto July 8, 2009

July 8, 2009

Hello, Would it be possible to include the following rgb to hsv conversion code in scipy (probably in misc along with misc.imread, etc.) ? What do you think? Thanks in advance. Best regards, -- Nicolas Pinto Ph.D. Candidate, Brain & Computer Sciences Massachusetts Institute of Technology, USA http://web.mit.edu/pinto # ------------------------------------------------------------------------------ import numpy as np def rgb_to_hsv_arr(arr): """ fast rgb_to_hsv using numpy array """ # adapted from Arnar Flatberg # http://www.mail-archive.com/numpy-discussion@scipy.org/msg06147.html # it now handles NaN properly and mimics colorsys.rgb_to_hsv output arr = arr/255. out = np.empty_like(arr) arr_max = arr.max(-1) delta = arr.ptp(-1) s = delta / arr_max s[delta==0] = 0 # red is max idx = (arr[:,:,0] == arr_max) out[idx, 0] = (arr[idx, 1] - arr[idx, 2]) / delta[idx] # green is max idx = (arr[:,:,1] == arr_max) out[idx, 0] = 2. + (arr[idx, 2] - arr[idx, 0] ) / delta[idx] # blue is max idx = (arr[:,:,2] == arr_max) out[idx, 0] = 4. + (arr[idx, 0] - arr[idx, 1] ) / delta[idx] out[:,:,0] = (out[:,:,0]/6.0) % 1.0 out[:,:,1] = s out[:,:,2] = arr_max # rescale back to [0, 255] out *= 255. # remove NaN out[np.isnan(out)] = 0 return out

4 5

Tutorial topics for SciPy'09 Conference
by Fernando Perez July 1, 2009

July 1, 2009

Hi all, The time for the Scipy'09 conference is rapidly approaching, and we would like to both announce the plan for tutorials and solicit feedback from everyone on topics of interest. Broadly speaking, the plan is something along the lines of what we had last year: one continuous 2-day tutorial aimed at introductory users, starting from the very basics, and in parallel a set of 'advanced' tutorials, consisting of a series of 2-hour sessions on specific topics. We will request that the presenters for the advanced tutorials keep the 'tutorial' word very much in mind, so that the sessions really contain hands-on learning work and not simply a 2-hour long slide presentation. We will thus require that all the tutorials will be based on tools that the attendees can install at least 2 weeks in advance on all platforms (no "I released it last night" software). With that in mind, we'd like feedback from all of you on possible topics for the advanced tutorials. We have space for 8 slots total, and here are in no particular order some possible topics. At this point there are no guarantees yet that we can get presentations for these, but we'd like to establish a first list of preferred topics to try and secure the presentations as soon as possible. This is simply a list of candiate topics that various people have informally suggested so far: - Mayavi/TVTK - Advanced topics in matplotlib - Statistics with Scipy - The TimeSeries scikit - Designing scientific interfaces with Traits - Advanced numpy - Sparse Linear Algebra with Scipy - Structured and record arrays in numpy - Cython - Sage - general tutorial - Sage - specific topics, suggestions welcome - Using GPUs with PyCUDA - Testing strategies for scientific codes - Parallel processing and mpi4py - Graph theory with Networkx - Design patterns for efficient iterator-based scientific codes. - Symbolic computing with sympy We'd like to hear from any ideas on other possible topics of interest, and we'll then run a doodle poll to gather quantitative feedback with the final list of candidates. Many thanks, f

1 2

Memory usage of scipy.io.loadmat
by Ariel Rokem June 30, 2009

June 30, 2009

Hi everyone, I have stumbled on some interesting behavior of scipy.io.loadmat. The short of it - it looks like loadmat is gobbling up memory in some unjustified manner and releasing it under some strange circumstances. Here's the long story. This is all happening on a Mac OS 10.5.7, running EPD 4.0.30001 (Python 2.5.2), but with a relatively new version of scipy (see below). I start ipython -pylab in one terminal and run 'top' in another, in order to monitor the memory usage. Here's what I get initially: PhysMem: 417M wired, 449M active, 183M inactive, 1055M used, 3041M free. Then: In [1]: import scipy In [2]: scipy.__version__ Out[2]: '0.8.0.dev5606' In [3]: import scipy.io as sio Here's what it looks like now: PhysMem: 418M wired, 450M active, 183M inactive, 1058M used, 3038M free. So far, so good. I read in a large matfile with tons of data in it: In [4]: a = sio.loadmat('/Users/arokem/Projects/SchizoSpread/Scans/SMR033109_MC/Gray/Original/TSeries/Scan1/tSeries1.mat') PhysMem: 419M wired, 1024M active, 183M inactive, 1632M used, 2464M free. So - about 600 MB of memory is taken up by this new variable. Now to the wierdness: In [5]: b --------------------------------------------------------------------------- NameError Traceback (most recent call last) /Users/arokem/<ipython console> in <module>() NameError: name 'b' is not defined Of course - 'b' doesn't exist! But now, the memory usage has dramatically gone down: PhysMem: 420M wired, 740M active, 183M inactive, 1350M used, 2746M free. So - just invoking an error in the ipython command line has freed up 300 MB. Where did they come from? I tried different things - assigning other variables doesn't seem to free up this memory. Neither do calls to other functions. Except "plot()", which does seem to do the trick for some reason. Interestingly, when I run all this in a python interactive session (and not ipython), I get a similar memory usage initially. Calling a non-existent variable does not free up the memory, but other things do. For example, import matplotlib.pylab into the namespace did the trick. Does anyone have any idea what is going on? Thanks -- Ariel -- Ariel Rokem Helen Wills Neuroscience Institute University of California, Berkeley http://argentum.ucbso.berkeley.edu/ariel

3 3

Tomorrow's Skypecon: Proposed Agenda
by David Goldsmith June 30, 2009

June 30, 2009

Proposed Agenda Old Business: Timetable for approval of Ralf's patch for ticket #1146 New Business: Clarification of how much and what should be in the docstrings of methods which are equivalent to some function Please email me any additional items ASAP. (And your Skype ID if you haven't already and want to participate.) Also, thanks to all who chipped in this past week to get Polynomials to the "Goal Met (almost)" state; there are a few pending road blocks to completion, but generally, I'd say it was a success. Here are a few proposals for "Category of the Week" for the coming week: Financial Functions - Status: 50% done Leader: Skipper Seabold Random Number Generation - Status: 57% done Leader: Alan Jackson Linear Algebra - Status: 24% done Leader: None (yet) Feel free to cast a vote via email if you won't be able to join the Skypecon. Talk to you tomorrow, hopefully. David Goldsmith Technical Editor

1 0

[ANN] SciPy 2009 conference opened up for registration
by Gael Varoquaux June 30, 2009

June 30, 2009

We are finally opening the registration for the SciPy 2009 conference. It took us time, but the reason is that we made careful budget estimations to bring the registration cost down. We are very happy to announce that this year registration to the conference will be only $150, sprints $100, and students get half price! We made this effort because we hope it will open up the conference to more people, especially students that often have to finance this trip with little budget. As a consequence, however, catering at noon is not included. This does not mean that we are getting a reduced conference. Quite on the contrary, this year we have two keynote speakers. And what speakers: Peter Norvig and Jon Guyer! Peter Norvig is the director of research at Google and Jon Guyer is a research scientist at NIST, in the Thermodynamics and Kinetics Group, where he leads a fiPy, a finite element project in Python. The SciPy 2009 Conference ========================== SciPy 2009, the 8th Python in Science conference (http://conference.scipy.org), will be held from August 18-23, 2009 at Caltech in Pasadena, CA, USA. Each year SciPy attracts leading figures in research and scientific software development with Python from a wide range of scientific and engineering disciplines. The focus of the conference is both on scientific libraries and tools developed with Python and on scientific or engineering achievements using Python. Call for Papers ================ We welcome contributions from the industry as well as the academic world. Indeed, industrial research and development as well academic research face the challenge of mastering IT tools for exploration, modeling and analysis. We look forward to hearing your recent breakthroughs using Python! Please read the full call for papers (http://conference.scipy.org/call_for_papers). Important Dates ================ * Friday, June 26: Abstracts Due * Saturday, July 4: Announce accepted talks, post schedule * Friday, July 10: Early Registration ends * Tuesday-Wednesday, August 18-19: Tutorials * Thursday-Friday, August 20-21: Conference * Saturday-Sunday, August 22-23: Sprints * Friday, September 4: Papers for proceedings due The SciPy 2009 executive committee ----------------------------------- * Jarrod Millman, UC Berkeley, USA (Conference Chair) * Gaël Varoquaux, INRIA Saclay, France (Program Co-Chair) * Stéfan van der Walt, University of Stellenbosch, South Africa * (Program Co-Chair) * Fernando Pérez, UC Berkeley, USA (Tutorial Chair)

3 10

Last week's Skypecon Minutes and request
by David Goldsmith June 29, 2009

June 29, 2009

Hi, all. Below are the Minutes of last week's Summer Marathon Weekly Skypecon. Before that, you'll note in them that the participants decided to fix the Skypecon time to Wednesday's, 19:00 UTC. Well, I already need to request a "variance" for Wed. July 15 - anyone object strongly to Tuesday July 14, 18:00 UTC that week? Thanks! DG PS: Note that that's three Weds. from now - the next two, July 1 and July 8, the Skypecon will be at the designated time, 19:00 UTC; please email me any items you'd like to see on the agenda. And now, without further ado, I give you... the Minutes! Numpy Doc Summer '09 Marathon June 24, 2009 Skypecon Minutes Start: 19:00 UTC End: 20:00 UTC Present: David Goldsmith, Secretary Stefan van der Walt Ralf Gommers Pauli Virtanen Discussions: 0) Clarifying the Doc Standard vis-à-vis classes and their constructors, attributes, instances (created by numpy), and constants Agreed that, for now at least, c-tors will be documented in the class docstring Agreed that, because we might want to group attributes into meaningful categories within their class, we’ll stick with manual generation and annotation of such lists of attributes Instances: Pauli explained how things work “behind the scenes,” and it was decided that the way it's currently done is okay. For what that way is, see patch attached to ticket #1146, http://projects.scipy.org/numpy/ticket/1146 Constants: At first, said it’s moot because we can’t easily modify constants’ docstrings, but then Pauli pointed out that the development version of NumPy has the subpackage numpy.doc.constants, wherein constants can be documented at module scope. Did not settle on exactly how it should look but were in agreement that it should be a) kept short, and b) use the same sections as for functions where applicable. 1) Two "pre-proof" reviews: technical (“Is it correct?”) and presentation (“Is it clear and as simple as possible, modulo being correct?”) Note: the presentation issue is seen as distinct from whether or not the docstring is grammatically correct and free of typos, these latter to be assured by the final proof Discussed the “pros” (our labor pool, in general, consists of persons who are stronger in one area or the other, but not both; dividing the review process in this way makes it more manageable and thus less intimidating for prospective reviewers) and “cons” (compounding and complicating the overall process in this way will just push back completion that much further); decided that the pros outweigh the cons, and thus this division should be added to the review process, but, recognizing that these reviews are beyond the stated goal of the Summer Marathon, decided that required changes to the Wiki can be designated as “medium” priority. It came up after the meeting that there is still confusion about the meaning of the proposal, and that if the “non-technical” review is as described above, then at least one meeting participants opted to change his “vote,” resulting in a “tie” regarding “endorsement” of the proposal; we await clarification from the proposer regarding his intended meaning. 2) Adding a FAQ page to the Wiki Discussed how recent questions/issues arising on the mailing list and in the Comments sections of the Wiki demonstrate the need for the Wiki to have something like a FAQ page. Pauli agreed to create a “mock-up” for dissemination and discussion, which he has already completed and published at http://docs.scipy.org/numpy/Questions+Answers/ 3) Meeting time Decided that 19:00 UTC Wednesdays is a good time, and that, all else equal, it's probably better to fix the time than have it "float." Consequently, next Summer Marathon Skypecon is scheduled for: Wednesday, July 1, 19:00 UTC Notes: Ralf noted an inconsistency on the Milestones page between the Goal statement and the color scheme, which the Secretary has corrected Again, had periodic dropout problems all around.

1 0

ANN: SciPy 2009 student sponsorship
by Jarrod Millman June 29, 2009

June 29, 2009

I am pleased to announce that the Python Software Foundation is sponsoring 10 students' travel, registration, and accommodation for the SciPy 2009 conference (Aug. 18-23). The focus of the conference is both on scientific libraries and tools developed with Python and on scientific or engineering achievements using Python. If you're in college or a graduate program, please check out the details here: http://conference.scipy.org/student-funding About the conference -------------------- SciPy 2009, the 8th Python in Science conference, will be held from August 18-23, 2009 at Caltech in Pasadena, CA, USA. The conference starts with two days of tutorials to the scientific Python tools. There will be two tracks, one for introduction of the basic tools to beginners, and one for more advanced tools. The tutorials will be followed by two days of talks. Both days of talks will begin with a keynote address. The first day’s keynote will be given by Peter Norvig, the Director of Research at Google; while, the second keynote will be delivered by Jon Guyer, a Materials Scientist in the Thermodynamics and Kinetics Group at NIST. The program committee will select the remaining talks from submissions to our call for papers. All selected talks will be included in our conference proceedings edited by the program committee. After the talks each day we will provide several rooms for impromptu birds of a feather discussions. Finally, the last two days of the conference will be used for a number of coding sprints on the major software projects in our community. For the 8th consecutive year, the conference will bring together the developers and users of the open source software stack for scientific computing with Python. Attendees have the opportunity to review the available tools and how they apply to specific problems. By providing a forum for developers to share their Python expertise with the wider commercial, academic, and research communities, this conference fosters collaboration and facilitates the sharing of software components, techniques, and a vision for high level language use in scientific computing. For further information, please visit the conference homepage: http://conference.scipy.org. Important Dates --------------- * Friday, July 3: Abstracts Due * Friday, July 10: Announce accepted talks, post schedule * Friday, July 10: Early Registration ends * Tuesday-Wednesday, August 18-19: Tutorials * Thursday-Friday, August 20-21: Conference * Saturday-Sunday, August 22-23: Sprints * Friday, September 4: Papers for proceedings due Executive Committee ------------------- * Jarrod Millman, UC Berkeley, USA (Conference Chair) * Gaël Varoquaux, INRIA Saclay, France (Program Co-Chair) * Stéfan van der Walt, University of Stellenbosch, South Africa (Program Co-Chair) * Fernando Pérez, UC Berkeley, USA (Tutorial Chair)

1 0