Mailman 3 February 2018 - SciPy-Dev

Use of approx_derivative (rel_step) vs approx_fprime (absolute step) in optimize
by Andrew Nelson 15 Feb '18

15 Feb '18

The default for bfgs, lbfgs, and cg scalar function minimizers in optimize is to use the same absolute step size, `sqrt(numpy.finfo(float).eps)` for all values in `x` when estimating the gradient. One can provide an array of step sizes, but I guess that the large majority of people would just use the default. The absolute step value is used by `approx_fprime` to work out the gradient. In contrast `approx_derivative` in optimize/_numdiff.py uses relative steps for all values in `x`. For values in `x` that could encompass values of different magnitudes, including those down to, or below, the default absolute step size, the latter approach of using relative steps seems more robust. Q1 - I'm wondering why the scalar minimizers don't use relative steps in calculating the gradient? Q2 - Might a better default be to use a relative step, reverting to absolute step if requested? This behaviour may not surprise those familiar with these optimizers operate, but I can guarantee that this behaviour will trip up the unwary (i.e. me) just looking to minimize a function, not knowing that the step could be close to a given value in x, and suffer crazy gradients as a result.

1 0

cython_optimize.zeros concept
by Mark Mikofski 15 Feb '18

15 Feb '18

Hi, I need to run a loop around the scipy.optimize.zeros functions like brentq and newton. Looping 100k takes a long time, so I started to try to cythonize some of the zeros functions. I've started with newton and bisect. Newton works, but bisect gives me "Illegal Instruction 4" on my mac running Python 2.7 and Xcode. If anyone has any feedback that would help a lot. https://github.com/mikofski/scipy/tree/cython_optimize/scipy/optimize/cytho… Thanks! “As I breath in, I calm my body, as I breath out, I smile” - Thich_Nhat_Hanh

1 0

Hello and thank you!
by Mark Mikofski 14 Feb '18

14 Feb '18

Hi everyone, my name is Mark Mikofski. I am a mechanical engineer, and have been modeling energy transfer for over 15 years. You can learn more about me at https://mikofski.github.io . I wanted to thank you all very much for your contributions to SciPy, and for allowing me to join this mailing list. From reading the posts I can tell this is a vibrant group of creative individuals who share a positive and respectful curiosity for science, math, and numerical computation. I will try to make equally positive and respectful contributions as well. Best regards! Sent from Yahoo Mail on Android

1 0

ENH: Extend peak finding capabilities in scipy.signal (#8264)
by Lars G. 14 Feb '18

14 Feb '18

Dear SciPy devs, I'd like to highlight my current pull request which extends SciPy's peak finding capabilities in the signal module. https://github.com/scipy/scipy/pull/8264 I've tried to address all feedback that was given (until now) in said PR. However there are still some areas that would profit from a code review and an additional pair of eyes. Furthermore there are still some outstanding issues. It may be best to discuss them here: - There are still "Changes requested" which I have addressed or are outdated. I don't know how to go about removing this flag. - I think the Travis CI build fail is unrelated to my PR (something in the stats module). Perhaps someone with more knowledge should confirm this. - How to handle if invalid peaks are passed to the functions peak_prominences and peak_widths? Is it okay to describe this as undefined behavior in the documentation? - argrelmax doesn't catch peaks that are wider than one sample. Decide how to deal with this. I have implemented an alternative here which may even be faster. But I plan to make a performance comparison to confirm this. I have a feeling this should be addressed (if wanted) in a new pull request. - Should the new functionality be covered in the tutorial for scipy.signal? I feel like that would be the best place to show more complex examples which are out of place in the docstrings itself. If so, I think a new PR would be the best place. - The decision whether find_peaks should have a postfix like "find_peaks_cwt" is still pending. If so which one? Again, thank you all in advance for taking the time to review this PR and any feedback given. Best regards, Lars

4 10

Help for scalar univariate zeros functions
by Mark Mikofski 10 Feb '18

10 Feb '18

Hi, I model photovoltaics (PV) for solar energy rooftop and utility systems, and I also contribute to a Sandia National Labs sponsored, open-source PV modeling Python package called pvlib. I need help with the scalar, univariate zeros functions in scipy.optimize. 1st a little background: The main governing equation in PV is Shockley's diode equation, which is implicit for either current or voltage. For a one diode model of a solar cell, we can use scipy.special.lambertw, but for a two diode model, we need a scalar, univariate root finding method like scipy.optimize.newton or scipy.optimize.brentq. In fact we can also use these methods for the one diode model in lieu of Lambert-W transform, and it is very fast for a single calculation. However, I often have to calculate thousands if not millions of these solar cell models, and looping over either Newton or brentq is slow. I tried using multi-processing and a multivariate root method like hybrj, hybrd, and Nelder-Mead, but they are just as slow. I've also tried cythonizing the Newton method, and it does run faster. But the LambertW function in scipy.special can take numpy arrays as arguments, and since scipy.optimize.newton is pure Python it is easy to edit it to also take numpy arrays, which runs faster than any other option by 10x. I think this is because doing each operation in the objective function on the entire array optimizes the use of cpu cache and simd registers, maybe? The pvlib maintainers and I agree, we would rather use a scipy method than a custom one so we are hoping that someone will take an interest in PR #8357. I'm also working on a cython optimize branch but I haven't had a chance to submit another PR yet. If you are interested in either of there's branches, I urge you to respond with your comments on the mailing list, on the PR, or issue #8354. Thanks! Best regards, Mark Mikofski, principal Engineer, DNV-GL, solar services

1 0

lgmres tolerance
by marc 09 Feb '18

09 Feb '18

Dear Scipy developers, I have a question regarding the tolerance of the lgmres function. From the documentation, it is written, *tol* : float, optional Tolerance to achieve. The algorithm terminates when either the relative or the absolute residual is below /tol/. For technical issues, I would like the algorithm to ignore the relative residual and to terminates only when the absolute residual is below tol. Do there is a way to do this? Best regards, Marc Barbry

2 1

Re: [SciPy-Dev] Proposal - optimize callback given a state keyword
by Antonio Ribeiro 09 Feb '18

09 Feb '18

Hi All, I have a few comments regarding the newcallback: 1. I have started a PR changing this: https://github.com/scipy/scipy/pull/7425 <https://github.com/scipy/scipy/pull/7425>. By what I remember you are helping with it as well Andrew. I never finished because I gave priority to other projects, but I may go back to work on it after finishing PR 8328. 2. By what I remember the methods that could easily benefit from the modification are: Powell, Nelder-Mead, CG, BFGS, Newton-CG, trust-ncg, trust-exact, dogleg. 3. I think we should allow the termination of the optimization solver by returning `True` (or False) in the callback function. 4. For the solver me and Nikolay are working right now (https://github.com/scipy/scipy/pull/8328) this option is already implemented. Please take a look at the new `minimize` documentation after the modifications we have introduced: https://5466-1460385-gh.circle-artifacts.com/0/home/ubuntu/scipy/doc/build/… <https://5466-1460385-gh.circle-artifacts.com/0/home/ubuntu/scipy/doc/build/…> Best, Antônio > On Feb 8, 2018, at 15:00, scipy-dev-request(a)python.org wrote: > > Send SciPy-Dev mailing list submissions to > scipy-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-dev > or, via email, send a message with subject or body 'help' to > scipy-dev-request(a)python.org > > You can reach the person managing the list at > scipy-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-Dev digest..." > > > Today's Topics: > > 1. Proposal - optimize callback given a state keyword (Andrew Nelson) > 2. Re: Proposal - optimize callback given a state keyword > (Jaime Fern?ndez del R?o) > 3. Re: Proposal - optimize callback given a state keyword > (Joshua Wilson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 8 Feb 2018 13:06:19 +1100 > From: Andrew Nelson <andyfaff(a)gmail.com> > To: scipy-dev <scipy-dev(a)python.org> > Subject: [SciPy-Dev] Proposal - optimize callback given a state > keyword > Message-ID: > <CAAbtOZeufOk-gn7yB8565riRd30m+ReJ+_jGGPYBGngLVsj-Zg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi all, > I propose that the optimize callbacks be asked to accept a new state > keyword in the future. > The state keyword will be an `OptimizeResult` giving the state of the > optimizer at that point. > i.e. n_iterations, n_fun, (n_jac?), x, fun, etc. > > To aid the transition I propose that the user callbacks are wrapped with > something along the lines of: > > class OptimizerCallback(object): > def __init__(self, callback): > self.callback = callback > argspec = _getargspec(callback) > self.accepts_state = True > if 'state' not in argspec.varkw: > warnings.warn(FutureWarning, "In future versions of scipy the" > " optimizer > callback should accept" > "a state kwd") > self.accepts_state = False > > def __call__(self, x, state): > if self.accepts_state: > self.callback(x, state) > else: > self.callback(x) > > and the optimizers be modified appropriately. > > Thoughts, comments? > > > > _____________________________________ > Dr. Andrew Nelson > > > _____________________________________ >

3 4

Re: [SciPy-Dev] CoC incidents
by Nathaniel Smith 08 Feb '18

08 Feb '18

Please send messages like this to the CoC committee address at scipy-conduct(a)googlegroups.com, not scipy-dev. On Feb 8, 2018 14:48, "xoviat" <xoviat(a)gmail.com> wrote: The purpose of this email is to give an email address such that I can respond to any Code of Conduct incidents that may arise. I am open to discuss this in public, but will be more than happy to discuss issues in private if asked. _______________________________________________ SciPy-Dev mailing list SciPy-Dev(a)python.org https://mail.python.org/mailman/listinfo/scipy-dev

1 0

CoC incidents
by xoviat 08 Feb '18

08 Feb '18

The purpose of this email is to give an email address such that I can respond to any Code of Conduct incidents that may arise. I am open to discuss this in public, but will be more than happy to discuss issues in private if asked.

1 0

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 172, Issue 7
by Matt Newville 08 Feb '18

08 Feb '18

Hi Andrew, All, Apologies for replying to the digest.... On Thu, Feb 8, 2018 at 11:00 AM, <scipy-dev-request(a)python.org> wrote: > > > Message: 1 > Date: Thu, 8 Feb 2018 13:06:19 +1100 > From: Andrew Nelson <andyfaff(a)gmail.com> > To: scipy-dev <scipy-dev(a)python.org> > Subject: [SciPy-Dev] Proposal - optimize callback given a state > keyword > Message-ID: > <CAAbtOZeufOk-gn7yB8565riRd30m+ReJ+_jGGPYBGngLVsj-Zg(a)mail.gmail. > com> > Content-Type: text/plain; charset="utf-8" > > Hi all, > I propose that the optimize callbacks be asked to accept a new state > keyword in the future. > The state keyword will be an `OptimizeResult` giving the state of the > optimizer at that point. > i.e. n_iterations, n_fun, (n_jac?), x, fun, etc. > Are you sure you want an OptimizeResult? Maybe it would be sufficient and clearer to have the callback given "the current iteration number" (a somewhat fuzzy concept, but useful nonetheless) and "the current set of values for the variables", and maybe "the current value of the residual"? Is there more that is actually reliable while the fit is proceeding and that a callback could use? How would you expect a callback function to use the current value of Jacobian, for example? --Matt Newville

1 0