On Wed, Apr 29, 2020 at 9:44 PM Andrew Nelson <andyfaff@gmail.com> wrote:
I sort of think these pros are overstated. The dispatching of what
function to call does not seem that difficult to do (either in `minimize` or in user code). The benefit of having that dispatch of function name happen within `minimize` is small. Normalizing the APIs so that the options sent to the underlying methods is harder and also more valuable. That is, in order for the dispatching to really be valuable, it has to unite and offer a translation layer to the calls to the underlying functions.
A recent issue (https://github.com/scipy/scipy/issues/11956) highlighted this issue. Here the author wanted to use a constraints dict with differential_evolution, similar to that can be provided to `minimize` (de uses the new style `NonLinearConstraint`). If differential_evolution was a `minimize` method that translation would be done automatically. The same argument applies for translation of new style and old style bounds. Is that what you mean by normalising the API?
It would be? Or: how would it be without having it also work with the function `differential_evolution`.
The global solvers have many different optional arguments with little overlap in name or meaning. Like, 'popsize' is only used by 'differential_evolution'. The plan would have to be to silently ignore keyword arguments for concepts not used by the currently used method. I'm not sure that helps achieve clarity and simplicity. To use these methods, the user has to read the docs for the actual solver to get the many optional arguments set anyway. At that point, they can just as easily change the name of the function.
There are many optional arguments for each of the methods as-is. The most common are jac and hess which are used across some, but not all methods. L-BFGS-B has `iprint`, `gtol`, `maxls` which aren't used by most other methods. Over supply/non-supply (i.e. concepts not used by specified method) of these keywords is already handled by minimize, and by the minimizers themselves (via `_check_unknown_options`). Your line of reasoning runs counter to the design of the `minimize` function, and would suggest a return to the old-style minimize functions: fmin, fmin_l_bfgs_b, etc. (The documentation states "The functions below are not recommended for use in new scripts; all of these methods are accessible via a newer, more consistent interfaces, provided by the interfaces above.")
Yes, you are correct that my line of reasoning runs counter to the `minimize` function. Well, I would say not so much "counter" as seeing some value with `minimize`, but also seeing some value with the old style too. I am definitely not in favor of the documentation that claims working functions are deprecated in favor of a multi-dispatch function. Somehow, `minimize` and the deprecation of `fmin`, etc grates on me much less than the alleged deprecation for `leastsq` in favor of `least_squares`, which is a nest of interdependent options. `Minimize` does not seem as bad (but it appears that you're working on it ;)). My point is that the dispatching of the function names is not, by itself, really that big of a win. Functions are first-class objects. At least with most of the current solvers covered by `minimize`, most of the non-uniformly-named options are also truly optional (except, for some methods, `jac` -- but at least that is more or less a uniform name) and "advanced options for fine tuning". My sense is that the uniquely named arguments to the global solvers are more important, and are less like "advanced options".
But, I think there is another concern that may not have been expressed
yet. `x0` is a required, positional argument for `minimize()`, as an array of initial parameter values. Most of the global optimizers in scipy.optimize do not use `x0`. Instead, they require bounds and explore the range of values between those bounds. Would `x0` be required AND ignored for these global optimizers?
The call signature for required positional arguments for `minimize` is different to the global optimizers. Being able to call the global minimizers via `minimize` would alleviate that. As you say the global minimizers do explore within bounds, and don't use an `x0`. The PR (as it currently exists) would still require `x0`, and it would be ignored.
If I understand correctly, with the proposed changes, I hope you would have to continue supporting minimize(objective, x0, method='Nelder-Mead') to work as it currently does: x0 required. To switch to solve with `differential_evolution`, the user would have to do: minimize(objective, x0, bounds=bounds, method='differential_evolution') Now, although `bounds` is a keyword argument, it is actually required for the method to work. And `x0` is a required positional argument, but the value is ignored. That seems profoundly weird to me. Are there other examples in scipy (outside of scipy.optimize) for which a) a required, positional argument has a value that is ignored when an optional keyword argument has some value(s)? b) a keyword argument is changed from optional to required due to the value of a different keyword argument? Each of these seems like a problem to me. And, yes, b) is currently the case for `minimize`: some values of `method` require a `jac` option, while other values for `method` do not use `jac`. So, yes, I think the idea of putting all of `scipy.optimize` into a single function is an understandable desire but also sort of a design mistake. If important keyword arguments are different and not translatable, I don't see why minimize(objective, x0, method='solver', **kws) is actually better than solver(objective, x0, **kws) For sure, having uniform keyword arguments and using common infrastructure to, say, having a common way of setting bounds and constraints is valuable. Or (ahem), make Parameters objects that have names and bounds, etc.... Or if there was a Minimizer class with attributes and methods, maybe having a `method` attribute would make sense. Either way, if you're looking to improve the uniformity or the ability of downstream code to use the functions as if they were an API, then OptimizerResult really ought to include the name of the method used. Cheers, --Matt