[SciPy-Dev] WIP: Class based Optimizers

[Pauli Virtanen]

Hi,

ke, 2018-02-14 kello 14:21 +1100, Andrew Nelson kirjoitti:
> The #8414 PR on github (https://github.com/scipy/scipy/pull/8414)
> introduces an `Optimize` class for `scipy.optimize`. The `Optimize`
> class
> can be advanced like an iterator, or be advanced to convergence with
> the
> `solve` method.

I think it would be necessary to make clearly a case what is actually
gained with such reorganization. This would be the third optimizer API
to the same solvers, and at first sight it's not clear to me if there
is a strong need for it.

One point that one can argue is that

solver = ...
for foo in solver:
   some_code

is identical with regard to functionality to

def callback(.., solver_info):
    some_code
solver(..., callback=callback)

with the difference that the latter does not leak unnecessary private
implementation details to the user code, requires only small API
changes, and won't require extensive code reorganization.

The use cases where one needs to continuously monitor convergence in
minimization are also somewhat expert-oriented, and for most users I
expect such API will not be as useful. (The situation with ODE solvers
is I think somewhat different, as with them one e.g. often does not
even know the time up to which to integrate a priori.)

Writing class-based solvers requires rewriting code in a reverse-
communication style, which inverts the structure of control flow. This
can result to code that is more complex, and more difficult to
maintain. There are some tricks one can do to retain more natural
structure such as using generators/coroutines, but these are not always
so nice to use in practice in Python in my experience, and also bring
more complexity.

The existing solvers in Scipy apart from L-BFGS-B are IIRC not written
in a reverse-communication style, and converting them would be
significant rewrite, particularly so for the solvers written in
C/Fortran.

The same applies also for tuning solver parameters during the
iteration. The solvers AFAIK have not been designed with parameters
that can be adjusted during run, and one would probably have to think
carefully about each one (instead of e.g. exposing them as class
attributes).

So, I think there are several questions that one should consider here.

	Pauli