https://github.com/scipy/scipy/issues/18568

https://github.com/scipy/scipy/pull/18954

Update/rewrite sparse_linalg_lobpcg.py to make it more reasonable

raise a discussion if a benchmark needs to make a meaningful comparison and assert accuracy or just barely run the code for timing even if the code produces garbage and the comparison baseline is irrelevant.

In my view, a benchmark is a tool for a user to choose a better performing solver. If a benchmark is meaningless computing garbage because the solver is called in a wrong way or the comparison baseline is wrong, it fails this goal - the issue with the present benchmark.

 

It is also a tool for a developer to validate that changes in a code do not degrade performance. For iterative solvers, performance is determined by a total execution time, which is the time per iteration times the number of iterations needed to get the requested accuracy. The present benchmark does not check the accuracy, so fails this goal too, i.e., is quite useless and misleading. The property that for a known problem a code of an iterative solver gives a guaranteed accuracy in a known fixed number of iterations is desired but not given and must be checked and rechecked with every chance of the code.