[SciPy-Dev] Backwards Compatibility for low level LAPACK routines

[Ilhan Polat]

Due their historical evolution, there are certain LAPACK wrappers that are
not standardized. Some work with minimum lwork variables instead of their
optimal values. Also these routines often return quite big arrays during
the lwork queries, to demonstrate :

import scipy.linalg as la
la.lapack.dgeqrf(a=np.random.rand(400,400), lwork=-1)

is a workspace size query (via lwork=-1). The current default size is
"3*a.shape[0] + 1" hence 1201. However the optimal workspace size is 12800
on my machine. Therefore the mismatch is sometimes quite dramatic
especially in some other routines. Notice also that to obtain this number
the routine actually returns a 400-long array tau and requires the input
matrix to be transferred back and forth. Moreover, they can't be handled
via scipy.linalg.lapack._compute_lwork function.

There are a few routines like this and I feel like they should be fixed and
I'm willing to. However this means that their output signature is going to
change which imply backwards compatibility breaks. I tried to see whether
we could deprecate them with new wrappers with modified names, but to be
honest, that would create too many duplicates. On the other hand I don't
have a feeling of how much break this would mean out there in the wild.

Is this break an acceptable one or not? (well, none is acceptable
preferably, but in despair...)

Any other alternatives, thoughts are most welcome.

best,
ilhan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180802/e6b838e7/attachment.html>