[Numpy-discussion] svd error checking vs. speed

[Sturla Molden]

Sturla Molden <sturla.molden at gmail.com> wrote:
> <josef.pktd at gmail.com> wrote:
>  maybe -1
>> 
>> statsmodels is using np.linalg.pinv which uses svd
>> I never ran heard of any crash (*), and the only time I compared with
>> scipy I didn't like the slowdown.
> 
> If you did care about speed in least-sqares fitting you would not call QR
> or SVD directly, but use the builting LAPACK least-squares drivers (*GELSS,
> *GELS, *GGGLM), which are much faster (I have checked), as well as use an
> optimized multi-core efficient LAPACK (e.g. Intel MKL). Any overhead from
> finiteness checking will be tiny compared to the Python/NumPy overhead your
> statsmodels code incurs, not to mention the overhead you get from using f2c
> lapack_lite.

By the way: I am not saying you should call this methods. Keeping most of
the QR and SVD least-squares solvers in Python has its merits as well, e.g.
for clarity. But if you do, it defeats any argument that finiteness
checking before calling LAPACK will be too slow. 

Sturla