Ive implemented the classical MultiDimensional Scaling for the scikit learn using both functions. Their behavior surprised me for "big" arrays (10000 by 10000, symmetric as it is a similarity matrix). linalg.svd() raises a memory error because it tries to allocate a (7000000,) array (in fact bigger than that !). This is strange because the test was made on a 64bits Linux, so memory should not have been a problem. linalg.eigh() fails to diagonalize the matrix, it gives me NaN as a result, and this is not very useful. A direct optimization of the underlying cost function can give me an adequate solution.
I cannot attach the matrix file (more than 700MB when pickled), but if anyone has a clue, I'll be glad.