Wow, this is great! Thanks for all the great questions. Sebastian Walter <sebastian.walter <at> gmail.com> writes:
is it really the covariance matrix you want to invert? Or do you want to compute something like x^T C^{-1} x, where x is an array of size N and C an array of size (N,N)?
Yes, this is what I am computing. I am computing the pdf of a very high- dimensional multivariate normal. Is there a specialized method to compute this?
It would also be interesting to know how the covariance matrix gets computed and what its condition number is, at least approximately.
Chuck and Sebastian are on to me. I should look into computing this using the small covariance matrix. I suppose I could even postpone evaluation of this covariance matrix by passing in the data matrix instead of the covariance matrix. What do you guys think? As for the condition number, the matrix will, initially, be ill-conditioned because the dimension of the data will far outnumber the number of data points. However, I am writing this code on the way to writing code that fits a mixture of Gaussians but can use any number of covariance regularization techniques to "fix" the covariance matrices. Basically, I am writing code that does this: http://www.cs.colostate.edu/~dane/papers/ANNIE2010.pdf. I am guessing you math and stats guys might shake your head at my clumsy, computer science ways... :) However, in addition to the two, simple regularization techniques described in that paper, I also want to be able to do things like mixture of factor analyzers and mixture of principal component analyzers and so on by simply supplying different regularization techniques. FYI, I am currently adding this to PyMix. Does this seem like a good package to extend?
On Wed, Sep 1, 2010 at 1:58 AM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:
On Tue, Aug 31, 2010 at 4:52 PM, Dan Elliott <danelliottster <at> gmail.com> wrote:
David Warde-Farley <dwf <at> cs.toronto.edu> writes:
On 2010-08-30, at 10:36 PM, Charles R Harris wrote: I think he means that if he needs both the determinant and to solve the system, it might be more efficient to do the SVD, obtain the determinant from the diagonal values, and obtain the solution by multiplying by U D^-1 V^T? Thank you, that is what I meant. Poorly worded on my part.
In particular, I am writing code to invert a very large covariance matrix. I think David has some good information in another post in this thread.
Where did the covariance array come from? It may be the case that you can use a much smaller one, for instance in PCA of images.
Chuck