mdp-toolkit

Download

mdp-toolkit@python.org

July 2017

4 participants
1 discussions

SVD-based alternative to solving the generalized EV problem (SFA nodes)
by Fabian Schoenfeld 11 Aug '17

11 Aug '17

Hi everyone, since everyone keeps running into the SFANode._stop_training() method's problem of singular matrices, I looked into how we can use SVD as an alternative. Here's the code snippet to replace said method (/nodes/sfa_nodes.py): _______________________________________________________________________________ def _stop_training(self, debug=False): ##### request the covariance matrices and clean up self.cov_mtx, self.avg, self.tlen = self._cov_mtx.fix() del self._cov_mtx # do not center around the mean: # we want the second moment matrix (centered about 0) and # not the second central moment matrix (centered about the mean), i.e. # the covariance matrix self.dcov_mtx, self.davg, self.dtlen = self._dcov_mtx.fix(center=False) del self._dcov_mtx rng = self._set_range() """ ######################################################################## OLD #### solve the generalized eigenvalue problem # the eigenvalues are already ordered in ascending order try: self.d, self.sf = self._symeig(self.dcov_mtx, self.cov_mtx, range=rng, overwrite=(not debug)) d = self.d # check that we get only *positive* eigenvalues if d.min() < 0: err_msg = ("Got negative eigenvalues: %s." " You may either set output_dim to be smaller," " or prepend the SFANode with a PCANode(reduce=True)" " or PCANode(svd=True)"% str(d)) raise NodeException(err_msg) except SymeigException, exception: errstr = str(exception)+"\n Covariance matrices may be singular." raise NodeException(errstr) ######################################################################## OLD """ ######################################################################## NEW U, s, _ = np.linalg.svd(self.cov_mtx) X1 = np.dot( U, np.diag(1.0/s**0.5) ) X2, _, _ = np.linalg.svd( np.dot(X1.T, np.dot(self.dcov_mtx,X1)) ) E = np.dot( X1, X2 ) e = np.dot( E.T, np.dot(self.dcov_mtx,E) ).diagonal() if rng is None: self.d, self.sf = e, E else: self.d, self.sf = e[rng[0]-1:rng[1]], E[:,rng[0]-1:rng[1]] ######################################################################## NEW """ _______________________________________________________________________________ As you can see the code contains both the new and the old version for easy testing. The how and why of the math behind this is based on this article: http://www.geo.tuwien.ac.at/downloads/tm/svd.pdf The relevant part for the implementation is found on p.15 under the final heading of the article. So far I tested it with a couple of examples and it seems to work out fine. If you have the chance to run a few tests with this, any feedback would be appreciated! If you feel this should be part of the mdp node we can add it as a new option (default: use the current EV solution). Stefan Richthofer also mentioned there might be other instances where this substitution might be useful, in which case we can also think about something like global switch to use the SVD-based EV solver. Let me know what you think! Cheers, Fabian

6 13