[Numpy-discussion] automatic differentiation with PyAutoDiff

James Bergstra bergstrj at iro.umontreal.ca
Thu Jun 14 18:52:58 EDT 2012

On Thu, Jun 14, 2012 at 5:53 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Thu, Jun 14, 2012 at 9:22 PM, srean <srean.list at gmail.com> wrote:
> No, I'm saying I totally see the advantages. Here's the code I'm talking about:
>    def _loglik(self, params):
>        alpha, beta = self.used_alpha_beta(params)
>        if np.any(alpha < 0):
>            return 1e20
>        total = 0
>        for group in self._model._groups.itervalues():
>            alpha_part = np.dot(group["alpha_matrix"], alpha)
>            eff_beta_matrix = group["beta_matrix"].copy()
>            nab = self._model._num_alpha_betas
>            eff_beta_matrix[:, :nab] *= np.log(alpha_part[:, np.newaxis])
>            exponent = np.dot(eff_beta_matrix, beta)
>            Z = np.exp(exponent).sum()
>            total += (group["counts"] * exponent).sum()
>            total += group["counts"].sum() * -np.log(Z)
>        return total

You're right, this is totally the kind of code that autodiff
can/should be able to help with.

I just pushed a first draft at support for np.any, log, exp, inplace
operators, and inplace array assignment... so there's a chance that
your example might currently run. (You might even see a speedup if
Theano graph optimizations work their magic). It's not clear from the
code fragment what the various types in play are (see previous rant on
static analysis!), but an autodiff PR with a test case would help sort
out any remaining problems if you want to follow up on this.

- James

More information about the NumPy-Discussion mailing list