[Numpy-discussion] automatic differentiation with PyAutoDiff

Frédéric Bastien nouiz at nouiz.org
Thu Jun 14 17:37:38 EDT 2012


On Thu, Jun 14, 2012 at 4:49 PM, James Bergstra
<bergstrj at iro.umontreal.ca> wrote:
> You're right - there is definitely a difference between a correct
> gradient and a gradient is both correct and fast to compute.
> The current quick implementation of pyautodiff is naive in this
> regard.  However, it is delegating the heavy lifting to Theano. Theano
> performs the sort of optimization-oriented tree manipulations you're
> talking about, and Theano contributors have been tweaking them for a
> few years now. They aren't always perfect but they are often pretty
> good. In fact, I would go as far as saying that they are often
> *better* than what you might do by hand if you don't sit down for a
> long time and tune the hell out of your computation.


I second James here, Theano do many of those optimizations. Only
advanced coder can do better then Theano in most case, but that will
take them much more time. If you find some optimization that you do
and Theano don't, tell us. We want to add them :)


More information about the NumPy-Discussion mailing list