[scikit-learn] [GSoC 2017] First Draft, request for suggestions - Improve Online Learning of Linear Models.

[Karan Desai]

> The problem with callbacks is that for callbacks on each iteration to be
feasible, they need to be cython functions.> Otherwise they will be too slow.
You could do python callbacks, but they could not be called at every iteration,
and so
> they wouldn't be suitable to implement something like adagrad or adam.

We can implement some plug and play callbacks in Cython and pass a list of
strings in constructor of a linear model, deciding which callbacks to execute.
How does that sound Andreas ? I can really help with more thoughts about the
idea.
Regards,Karan.  





On Wed, Mar 15, 2017 8:18 PM, Andreas Mueller t3kcit at gmail.com  wrote:






On 03/15/2017 04:48 AM, Karan Desai wrote:

> 4. About a tool to anneal learning rate: I suggest a new approach to > look at
this - as a callback. I searched through the documentation and > I could not
find this way of handling tidbits during training of > models. We should be able
to provide a callback to the constructor of > a linear model which can do any
dedicated job after every epoch, be it > learning rate annealing, saving model
checkpoint, getting custom > verbose output, or as creative as uploading data to
server for real > time plots on any website.

There has been some effort on doing adagrad but it was ultimately discontinued,
I think.

There was quite a bit of complexity to handle.

The problem with callbacks is that for callbacks on each iteration to be
feasible, they need to be cython functions.

Otherwise they will be too slow. You could do python callbacks, but they could
not be called at every iteration, and so

they wouldn't be suitable to implement something like adagrad or adam.




Best,

Andy

_______________________________________________

scikit-learn mailing list

scikit-learn at python.org

https://mail.python.org/mailman/listinfo/scikit-learn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170316/b3f35d98/attachment.html>