[issue16099] robotparser doesn't support request rate and crawl delay parameters

[Nikolay Bogoychev]

Nikolay Bogoychev added the comment:

Updated patch, all comments addressed, sorry for the 6 months delay. Please review

----------
Added file: http://bugs.python.org/file35377/robotparser_v3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16099>
_______________________________________