[issue1437699] allow unicode arguments for robotparser.can_fetch
Terry J. Reedy
report at bugs.python.org
Thu Aug 26 20:58:52 CEST 2010
Terry J. Reedy <tjreedy at udel.edu> added the comment:
While Python is 'GPL compatible', whatever that means, it cannot incorporate GPLed code in the PSF distribution. Code must be contributed under one on the two licenses in the contributor agreement. Philip, can you contribute a patch appropriate to 3.x?
In 3.x, robotparser is urllib.robotparser. Under the 'be generous what you accept' principle, expansion of accepted names would seem to be good.
DOC PATCH NEEDED The doc says "For more details on the structure of robots.txt files, see http://www.robotstxt.org/orig.html ."
That link seems not to exist. The safest link is to the site. The specific replacement is http://www.robotstxt.org/robotstxt.html .
----------
stage: -> unit test needed
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1437699>
_______________________________________
More information about the Python-bugs-list
mailing list