[Python-checkins] python/dist/src/Lib robotparser.py,1.17,1.18

rhettinger at users.sourceforge.net rhettinger at users.sourceforge.net
Sat Mar 13 15:27:26 EST 2004


Update of /cvsroot/python/python/dist/src/Lib
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30511/Lib

Modified Files:
	robotparser.py 
Log Message:
SF patch #911431: robot.txt must be robots.txt
(Contributed by George Yoshida.)



Index: robotparser.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/robotparser.py,v
retrieving revision 1.17
retrieving revision 1.18
diff -C2 -d -r1.17 -r1.18
*** robotparser.py	27 Feb 2003 20:14:40 -0000	1.17
--- robotparser.py	13 Mar 2004 20:27:22 -0000	1.18
***************
*** 84,88 ****
  
      def parse(self, lines):
!         """parse the input lines from a robot.txt file.
             We allow that a user-agent: line is not preceded by
             one or more blank lines."""
--- 84,88 ----
  
      def parse(self, lines):
!         """parse the input lines from a robots.txt file.
             We allow that a user-agent: line is not preceded by
             one or more blank lines."""
***************
*** 149,153 ****
      def can_fetch(self, useragent, url):
          """using the parsed robots.txt decide if useragent can fetch url"""
!         _debug("Checking robot.txt allowance for:\n  user agent: %s\n  url: %s" %
                 (useragent, url))
          if self.disallow_all:
--- 149,153 ----
      def can_fetch(self, useragent, url):
          """using the parsed robots.txt decide if useragent can fetch url"""
!         _debug("Checking robots.txt allowance for:\n  user agent: %s\n  url: %s" %
                 (useragent, url))
          if self.disallow_all:




More information about the Python-checkins mailing list