<br><br><div><span class="gmail_quote">On 8/1/07, <b class="gmail_sendername">A.M. Kuchling</b> <<a href="mailto:amk@amk.ca">amk@amk.ca</a>> wrote:</span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Was the Wiki crawler using a consistent user agent that can be banned<br>(e.g. nutch, wget, etc.)?</blockquote><div><br>Nope, the user agent claimed to be MSIE 5:<br></div></div>"Mozilla/4.0 (compatible; MSIE 5.01; Windows NT
5.0)"<br><br>-- <br>Thomas Wouters <<a href="mailto:thomas@python.org">thomas@python.org</a>><br><br>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!