[Python-Dev] python.org mail unclogged

Guido van Rossum guido@beopen.com
Tue, 27 Jun 2000 16:36:57 -0500


It seems we've removed a major source of email cloggage on python.org
-- a site in France was hitting every page in the mailman membership
administration using a commercial crawler (Teleport Pro) that didn't
honor our robots.txt file, and forking a 4 Mb CGI script for each user
in each mailing list.  Thanks for Barry and Andrew!  This was on top
of problems caused when the Ultraseek crawler ran out of memory and
caused postfix or mailman to fail. :-(

Mail seems to be flowing again, the queues are emptying.  Once we're
happy, we'll turn on the Ultraseek crawler and search service.

--Guido van Rossum (home page: http://www.python.org/~guido/)