urllib2 - iteration over non-sequence

Erik Max Francis max at alcyone.com
Sun Jun 10 07:56:38 CEST 2007

Paul Rubin wrote:

> If you know in advance that the page you're retrieving will be
> reasonable in size, then using readlines is fine.  If you don't know
> in advance what you're retrieving (e.g. you're working on a crawler)
> you have to assume that you'll hit some very large pages with
> difficult construction.

And that's before you even mention the point that, depending on the 
application, it could easily open yourself up to a DOS attack.

There's premature optimization, and then there's premature completely 
obvious and pointless waste.  This falls in the latter category.

Besides, someone was asking for/needing an older equivalent to iterating 
over a file.  That's obviously .xreadlines, not .readlines.

Erik Max Francis && max at alcyone.com && http://www.alcyone.com/max/
  San Jose, CA, USA && 37 20 N 121 53 W && AIM, Y!M erikmaxfrancis
   The more violent the love, the more violent the anger.
    -- _Burmese Proverbs_ (tr. Hla Pe)

More information about the Python-list mailing list