Unsung Python modules

[Tim Hammerquist]

Tim Hammerquist <tim at vegeta.ath.cx> graced us by uttering:
> Skip Montanaro <skip at pobox.com> graced us by uttering:
>> I agree on the readline module (don't have a lot of need to text
>> file processing that sys.stdin.readlines() won't handle).
> 
> ... Sure, sys.stdin.readlines() will do until I have to parse a
> daemon log or flat text db; then there's no good reason or even
> advantage to loading the whole file in memory at once.

I found an excellent example of this in the course of almost crashing
my computer:

Using kpackage I tried to get the list of files included in the
mozilla-0.9.5 rpm that came with my Mandrake 7.2 distro.  There were
a lot of files. Not to put too find a point on it, but if I hadn't
doubled the recommended size of my swap partition, kpackage would've
crashed the machine!

This is no big deal for the command-line rpm because it iterates over
the filenames as it reaches them, and doesn't have to store them a
single, huge memory glob.

...which is why:

    for x in fileobject.xreadlines():
        ...

...is more scalable than:

    for x in fileobject.readlines():
        ...

...and isn't scalability the buzzword that everyone drops when
promoting Python? =)

Tim Hammerquist
-- 
The biggest problem with communication is the illusion that it has occurred.