[Baypiggies] reading files quickly and efficiently

[wesley chun]

ok, so i think you all got the thing working for the OP using other
libraries. so as we don't lose this opportunity to learn about various
ways to improve our Python code, let's take a look at the existing
source and see if we can further improve on it and/or make it more
"Pythonic:"

ORIGINAL:
f = open ('nr')
count = 0
for i in f.readlines():
    line = f.next().strip()
    count = count + 1
f.close()
print count

1. i know i'm a stickler (hey, *some*one's gotta do it), but i like to
put the 'r' in my open() statements. call it a habit, but it is what
it is.

2. you don't need a counter when you can sum() up the total number of
lines merely by iterating through it. i'll use a generator expression
to save memory.

3. strip()ping the leading and trailing whitespace adds no value to
your counting and makes things run slower and uses more CPU, so remove
that.

4. file.next() contradicts f.readlines() but is the right idea however
it's unnecessary as the for-loop automagically calls next() on your
behalf

5. f.readlines() reads in all the lines of a file, so you don't want
to do that as it will chew up all your memory. instead, iterate
through the file.

given the above, i think you can do everything you want in 3 lines of
readable Python code:

MODIFIED:
f = open ('nr', 'r')
print sum(1 for line in f)
f.close()

can anyone else improve on this? yeah, if you're wasteful and don't
properly close the file, you can reduce this down to a single line
(less desired/not recommended). since this is so bad, you might as
well remove the 'r' also:

print sum(1 for line in open('nr')) # very bad... don't try this at
home (or work!)

(this progression is pretty much what you would do if you were playing
"code golf" where engineers at a former employer of mine would try to
have the lowest character count.)

hope this helps!
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
"Core Python Programming", Prentice Hall, (c)2007,2001
"Python Fundamentals", Prentice Hall, (c)2009
    http://corepython.com

"Python Web Development with Django", Addison Wesley, (c) 2009
    http://withdjango.com

wesley.j.chun :: wescpy-at-gmail.com
python training and technical consulting
cyberweb.consulting : silicon valley, ca
http://cyberwebconsulting.com