How to read such file and sumarize the data?

[Steve Holden]

On 11/17/2010 7:51 PM, Terry Reedy wrote:
> On 11/17/2010 6:10 PM, Steve Holden wrote:
> 
>> $ cat data.py
>> lines = open("data.txt").readlines()
> 
> Since you iterate through the file just once, there is no reason I can
> think of to make a complete in-memory copy. That would be a problem with
> a multi-gigabyte log file ;=). In 3.x at least, open files are line
> iterators and one would just need
> 
You are indeed perfectly correct. Thank you. Probably old-ingrained
habits showing through. Open files have been line iterators since 2.2, I
believe.

regards
 Steve

> lines = open("data.txt")
> 
>> from collections import defaultdict
>> c = defaultdict(int)
>> for line in lines:
>>      ls = line.split()
>>      if len(ls)>  3 and ls[3].startswith("NCPU="):
>>          amt = int(ls[3][5:])
>>          c[ls[0]] += amt
>> for key, value in c.items():
>>      print key, ":", value
>>
>>
>> $ python data.py
>> xyz : 4
>> tanhoi : 1
>> sabril : 6
>>
>> regards
>>   Steve
> 
> 


-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17       http://us.pycon.org/
See Python Video!       http://python.mirocommunity.org/
Holden Web LLC                 http://www.holdenweb.com/