[Tutor] Compute data usage from log

[Christian Witts]

bibi midi wrote:
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
>
> '''
> Calculate internet data consumption
> of service provider
> Code as per help of python tutor mailing list
> Created: 26-Oct-2009
>
> '''
>
> intro = raw_input('press enter to view your internet data consumption: ')
> log_file = '/home/bboymen/mobily.data.plan'
> total_consumed = 0
> for line in open(log_file):
>     total_consumed += int(line.split(' ')[9])
>
>
> total_consumed = total_consumed/1000000.0
> print "total data consumed is: %0.3f MB\n" % total_consumed
>
>
> #TODO: 
> #1) show the list comprehension alternative method
> #2) add exception handling e.g. when log_file cant be found or when key interrupt is pressed
>
> #      or when index[9] is not a number, etc
> #3) print latest date of calculated data
>   
>
> I'm working on TODO no. 3 e.g. I want to show the latest date when 
> wvdial generated the ppp data. This is normally the date of last line 
> of the ppd:
>
> Oct 14 11:03:45 cc000002695 pppd[3092]: Sent 3489538 bytes, received 
> 43317854 bytes.
> ^^^^^^^^^
>
> For the exception handling i *think* i just use the general exception 
> method e.g. will catch all kinds of error. I really dont know what 
> other errors will show up aside from the ones i listed in the TODO. 
> Advise is appreciated.
>
>
>
>
>
> On Mon, Oct 26, 2009 at 2:12 PM, Luke Paireepinart 
> <rabidpoobear at gmail.com <mailto:rabidpoobear at gmail.com>> wrote:
>
>
>
>     On Mon, Oct 26, 2009 at 3:20 AM, Christian Witts
>     <cwitts at compuscan.co.za <mailto:cwitts at compuscan.co.za>> wrote:
>
>         fInput = open('/path/to/log.file', 'rb')
>         total_usage = 0
>         for line in fInput:
>           total_usage += int(line.split(' ')[9].strip())
>         print total_usage
>
>
>     It's actually bad to assign a variable to the file object in this
>     case (flinput = ....) because Python will automatically close a
>     file after you're done with it if you iterate over it directly,
>     but if you include a reference it will stay open until the python
>     program ends or you explicitly call flinput.close().  It doesn't
>     matter much in this example but in general it is good practice to
>     either
>     1) call foo.close() immediately after you're done using a file
>     object, or
>     2) don't alias the file object and just over it directly so Python
>     will auto-close it.
>
>     Therefore a better (and simpler) way to do the above would be:
>
>     total_usage = 0
>     for line in open('/path/to/log.file'):
>         total_usage += int(line.split(' ')[9])
>
>     Also note you don't need to strip the input because int() coersion
>     ignores whitespace anyway. And additionally you shouldn't be
>     opening this in binary mode unless you're sure you want to, and
>     I'm guessing the log file is ascii so there's no need for the
>     'rb'.  (reading is default so we don't specify an 'r'.)
>
>
>     And since I like list comprehensions a lot, I'd probably do it
>     like this instead:
>
>     total_usage = sum([int(line.split(' ')[9]) for line in
>     open('/path/to/log.file')])
>
>     Which incidentally is even shorter, but may be less readable if
>     you don't use list comprehensions often.
>
>     Also, the list comprehension version is likely to be more
>     efficient, both because of the use of sum rather than repeated
>     addition (sum is implemented in C) and because list comprehensions
>     in general are a tad faster than explicit iteration, if i recall
>     correctly (don't hold me to that though, I may be wrong.)
>
>
>         Of course this has no error checking and or niceties, but I
>         will leave that up to you.
>
>     The same applies to my modifications.
>
>     Good luck, and let us know if you need anything else!
>
>     -Luke
>
>
>
>
> -- 
> Best Regards,
> bibimidi
>
>

Exceptions:
* Not finding the log file would be IOError.
* Casting an alphanumeric or alpha string to integer would be a 
ValueError, in this context you won't have a None so you shouldn't need 
to worry about a TypeError
* Selecting the 10th element in your list can raise an IndexError if 
your line did not contain enough delimiters to create a large enough list.

Pedantic:
1MB = 1,024KB = 1,024*1,024B
So your total consumed should be div (1024*1024.0) or div 1048576.0

For the date you can look at the time module to get a nice string 
representation of the date/time.  Or as you said you want the last date 
listed in the log file then you could add something like

for line in open(log_file):
    last_log_date = ' '.join(line.split(' ')[:3]

which would take the first 3 elements in your list and combine them 
again.  Of course this is again just a simple representation of what to do.

-- 
Kind Regards,
Christian Witts