[Tutor] Downloading S3 Logs
laomao1975 at googlemail.com
Thu Feb 11 17:23:20 CET 2010
I've written the below to get the previous day's logs from an Amazon S3
from datetime import datetime
daily_s3_log = open("/tmp/s3logs", "w+")
now = datetime.now()
connection = boto.connect_s3()
bucket = connection.get_bucket("downloads.sekrit.co.uk")
todays_keys = 
for key in bucket:
time_difference = (now -
if time_difference < 1 and key.name.startswith("log/access_log"):
for key in todays_keys:
This takes about 2 mins to download a day's logs (about 25M).
I'd appreciate any improvements or feedback on the above.
For example, would it make sense to make the first loop into a generator
function that yields the interesting keys? Also is there a cleaner way to
do the date comparison in Python 2.4?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Tutor