On Thu, Feb 11, 2010 at 11:23 AM, Lao Mao <span dir="ltr"><<a href="mailto:laomao1975@googlemail.com">laomao1975@googlemail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<p>Hello,</p><p>I've written the below to get the previous day's logs from an Amazon S3 bucket.<br></p><p>#!/usr/bin/python <br> import time <br> from datetime import datetime <br> import boto <br> </p><p>daily_s3_log = open("/tmp/s3logs", "w+") <br>
now = datetime.now() <br> connection = boto.connect_s3() <br> bucket = connection.get_bucket("<a href="http://downloads.sekrit.co.uk" target="_blank">downloads.sekrit.co.uk</a>") <br> todays_keys = [] <br> </p>
<p>for key in bucket: <br>
time_difference = (now - <br> datetime(*time.strptime(key.last_modified.split(".")[0], "%Y-%m-%dT%H: <br> %M:%S")[0:6])).days <br> if time_difference < 1 and key.name.startswith("log/access_log"): <br>
todays_keys.append(key) <br> </p><p>for key in todays_keys: <br> key.get_file(daily_s3_log) <br> </p><p>daily_s3_log.close() <br> </p><p>This takes about 2 mins to download a day's logs (about 25M). <br> </p><br>
</blockquote><div>What I would do would be to profile your code:<br>You can find more profiling info @ <a href="http://docs.python.org/library/profile.html">http://docs.python.org/library/profile.html</a><br><br>-Tino<br>
<br></div></div>