<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 05/06/2012 09:29 AM, J. Mwebaze wrote:
<blockquote
cite="mid:CAH0SsdJ_wxeC_Hc_dehrJd5Ltyk+sGO5rOy7i6sT06NGG=Szsg@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div>
<div>sorry see, corrected code</div>
<div><br>
</div>
<div><br>
</div>
<div>for filename in txtfiles:</div>
<div> temp=[]</div>
<div> f=open(filename)</div>
<div> for line in f.readlines():</div>
<div> line = line.strip()</div>
<div> line=line.split()</div>
<div> temp.append((parser.parse(line[0]), float(line[1])))</div>
<div> temp=sorted(temp)</div>
<div> with open(filename.strip('.txt')+ '.sorted', 'wb') as p:</div>
<div> for i, j in temp:</div>
<div> p.write('%s %s\n' %(str(i),j))</div>
</div>
</blockquote>
<br>
Don't do<br>
temp = sorted(temp)<br>
That will create a *new* copy of the list to sort, and the
assignment will free up the original list for deletion and garbage
collection.<br>
<br>
Instead do the in-place sort:<br>
temp.sort()<br>
Same result, less thrashing.<br>
<br>
This will make your program slightly more efficient, HOWEVER, it is
not the solution of your week-long sort problem.<br>
<br>
<br>
Gary Herron<br>
<br>
<br>
<br>
<blockquote
cite="mid:CAH0SsdJ_wxeC_Hc_dehrJd5Ltyk+sGO5rOy7i6sT06NGG=Szsg@mail.gmail.com"
type="cite">
<div><br>
</div>
<br>
<div class="gmail_quote">On Sun, May 6, 2012 at 6:26 PM, J.
Mwebaze <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:jmwebaze@gmail.com" target="_blank">jmwebaze@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">I have
attached one of the files, try to sort and let me know the
results. Kindly sort by date. ooops - am told the file exceed
25M.
<div>
<div><br>
</div>
<div>below is the code</div>
<div><br>
</div>
<div>
<div>import glob</div>
<div>txtfiles =glob.glob('*.txt') </div>
<div>import dateutil.parser as parser</div>
<div><br>
</div>
<div><br>
</div>
<div>for filename in txtfiles:</div>
<div> temp=[]</div>
<div> f=open(filename)</div>
<div> for line in f.readlines():</div>
<div> line = line.strip()</div>
<div> line=line.split()</div>
<div> temp.append((parser.parse(line[0]),
float(line[1])))</div>
<div> temp=sorted(temp)</div>
<div> with open(filename.strip('.txt')+ '.sorted', 'wb')
as p:</div>
<div> for i, j in temp:</div>
<div> p.write('%s %s\n' %(str(i),j))</div>
</div>
<div><br>
<div>
<div>
<div class="h5"><br>
<div class="gmail_quote">On Sun, May 6, 2012 at 6:21
PM, Devin Jeanpierre <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:jeanpierreda@gmail.com"
target="_blank">jeanpierreda@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">On Sun, May 6, 2012 at
12:11 PM, J. Mwebaze <<a
moz-do-not-send="true"
href="mailto:jmwebaze@gmail.com"
target="_blank">jmwebaze@gmail.com</a>>
wrote:<br>
> [ (datatime, int) ] * 1172026<br>
<br>
I can't duplicate slowness. It finishes fairly
quickly here. Maybe you<br>
could try posting specific code? It might be
something else that is<br>
making your program take forever.<br>
<br>
>>> x = [(datetime.datetime.now() +
datetime.timedelta(random.getrandbits(10)),
random.getrandbits(32)) for _ in
xrange(1172026)]<br>
>>> random.shuffle(x)<br>
>>> x.sort()<br>
>>><br>
<span><font color="#888888"><br>
-- Devin<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
</div>
</div>
<div class="im">-- <br>
<font style="color:rgb(0,51,0)" size="1"><b>Mob UG: <a
moz-do-not-send="true"
href="tel:%2B256%20%280%29%2070%201735800"
value="+256701735800" target="_blank">+256 (0)
70 1735800</a> | NL <a moz-do-not-send="true"
href="tel:%2B31%20%280%29%206%20852%20841%2038"
value="+31685284138" target="_blank">+31 (0) 6
852 841 38</a> | Gtalk: jmwebaze | skype:
mwebazej | URL: <a moz-do-not-send="true"
href="http://www.astro.rug.nl/%7Ejmwebaze"
target="_blank">www.astro.rug.nl/~jmwebaze</a><br>
<br>
/* Life runs on code */</b></font><br>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<font style="color:rgb(0,51,0)" size="1"><b>Mob UG: +256 (0) 70
1735800 | NL +31 (0) 6 852 841 38 | Gtalk: jmwebaze | skype:
mwebazej | URL: <a moz-do-not-send="true"
href="http://www.astro.rug.nl/%7Ejmwebaze" target="_blank">www.astro.rug.nl/~jmwebaze</a><br>
<br>
/* Life runs on code */</b></font><br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
<br>
</body>
</html>