sorting 1172026 entries

[Chris Rebert]

On Sun, May 6, 2012 at 9:29 AM, J. Mwebaze <jmwebaze at gmail.com> wrote:
> sorry see, corrected code
>
>
> for filename in txtfiles:
>    temp=[]
>    f=open(filename)

Why not use `with` here too?

>    for line in f.readlines():

readlines() reads *the entire file contents* into memory all at once!
Use `for line in f:` instead, which will read from the file one line
at a time.

>      line = line.strip()
>      line=line.split()
>      temp.append((parser.parse(line[0]), float(line[1])))
>    temp=sorted(temp)

As already pointed out, use temp.sort() instead.

>    with open(filename.strip('.txt')+ '.sorted', 'wb') as p:

strip() doesn't do quite what you think it does:
$ python
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
>>> '.xtx.foo'.strip('.txt')
'foo'
>>>

Consult the docs.


Also, please avoid top-posting in the future.

Cheers,
Chris