Replace in large text file ?

[MRAB]

Steven D'Aprano wrote:
> On Sat, 05 Jun 2010 00:53:23 -0700, Steve wrote:
> 
>> I am new to Python and am wanting  to replace characters in a very large
>> text file.....6 GB
>> In plain language what I wish to do is:
>>
>> Remove all comma's
>> Replace all @ with comma's
>> Save as a new file.
> 
> 
> input_file = open("some_huge_file.txt", "r")
> output_file = open("newfilename.txt", "w")
> for line in input_file:
>     line = line.replace(",", "")
>     line = line.replace("@", ",")
>     output_file.write(line)
> output_file.close()
> input_file.close()
> 
I'd probably process it in larger chunks:

     CHUNK_SIZE = 1024 ** 2 # 1MB at a time
     input_file = open("some_huge_file.txt", "r")
     output_file = open("newfilename.txt", "w")
     while True:
         chunk = input_file.read(CHUNK_SIZE)
         if not chunk:
              break
         chunk = chunk.replace(",", "")
         chunk = chunk.replace("@", ",")
         output_file.write(chunk)
     output_file.close()
     input_file.close()