[Tutor] endless processing through for loop

[bob gailer]

Dinesh B Vadhia wrote:
> I have a program with 2 for loops like this (in pseudocode):
>  
> fw = open(newLine.txt, 'w')
> for i in xrange(0, 700,000, 1):
>     read a file fname from folder
>     for line in open(fname, 'r'):
>         do some simple string processing on line
>         fw.write(newline)
> fw.close()
>  
> That's it.  Very simple but after i reaches about 550,000 the program 
> begins to crawl.  As an example, the loops to 550,000 takes about an 
> hour.  From 550,000 to 580,000 takes an additional 4 hours.

Here's my simplistic Python version:

fw = open("newLine.txt", 'w')
for i in xrange(700000):
  if i % 1000 == 0: print i
  for line in open("file1.txt"): # 10 lines each 7 characters
    newline = line.replace("x","y")
    fw.write(newline)
fw.close()

It runs evenly, takes a couple of minutes.

Try it, see if you get the same outcome. If you do then let's examine 
how your program differs.

-- 
Bob Gailer
919-636-4239 Chapel Hill, NC