Reading a file that is changing and getting the new lines

[r0g]

Ouray Viney wrote:
> Hi:
> 
> Problem:
> =========
> I want to read a ASCII text file that can have data appended to it.  I
> have hacked some code together that handles the basics, but it falls
> short.  My code doesn't read in the new lines that could have been
> added to the end of the file.  Not python's fault, more that I don't
> know how to do it in python.
> 
> Example scenario:  As the python script is running a user/application
> adds new entries to the end of the test case file, example, adds the
> following to the file.
> 
> test4
> test5
> test6
> 
> The python code below, won't pick these changes up.
> 
> Analysis:
> ========
> I understand why my code doesn't handle the addition of new data.
> This is simply because the os.open reads all the lines in the
> beginning and since the data is added after that, the for loop doesn't
> iterate through it since it doesn't know about it.  Please note, my
> code is not pretty and isn't at all advanced.
> 
> Code:
> ======
> import os,string,sys,time
> 
> # define locals
> tcListFile="testcase_list.txt"
> fh=open(tcListFile,"r")
> tcList=fh.readlines()
> fh.close()
> 
> for tc in tcList:
> 	print tc.strip()
> 	print "sleeping for 2 seconds"
> 	time.sleep(2)
> 
> # close the file handle
> fh.close()
> 
> text file:
> ======
> test1
> test2
> test3
> 
> Question:
> =========
> What is the best way to handle this type of scenario using python?

Read your file once.
Make a note of the length (X)
Close the file
Setup a loop with a sleep at the end.
  Each time round check the file size
  If it has increased, make a note of it (Y) and
  open the file, seek to X and read Y - X bytes
  This gives you the new data, you can then append it to the inital read
or whatever.

Do bar in mind though that all OS's buffer IO so the program that's
generating the file might have a different view of it than your python
script i.e. program A might output 300 chars to file but they may not be
written to disk until the buffer receives 4096 chars. If this might be a
problem to you make sure the program generating the data flushes the
cache after every write (sys.stdout.flush() in python).

Roger.