Reading a file that is changing and getting the new lines
Dave Angel
davea at ieee.org
Tue Dec 1 22:28:00 EST 2009
Ouray Viney wrote:
> Hi:
>
> Problem:
> =========
> I want to read a ASCII text file that can have data appended to it. I
> have hacked some code together that handles the basics, but it falls
> short. My code doesn't read in the new lines that could have been
> added to the end of the file. Not python's fault, more that I don't
> know how to do it in python.
>
> Example scenario: As the python script is running a user/application
> adds new entries to the end of the test case file, example, adds the
> following to the file.
>
> test4
> test5
> test6
>
>
I need clarification. Is this an independent process, that appends data
onto the same file, after the python script has opened it, but while
it's still running? Can you give me an example point in your source
code that your script might be running when the other application does
its append?
> The python code below, won't pick these changes up.
>
> Analysis:
> ========
> I understand why my code doesn't handle the addition of new data.
> This is simply because the os.open reads all the lines in the
>
os.open() doesn't read the lines, the readlines() call does. Once
you've got the data in a list, of course changing the file has no impact
on your code. And by the way, you didn't call os.open(), you called
open(). They are not interchangeable.
> beginning and since the data is added after that, the for loop doesn't
> iterate through it since it doesn't know about it. Please note, my
> code is not pretty and isn't at all advanced.
>
> Code:
> ======
> import os,string,sys,time
>
> # define locals
> tcListFile="testcase_list.txt"
> fh=open(tcListFile,"r")
> tcList=fh.readlines()
> fh.close()
>
> for tc in tcList:
> print tc.strip()
> print "sleeping for 2 seconds"
> time.sleep(2)
>
> # close the file handle
> fh.close()
>
> text file:
> ======
> test1
> test2
> test3
>
> Question:
> =========
> What is the best way to handle this type of scenario using python?
>
You can solve your first level of problem by simply skipping the
readlines() call, and eliminating the first close() call. You close it
again later, let that be the only close. You want your loop to actually
get a line from the file, not from a list which is frozen in time.
Naturally, you'd want a better name than tcList.
Next problem is likely to be one of sharing the file. You open it
read-only, the other application simultaneously tries to append. Does
the OS permit that? You find out. You might actually have to close the
file during those sleep() calls, and then do an open/seek to get to the
earlier position. You most likely will have to switch from open() to
os.open(). The calls are very different.
Final problem that I note is how to decide when to quit your loop. Just
because you hit eof (end of file), doesn't mean the other application
isn't about to append another 50 lines. So you need a different
termination condition.
I suspect the sharing problems aren't going to be portable between
operating systems, so you might want to answer the usual "what version
of Python, on what OS" question that we seem to have for most new questions.
DaveA
More information about the Python-list
mailing list