Hi reliability files, writing,reading and maintaining

John Pote johnpote at blueyonder.co.uk
Tue Feb 7 12:46:32 EST 2006


Hello, help/advice appreciated.

Background:
I am writing some web scripts in python to receive small amounts of data 
from remote sensors and store the data in a file. 50 to 100 bytes every 5 or 
10 minutes. A new file for each day is anticipated.  Of considerable 
importance is the long term availability of this data and it's gathering and 
storage without gaps.

As the remote sensors have little on board storage it is important that a 
web server is available to receive the data. To that end two separately 
located servers will be running at all times and updating each other as new 
data arrives.

I also assume each server will maintain two copies of the current data file, 
only one of which would be open at any one time, and some means of 
indicating if a file write has failed and which file contains valid data. 
The latter is not so important as the data itself will indicate both its 
completeness (missing samples) and its newness because of a time stamp with 
each sample.
I would wish to secure this data gathering against crashes of the OS, 
hardware failures and power outages.

So my request:
1. Are there any python modules 'out there' that might help in securely 
writing such files.
2. Can anyone suggest a book or two on this kind of file management. (These 
kind of problems must have been solved in the financial world many times).

Many thanks,

John Pote








More information about the Python-list mailing list