Concurrent writes to the same file
cs at zip.com.au
Thu Jul 11 08:31:58 CEST 2013
On 10Jul2013 22:57, Jason Friedman <jsf80238 at gmail.com> wrote:
| Other than using a database, what are my options for allowing two processes
| to edit the same file at the same time? When I say same time, I can accept
| delays. I considered lock files, but I cannot conceive of how I avoid race
Sure. (Assuming UNIX. Windows probably excludes both processes
having the file open at the one time, but that's not a show stopper.)
You need to use a lock file to ensure only one process accesses the file at a time.
While a process holds the lock, access the file.
There are two basic approaches here:
open file for write, modify the file, close it
both processes have the file open for update
modify file, flush buffers
The code I use to take a lockfile is my "lockfile" context manager:
An example caller goes:
backup = "%s.bak-%s" % (self.csvpath, datetime.datetime.now().isoformat())
if not self.keep_backups:
as shown here:
Simplify as necessary; you just want:
if you have no wish for backups in case of a failed modify.
If both processes need to see each other's changes then there's
some more logic needed to monitor the file for changes. I've got
such a scheme for CSV files in beta at present for a personal
project. It won't suit all use cases; mine is well defined.
Cameron Simpson <cs at zip.com.au>
Is it true, Sen. Bedfellow, that your wife rides with bikers? - Milo Bloom
More information about the Python-list