Help: using msvcrt for file locking

[David Bolen]

Sheila King <sheila at spamcop.net> writes:

> :Note that the atomicity of renaming is critical to making this work. E.g.,
> 
> Would someone please explain the concept of "atomic" or "atomicity" with
> respect to file operations. Do you mean, that you are (below) showing in
> detail, step-by-step what would happen at each point in the process that
> you suggest?

Atomicity or ensuring that an operation is "atomic" refers to the fact
that the operation in question can be considered a single
uninterruptible unit of activity, at some level of granularity.  The
jargon file says it comes from the Greek "atomos" or indivisible.

For a CPU, atomic often means a single instruction since anything more
than a single instruction can be interrupted.  But you can do stuff
like disable all processor interrupts, and then execute a series of
instructions as a single atomic activity.  Higher level operations can
be atomic as well, at least as when compared to other operations at
the same general level and affecting the same resource.

Particularly when synchronization is involved, the operations being
used as part of the synchronization must guarantee that they cannot be
broken into separate steps or else they may fail to provide proper
synchronization.

This can not always be deduced simply by the top level entry points
into operations (e.g., the functions called, or the instructions
executed), since the operations being called may be formed by lower
level operations that themselves provide separate steps that might be
interrupted.

For example, take a really simple manual synchronization primitive of a
binary value used as a flag to control access to a shared resource.
If the value is 0, then nobody is using the resource, while if 1, then
the resource is in use.  You might write code like:

    if flag == 0:
        # Mark that we're working with the data
        flag = 1

        # ...
        # Use protected information
        # ...

        # Mark that we're done
        flag = 0

The problem, of course, is that something could change in between your
querying the value to see if it's clear (0) and then resetting it to
1.  That operation is therefore not atomic.  Even if a compiler were
to offer a function that could test and set a value in one call, if it
compiled to separate processor instructions to load the current
contents from memory, test it and then write it back, an interrupt
during that sequence of operations could change the memory location.

Since this concept (testing and setting a value) is so common, many
systems - even down at the chip level - implement specific operations
for an atomic test-and-set operation.

In this case, instead of a flag, a file might be getting used as a
sentinel, and the rename operation on the file treated as a high level
way to change the state of that sentinel.  What is important is that
the rename operation not be implemented in such a way as to permit its
activity to be interrupted or interfered with by other filesystem
activity related to the files in question.  Otherwise you couldn't be
sure that after you renamed the file that the result hadn't been
influenced by other filesystem activity (equivalent to changing the
flag value in the above example) and thus it would no longer be
suitable as a consistent sentinel status.

In many (most? I'm not sure) filesystems, rename is an atomic (through
other internal locks or procedures) operation to help ensure the
integrity of the filesystem, at least when renaming is an integral
part of the filesystem interface.  There are other times when it
doesn't hold true though, many of which occur when using networked
filesystems.  Another good odds atomic operation is making a directory
- that's normally my preference if I have to have a cross-platform
sentinel file (it should even work with NFS).  The big problem with
any non-active-filehandle approach (like a file or directory) is that
cleanup is critical if you don't want to block other tasks.  So if
there's any chance the process managing the lock might die, you need
some way to remove the lock if that happens.

> Actually, the win32file.CreateFile does have an option for exclusive
> access. I don't believe there is a specified timeout, but it does have
> the exclusive access part.

It doesn't have the timeout, thus as my suggested code in another post
does, you have to implement a retry mechanism yourself.  This leaves a
small risk of starvation (depending on load and retry mechanism) but
in most cases it's not a major problem.

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/