[Python-Dev] Re: RELEASED Python 2.3.1

[Jeremy Hylton]

On Sat, 2003-09-27 at 13:48, "Martin v. Löwis" wrote:
> 3. There is an operating system crash (kernel panic or similar).
>     fsync does not help, as, for a buggy kernel, anything might have
>     happened to the data before.

This is a vacuously true statement :-).  Anything might have happened;
among other things, the kernel buffers could have been flushed to disk
before the kernel panic.  I imagine that there are many bugs that could
cause a kernel panic or deadlock that would not also corrupt data on its
way to disk.  That is, we can't avoid bugs that are going to cause
problems writing to disk, but we can do our best to minimize the effects
of other bugs.

> 5. There is a power outage. This is the case where fsync should help:
>     everything up to the write barrier is on disk. Of course, if the
>     disk drive itself has write caching, fsync may have completed
>     without the data being on the disk (this would be an fsync bug, but
>     I believe Linux suffers from this particular bug).

(There are OSes other than Linux.)

> So in short, fsync(2) helps only in case of a power outage; for normal
> operation, it is not needed. In the case of a power outage, it is
> doubtful whether it has the desired effect.

I see two cases where it improves the changes that data is on persistent
storage before the database reports to the client that the write
succeeded.

> Slightly more on-topic: os.fsync is even worse, as it cannot be
> used to implement a write barrier in case of multiple threads. It
> runs with the GIL released, so while fsync is running, other
> threads might change the file. The semantics of fsync in this case
> is pretty unclear, but it is likely not a write barrier.

Sure.  ZODB only allows a single thread to write at a time.

Jeremy