fsync() doesn't work as advertised?

[Nobody]

On Mon, 04 Jan 2010 08:09:56 -0800, Brian D wrote:

> If I'm running a process in a loop that runs for a long time, I
> occasionally would like to look at a log to see how it's going.
> 
> I know about the logging module, and may yet decide to use that.
> 
> Still, I'm troubled by how fsync() doesn't seem to work as advertised:
> 
> http://docs.python.org/library/os.html
> 
> "If you’re starting with a Python file object f, first do f.flush(),
> and then do os.fsync(f.fileno())"

The .flush() method (and the C fflush() function) causes the
contents of application buffers to be sent to the OS, which basically
copies the data into the OS-level buffers.

fsync() causes the OS-level buffers to be written to the physical drive.

File operations normally use the OS-level buffers; e.g. if one process
write()s to a file and another process read()s it, the latter will see
what the former has written regardless of whether the data has been
written to the drive.

The main reason for using fsync() is to prevent important data from being
lost in the event of an unexpected reboot or power-cycle (an expected
reboot via the "shutdown" or "halt" commands will flush all OS-level
buffers to the drive first). Other than that, fsync() is almost invisible
(I say "almost", as there are mechanisms to bypass the OS-level buffers,
e.g. the O_DIRECT open() flag).