[issue11877] Change os.fsync() to support physical backing store syncs

Steffen Daode Nurpmeso report at bugs.python.org
Sat May 7 15:37:36 CEST 2011


Steffen Daode Nurpmeso <sdaoden at googlemail.com> added the comment:

11877.5.diff incorporates all changes suggested by
Charles-François except for the 'auto' keyword, which is extremely
important and which would need to be invented if it would not
yet exist.

I'm dropping the old stuff.  And i think this is the final version
of the patch.  I've changed the default argument to 'True' as
i really think it's better to be on the safe side here.  (The
french are better off doing some gracy and dangerous sports to
discover edges of life!)  I'm still of the opinion that this
should be completely hidden, but since it's completely transparent
wether a Python function gets yet another named argument or not...

So, thanks to Ronald, i detected that also NetBSD introduced
a FDISKSYNC flag in 2005 and that you really need fsync_range()
there (at least by definition)!  How could they do that?  But i'm
also happy to see that all other systems try hard to achieve
security transparently and by default and unless i missed
something once again.

----------
Added file: http://bugs.python.org/file21924/11877.5.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11877>
_______________________________________
-------------- next part --------------
diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -798,7 +798,7 @@
    Availability: Unix.
 
 
-.. function:: fsync(fd)
+.. function:: fsync(fd, full_fsync=True)
 
    Force write of file with filedescriptor *fd* to disk.  On Unix, this calls the
    native :c:func:`fsync` function; on Windows, the MS :c:func:`_commit` function.
@@ -807,6 +807,12 @@
    ``f.flush()``, and then do ``os.fsync(f.fileno())``, to ensure that all internal
    buffers associated with *f* are written to disk.
 
+   The POSIX standart only requires that :c:func:`fsync` must transfer the
+   buffered data to the storage device, not that the data is actually
+   written by the device itself.  On operating systems where it is
+   necessary and possible the optional *full_fsync* argument can be used to
+   initiate additional steps to synchronize the physical backing store.
+
    Availability: Unix, and Windows.
 
 
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -2121,13 +2121,46 @@
 
 #ifdef HAVE_FSYNC
 PyDoc_STRVAR(posix_fsync__doc__,
-"fsync(fildes)\n\n\
-force write of file with filedescriptor to disk.");
-
-static PyObject *
-posix_fsync(PyObject *self, PyObject *fdobj)
-{
-    return posix_fildes(fdobj, fsync);
+"fsync(fildes, full_fsync=True)\n\n"
+"force write of file buffers with fildes to disk;\n"
+"full_fsync forces flush of disk caches in case fsync() alone is not enough.");
+
+static PyObject *
+posix_fsync(PyObject *self, PyObject *args, PyObject *kwargs)
+{
+    auto PyObject *fdobj;
+    auto int full_fsync = 1;
+    static char *keywords[] = {"fd", "full_fsync", NULL };
+
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i", keywords,
+                                     &fdobj, &full_fsync))
+        return NULL;
+
+    /* See issue 11877 discussion */
+# if ((defined __APPLE__ && defined F_FULLFSYNC) || \
+      (defined __NetBSD__ && defined FDISKSYNC))
+    if (full_fsync != 0) {
+        int res, fd = PyObject_AsFileDescriptor(fdobj);
+        if (fd < 0)
+            return NULL;
+        if (!_PyVerify_fd(fd))
+            return posix_error();
+
+        Py_BEGIN_ALLOW_THREADS
+#  if defined __APPLE__
+        res = fcntl(fd, F_FULLFSYNC);
+#  elif defined __NetBSD__
+        res = fsync_range(fd, FFILESYNC|FDISKSYNC, 0, 0);
+#  endif
+        Py_END_ALLOW_THREADS
+
+        if (res < 0)
+            return posix_error();
+        Py_INCREF(Py_None);
+        return Py_None;
+    } else
+# endif
+        return posix_fildes(fdobj, fsync);
 }
 #endif /* HAVE_FSYNC */
 
@@ -9472,7 +9505,8 @@
     {"fchdir",          posix_fchdir, METH_O, posix_fchdir__doc__},
 #endif
 #ifdef HAVE_FSYNC
-    {"fsync",       posix_fsync, METH_O, posix_fsync__doc__},
+    {"fsync",           (PyCFunction)posix_fsync, METH_VARARGS|METH_KEYWORDS,
+                        posix_fsync__doc__},
 #endif
 #ifdef HAVE_SYNC
     {"sync",        posix_sync, METH_NOARGS, posix_sync__doc__},


More information about the Python-bugs-list mailing list