[issue11277] Crash with mmap and sparse files on Mac OS X

[Steffen Daode Nurpmeso]

Steffen Daode Nurpmeso <sdaoden at googlemail.com> added the comment:

So sorry that i'm stressing this, hopefully it's the final message.
Apples iterative kernel-update strategy resulted in these versions:

    14:02 ~/tmp $ /usr/sbin/sysctl kern.version
    kern.version: Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386
    14:02 ~/tmp $ gcc -o zt osxversion.c -framework CoreServices
    14:03 ~/tmp $ ./zt 
    OS X version: 10.6.8
    apple_osx_needs_fullsync: -1

I.e. the new patch uses >10.7.0 or >=10.6.8 to avoid that
FULLFSYNC disaster (even slower than the Macrohard memory
allocator during "Wintel" partnership!), and we end up as:

    14:03 ~/src/cpython $ ./python.exe -E -Wd -m test -r -w -uall test_mmap
    Using random seed 8466468
    [1/1] test_mmap
    1 test OK.

P.S.: i still have no idea how to do '-framework CoreServices'
regulary.  Like i've said in #11046 i never used GNU Autoconf/M4,
sorry.  You know.  Maybe the version check should be moved
somewhere else and simply be exported, even replacing the stuff
from platform.py?  I don't know.  Bye.
--
Ciao, Steffen
sdaoden(*)(gmail.com)
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments

----------
Added file: http://bugs.python.org/file22593/11277.apple-fix-3.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11277>
_______________________________________
-------------- next part --------------
diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst
--- a/Doc/library/mmap.rst
+++ b/Doc/library/mmap.rst
@@ -88,7 +88,8 @@
 
    To ensure validity of the created memory mapping the file specified
    by the descriptor *fileno* is internally automatically synchronized
-   with physical backing store on Mac OS X and OpenVMS.
+   with physical backing store on operating systems where this is
+   necessary, e.g. OpenVMS, and some buggy versions of Mac OS X.
 
    This example shows a simple way of using :class:`mmap`::
 
diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c
--- a/Modules/mmapmodule.c
+++ b/Modules/mmapmodule.c
@@ -25,6 +25,8 @@
 #define UNIX
 # ifdef __APPLE__
 #  include <fcntl.h>
+
+#  include <CoreServices/CoreServices.h>
 # endif
 #endif
 
@@ -65,6 +67,44 @@
 #define my_getpagesize getpagesize
 #endif
 
+# ifdef __APPLE__
+static void
+apple_osx_needs_fullsync(long *use_fullsync)
+{
+    /* Issue #11277: mmap(2) bug with >32 bit sparse files.
+     * Apple fixed the bug before announcement of OS X "Lion", but since we
+     * need to handle buggy versions, perform a once-only check to see if the
+     * running kernel requires the expensive sync.  Fixed in 10.6.8, 10.7++.
+     * >0: F_FULLSYNC is required, <0: kernel has mmap(2) bug fixed */
+    SInt32 ver;
+    *use_fullsync = 1;
+
+    if (Gestalt(gestaltSystemVersion, &ver) != noErr)
+        goto jleave;
+    /* SystemVersion(Major|Minor|BugFix) available at all? */
+    if (ver < 0x1040)
+        goto jleave;
+    if (Gestalt(gestaltSystemVersionMajor, &ver) != noErr)
+        goto jleave;
+    if (ver > 10)
+        goto jgood;
+    if (Gestalt(gestaltSystemVersionMinor, &ver) != noErr)
+        goto jleave;
+    if (ver >= 7)
+        goto jgood;
+    if (ver < 6)
+        goto jleave;
+    if (Gestalt(gestaltSystemVersionBugFix, &ver) != noErr)
+        goto jleave;
+    if (ver < 8)
+        goto jleave;
+jgood:
+    *use_fullsync = -1;
+jleave:
+    return;
+}
+# endif /* __APPLE__ */
+
 #endif /* UNIX */
 
 #include <string.h>
@@ -1150,8 +1190,14 @@
 #ifdef __APPLE__
     /* Issue #11277: fsync(2) is not enough on OS X - a special, OS X specific
        fcntl(2) is necessary to force DISKSYNC and get around mmap(2) bug */
-    if (fd != -1)
-        (void)fcntl(fd, F_FULLFSYNC);
+    if (fd != -1) {
+        /* (GIL protected) */
+        static long use_fullsync /*= 0*/;
+        if (!use_fullsync)
+            apple_osx_needs_fullsync(&use_fullsync);
+        if (use_fullsync > 0)
+            (void)fcntl(fd, F_FULLFSYNC);
+    }
 #endif
 #ifdef HAVE_FSTAT
 #  ifdef __VMS