[Distutils] Concurrent distutils builds can fail due to mkdir

[Tom Epperly]

I have a build where two concurrent Python processes are both building C
extension modules using distutils in the same directory.  I have a "python
setup.py build_ext" and a "python synch_setup.py build_ext" both in the
same directory. Each setup.py refers to different files that need to be
built, but occasionally one of the builds fails because it fails when
executing mkdir("build"). With two concurrent distutils building in the
same directory, there is a race condition when making the "build"
directories.

Here is an excerpt from dir_util.py:

        if not dry_run:
            try:
                os.mkdir(head)
                created_dirs.append(head)
            except OSError, exc:
                raise DistutilsFileError, \
                      "could not create '%s': %s" % (head, exc[-1])

mkdir can fail for two reasons -- one because the directory already exists
and two because it couldn't create the directory. It would be nice if the
except clause would ignore the failure when the directory already exists
by testing if the directory exists and is accessible. This would prevent
one of the setup.py's from stopping when it could really safely continue.

I am not a member of the distutils-sig mailing list, so please Cc me on
replies.

Thanks,

Tom

--
------------------------------------------------------------------------
Tom Epperly
Center for Applied Scientific Computing   Phone: 925-424-3159
Lawrence Livermore National Laboratory      Fax: 925-424-2477
L-661, P.O. Box 808, Livermore, CA 94551  Email: tepperly at llnl.gov
------------------------------------------------------------------------