[issue41291] Race conditions when opening and deleting a file on Mac OS X

[Giovanni Pizzi]

New submission from Giovanni Pizzi <gio.piz at gmail.com>:

Hello,

when creating deleting (with `os.remove`/`os.unlink`) a file and opening it in a different process at (almost) the same time, on Mac OS X relatively often I get a file that is empty instead of either a `FileNotFoundError` exception, or an open handle with the original content, i.e. at least one of the two operations (unlinking or opening) seems to be non-atomic.
(With empty I mean that if after I open the file (mode 'rb'), then `fhandle.read()` returns me b''.

More in particular, after quite some debugging I noticed that what happens is that if I stat the file descriptor, `st_ino` is zero.

This can reproduced very easily.
I set up a GitHub repository here: https://github.com/giovannipizzi/concurrent-delete-race-problem
with simple examples and tests (and both a Python implementation and a C implementation).
There are also GitHub Actions that run on Mac, Ubuntu and Windows, and apparently the problem exists only on Mac (and happens on all versions).

For completeness I attach also here a tar.gz with the two very short python files that just need be run in parallel - in my Mac OS X (macOS 10.14.6, 15" 2016 MacBook Pro, python 3.6) I get the error essentially at every run. 

Important: while much harder to reproduce, I can get the same error and behaviour also with the (I think equivalent) C code. 
Therefore, this seems to be more a problem with the Mac OS standard libraries?

My first question is: has anybody ever seen this problem? It seems to me quite easy to reproduce, but I'm surprised that I couldn't find any reference in the internet even after searching for various hours (maybe I used the wrong keywords?)

Second question: should this be reported directly to Apple?

Third question: Even if this is a bug, would it make sense to implement a patch in python that raises some exception if st_ino is zero when opening a file? Or am I simplifying too much and in some conditions st_ino=0 is valid on some types of mounted filesystems? Is there some other way to have a more atomic behaviour for this race condition in python?

Thanks a lot!

----------
components: IO, macOS
files: concurrency-tests.tar.gz
messages: 373606
nosy: Giovanni Pizzi, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Race conditions when opening and deleting a file on Mac OS X
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49315/concurrency-tests.tar.gz

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue41291>
_______________________________________