On Sat, Dec 26, 2020 at 07:39:42PM -0500, Michael Smith wrote:
I love context managers when they're alone, but I dislike stacking them. It is less clear how we can ensure the fsync happens exactly between flush and close with a context manager than a keyword argument to open. That is, if open is the only context manager, everything is great. But if is up to users to stack context managers including open and some fsync, I think correct ordering will be a problem.
Indeed. I started playing with this idea, and even though I intellectually know that you can't sync a closed file (fsync needs to know the file descriptor ID) I keep wanting to write:
with sync(): with open(...) as f: ...
I dunno, maybe it's just me, and I'm sure that if I get a sensible exception on that I would soon learn to write it as:
with open(...) as f: with sync(f): ...
But what concerns me more is what happens in more complex cases, for example with two or more files being written:
with open(..., 'w') as f: with open(..., 'w') as g: with sync(g): with sync(f): ...
Aside from the deeper nesting, its not at all obvious to me whether the syncs need to be in the same or opposite order to the opens, or if it even matters. And what if people insert code around the sync?
with open(...) as f: # code writing to the file here is probably safe with sync(f): ... # but code writing to the file here is not
If I have to remember all these fiddly little rules for the order of context managers and what can go where, I'd rather than stick to syncing manually:
with open(...) as f: write stuff sync
even though I have no idea of the pros and cons of fdatasync versus fsync and will probably just forget and use os.sync.
Even though I don't usually like functions with a million arguments, open is an exception. I'm +1 on adding a keyword-only "sync=False" parameter to open:
- if the file is opened in read-only mode, has no effect;
- if the file is opened for writing, guarantees that fdatasync or fsync will be called after flush and before closing.
(I'm not entirely sure fdatasync is better than fsync, it's only available on non-OS-X Unixes and I don't know why I would want to update the file data but not the metadata.)