[issue42606] Support POSIX atomicity guarantee of O_APPEND on Windows

[Eryk Sun]

Eryk Sun <eryksun at gmail.com> added the comment:

> So the idea is to delete the file for a brief period, but then 
> undelete it. 

Currently NTFS defaults to using classic delete semantics for delete-on-close. However, it supports POSIX delete semantics for delete-on-close, so the default could change in a future release. When delete-on-close uses POSIX semantics, closing the last handle to the kernel File immediately unlinks the file. The upside is that a filesystem that uses POSIX delete semantics for delete-on-close should support SetFileInformationByHandle: FileDispositionInfoEx, which allows directly removing the delete-on-close flag.

That said, I can't stomach having to manually overwrite, truncate, delete and undelete files in order to avoid side effects if _open_osfhandle() fails. IMO, if opening a file has serious side effects, then we need a public API to allocate the fd beforehand. Or the idea needs to be put on hold until Python divorces itself from the C runtime's low I/O layer.

> If the file type is FILE_TYPE_CHAR and the first byte to write 
> was 26 (Ctrl-Z), it's treated as success. I don't think I 
> understand: it *seems* like it's handling something like writing 
> to the *input* of a console, but I'm not sure it's even possible 
> in this manner.

Writing to the console input buffer is possible, but not supported with WriteFile() or WriteConsoleW(). It requires WriteConsoleInputW(), which writes low-level input records.

ReadFile() on a console input handle is special cased to return that 0 bytes were read if the string read from the console starts with Ctrl+Z. But I don't know of a device that special cases WriteFile() like this. The documentation of _write() says "[w]hen writing to a device, _write treats a CTRL+Z character in the buffer as an output terminator". Whatever this meant in the past in OS/2 or DOS, I doubt that it's meaningful nowadays.

> Anything else is a failure with ENOSPC. This is mysterious too.

Possibly someone picked an error code that was good enough. Maybe it was selected for the case of a full pipe that's in non-blocking mode. The named pipe device doesn't fail an NtWriteFile() system call for a non-blocking pipe when there isn't enough space. It simply succeeds with 0 bytes written. For example:

    >>> fdr, fdw = os.pipe()
    >>> hw = msvcrt.get_osfhandle(fdw)
    >>> win32pipe.SetNamedPipeHandleState(hw, 1, None, None)
    >>> os.write(fdw, b'a'*4096)
    4096
    >>> os.write(fdw, b'a')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 28] No space left on device

A POSIX developer would expect this case to fail with EAGAIN and raise BlockingIOError. But support for non-blocking mode is poorly implemented by pipes in Windows. Developers are encouraged to use asynchronous I/O instead.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42606>
_______________________________________