[New-bugs-announce] [issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize
jimbo1qaz_ via Gmail
report at bugs.python.org
Wed Jan 16 16:33:08 EST 2019
New submission from jimbo1qaz_ via Gmail <jimbo1qaz at gmail.com>:
Windows 10 1709 x64, Python 3.7.1.
Minimal example and stack traces at https://gist.github.com/jimbo1qaz/75d7a40cac307f8239ce011fd90c86bf
Essentially I create a subprocess.Popen, using a process (msys2 head.exe) which closes its stdin after some amount of input, then write nothing but b"\n"*1000 bytes to its stdin.
If the bufsize is small (1000 bytes), I always get OSError: [Errno 22] Invalid argument
If the bufsize is large (1 million bytes), I always get BrokenPipeError: [Errno 32] Broken pipe. (This happens whether I write 1 million newlines or 1000 at a time).
Originally I created a ffmpeg->ffplay pipeline with a massive bufsize (around 1280*720*3 * 2 frames), then wrote 1280*720*3 bytes of video frames at a time. Closing ffplay's window usually created BrokenPipeError, but occasionally OSError. This was actually random.
It seems that this is known to some extent, although I couldn't find any relevant issues on the bug tracker, and "having to catch 2 separate errors" isn't explained on the documentation. (Is it intended though undocumented behavior?)
Popen._communicate() calls Popen._stdin_write(), but specifically ignores BrokenPipeError and OSError where exc.errno == errno.EINVAL == 22 (the 2 cases I encountered).
But I don't call Popen.communicate() but instead write directly to stdin, since I have a loop that outputs 1 video frame at a time, and rely on pipe blocking to stop my application from running too far ahead of ffmpeg/ffplay.
popen.stdin is a <_io.BufferedWriter name=3>.
>Write the given bytes-like object, b, and return the number of bytes written (always equal to the length of b in bytes, since if the write fails an OSError will be raised). Depending on the actual implementation, these bytes may be readily written to the underlying stream, or held in a buffer for performance and latency reasons.
The page doesn't mention BrokenPipeError at all (Ctrl+F). So why do I *sometimes* get a BrokenPipeError (subclasses ConnectionError subclasses OSError) instead?
title: When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize
versions: Python 3.7
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce