Python child process in while True loop blocks parent
Cameron Simpson
cs at cskk.id.au
Wed Dec 8 16:44:52 EST 2021
On 08Dec2021 18:11, Jen Kris <jenkris at tutanota.com> wrote:
>Python must terminate its write strings with \n, or read will block in
>C waiting for something that never comes.
There are two aspects to this:
- if upstream is rding "lines of text" then you need a newline to
terminate the lines
- you (probably) should flush the output pipe (Python to C) after the
newline
I see you're using file descriptors and os.write() to sent data. This is
unbuffered, so there is nothing to flush, so you have not encountered
the second point above.
But if you shift to using a Python file object for your output (eg
f=os.fdopen(pw)), which would let you use print() or any number of other
things which do things with Python files) your file object would have a
buffer and normally that would not be sent to the pipe unless it was
full.
So your deadlock issue has 2 components:
- you need to terminate your records for upstream (C) to see complete
records. Your records are lines, so you need a newline character.
- you need to ensure the whole record has been sent upstream (been
written to the pipe). If you use a buffered Python file object for
your output, you need to flush it at your synchronisation points or
upstream will not receive the buffer contents. That synchronisation
point for you is the end of the record.
Hopefully this makes the flow considerations more clear.
Cheers,
Cameron Simpson <cs at cskk.id.au>
More information about the Python-list
mailing list