Dear list,
I recently noticed a python program which uses forks and pipes for
communication between the processes not behaving as expected. The minimal
example program:
--------------------------------------------------------------------------------
#!/usr/bin/python
import os, sys
r, w = os.pipe()
write = os.fdopen(w, 'w')
print >> write, "foo"
pid = os.fork()
if pid:
os.waitpid(pid, 0)
else:
sys.exit(0)
write.close()
read = os.fdopen(r)
print read.read()
read.close()
--------------------------------------------------------------------------------
This prints out "foo" twice although it's only written once to the pipe. It
seems that python doesn't flush file descriptors before copying them to the
child process, thus resulting in the duplicate message. The equivalent C
program behaves as expected,
--------------------------------------------------------------------------------
#include
В Втр, 09/12/2008 в 19:26 +0000, Lars Kotthoff пишет:
Dear list,
I recently noticed a python program which uses forks and pipes for communication between the processes not behaving as expected. The minimal example program:
If you write ==== r, w = os.pipe() os.write(w, 'foo') pid = os.fork() ==== You'll get the same result as C program. Or if you use fdopen in C program you'll get the same result as Python. The problem with the example is libc buffering. If you say write.flush(), buffer won't be shared with child process and you'll see only one 'foo'.
On Dec 9, 2008, at 2:26 PM, Lars Kotthoff wrote:
Dear list,
I recently noticed a python program which uses forks and pipes for communication between the processes not behaving as expected. The minimal example program:
[snip]
This prints out "foo" twice although it's only written once to the pipe. It seems that python doesn't flush file descriptors before copying them to the child process, thus resulting in the duplicate message. The equivalent C program behaves as expected,
[snip]
Is this behaviour intentional? I've tested both python and C on Linux, OpenBSD and Solaris (python versions 2.5.2 and 2.3.3), the behaviour was the same everywhere.
Yes, it's intentional. And, no, your programs aren't equivalent. Rewrite your C program to use fdopen, and fread/fwrite. *Then* it will be equivalent and have the same behavior as the python program. Alternatively, you can change your python program to use os.read/ os.write instead of fdopen and fileobject.read/fileobject.write, if you want your python program to work like the C program. James
Lars Kotthoff wrote:
This prints out "foo" twice although it's only written once to the pipe. It seems that python doesn't flush file descriptors before copying them to the child process, thus resulting in the duplicate message. The equivalent C program behaves as expected,
Your Python and C programs are not equivalent -- the C one is writing directly to the file descriptor, whereas the Python one is effectively using a buffered stdio stream. The unflushed stdio buffer is getting copied by the fork, hence the duplicate output. Solution: either (a) flush the Python file object before forking or (b) use os.write() directly on the fd to avoid the buffering. -- Greg
participants (4)
-
Alexander Shigin -
Greg Ewing -
James Y Knight -
Lars Kotthoff