Windows binary stdin goes EOF after \x1a character

[MRAB]

On 15/10/2010 20:28, Dan wrote:
> I am writing a Windows program in Python 3.1.2 that reads binary data
> from stdin.  Whenever it hits a \x1a character, stdin goes EOF and no
> more data can be read.  A short program that exhibits this problem is:
>
> #listing of main.pyw
> import sys
> def go():
>      bb=sys.stdin.buffer.raw.read(10000)
>      print(bb,file=sys.stderr)
>      sys.stderr.flush()
> go()
> go()
>
>
> The program is run as "pythonw.exe main.pyw"
>
>
> When fed with a block of bytes equivalent to "bytes(range(1,255))",
> the following output is observed...
>
> b'\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f
> \x10\x11\x12\x13\x14\x15\x16\x17\x18\x19'
> b''
>
> ...and according to the documentation, the empty bytes object means
> EOF.
>
> I wrote an equivalent program in C++ using the win32 ReadFile() call,
> and it read all 255 bytes just fine.  What am I doing wrong with the
> python code?
>
>
> I am using Erlang to launch the Python program as a subprocess.  The
> Erlang fragment that launches this and sends the data is:
>
>      Port=open_port( {spawn_executable, "c:/Python31/pythonw.exe"},
> 		    [{args, ["c:/iotest/main.pyw"]}]),
>      Port ! {self(),{command,lists:seq(1,255)}},
>
By default, Python opens stdin in buffered text mode in which '\x1A'
marks the end of the text. Try adding the "-u" option ("unbuffered") to
Python's command line:

     pythonw.exe -u main.pyw