[ python-Bugs-1633941 ] for line in sys.stdin: doesn't notice EOF the first time

[SourceForge.net]

Bugs item #1633941, was opened at 2007-01-12 05:34
Message generated for change (Comment added) made by draghuram
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633941&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Nobody/Anonymous (nobody)
Summary: for line in sys.stdin: doesn't notice EOF the first time 

Initial Comment:
[forwarded from http://bugs.debian.org/315888]

for line in sys.stdin: doesn't notice EOF the first time when reading from tty.

The test program:

    import sys
    for line in sys.stdin:
            print line,
    print "eof"

A sample session:

    liw at esme$ python foo.py
    foo         <--- I pressed Enter and then Ctrl-D
    foo         <--- then this appeared, but not more
    eof         <--- this only came when I pressed Ctrl-D a second time
    liw at esme$

Seems to me that there is some buffering issue where Python needs to
read end-of-file twice to notice it on all levels. Once should be 
enough.



----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-25 14:04

Message:
Logged In: YES 
user_id=984087
Originator: NO


BTW, I opened bug 1643712 for doc change.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-24 12:20

Message:
Logged In: YES 
user_id=984087
Originator: NO


I tested two kinds of inputs with iter and noiter verisons. I posted
"noter" code and OP's code is the iter version.

1) For input without newline at all (line1<CTRL-D><CTRL-D><CTRL-D>)
behaves same with both versions.
2) The noiter version prints "eof" with "line1\n<CTRL-D>" while the iter
version requires an additional CTRL-D. This is because iter version uses
read ahead which is implemented using fread() . A simple C program using
fread() behaves exactly same way. 

I tested on Linux but am sure windows behaviour (as posted by 
gagenellina) will have same reasons. Since the issue is with platform's
stdio library, I don't think python should fix anything here. However, it
may be worthwhile to mention something about this in documentation. I will
open a bug for this purpose. 







----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 12:45

Message:
Logged In: YES 
user_id=984087
Originator: NO


Ok. This may sound stupid but I couldn't find a way to attach a file to
this bug report. So I am copying the code here:

************
import sys

line = sys.stdin.readline()
while (line):
    print  line,
    line = sys.stdin.readline()

print "eof"
*************


----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 12:37

Message:
Logged In: YES 
user_id=984087
Originator: NO


Sorry for my duplicate comment. It was a mistake. On closer examination,
the OP's description does seem to indicate some issue. Please look at
(attached) stdin_noiter.py which uses readline() directly and it does not
have the problem described here. It properly detects EOF on first CTRL-D.
This points to some problem with the iterator function
fileobject.c:file_iternext(). I think that the first CTRL-D might be
getting lost somewhere in the read ahead code (which only comes into
picture with iterator).

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 11:34

Message:
Logged In: YES 
user_id=984087
Originator: NO


I am not entirely sure that this is a bug.

$ cat testfile
line1
line2

$ python foo.py < testfile

This command behaves as expected. Only when the input is from tty, the
above described behaviour happens. That could be because of the terminal
settings where characters may be buffered until a newline is entered.



----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 11:34

Message:
Logged In: YES 
user_id=984087
Originator: NO


I am not entirely sure that this is a bug.

$ cat testfile
line1
line2

$ python foo.py < testfile

This command behaves as expected. Only when the input is from tty, the
above described behaviour happens. That could be because of the terminal
settings where characters may be buffered until a newline is entered.



----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-01-13 23:20

Message:
Logged In: YES 
user_id=479790
Originator: NO

Same thing occurs on Windows. Even worse, if the line does not end with
CR, Ctrl-Z (EOF in Windows, equivalent to Ctrl-D) has to be pressed 3
times:

D:\Temp>python foo.py
foo  <--- I pressed Enter
^Z   <--- I pressed Ctrl-Z and then Enter again
foo  <--- this appeared
^Z   <--- I pressed Ctrl-Z and then Enter again
<EOF>

D:\Temp>python foo.py
foo^Z   <--- I pressed Ctrl-Z and then Enter
^Z      <--- cursor stays here; I pressed Ctrl-Z and then Enter again
^Z      <--- cursor stays here; I pressed Ctrl-Z and then Enter again
foo <EOF>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633941&group_id=5470