[ python-Bugs-1396471 ] file.tell() returns illegal value on Windows

[SourceForge.net]

Bugs item #1396471, was opened at 2006-01-04 02:53
Message generated for change (Comment added) made by pterk
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1396471&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.4
Status: Open
Resolution: None
Priority: 7
Submitted By: Tom Goddard (tom_goddard)
Assigned to: Nobody/Anonymous (nobody)
Summary: file.tell() returns illegal value on Windows

Initial Comment:
The file tell() method returns an illegal value that causes an exception 
when passed to seek().  This happens on Windows when a file that 
contains unix-style newlines '\n' is opened and read in text mode 'r'.  
Below is code that produces the problem on Windows 2.4.2 in an IDLE 
shell.

The bug does not occur when using mode 'rU' or 'rb'.  But I expect 
correct behaviour with mode 'r'.  My understanding is that 'rU' 
translates line endings to '\n' in the returned string while mode 'r' still 
correctly reads the lines using readline(), recognizing all 3 common 
endline conventions, but does not translate (ie includes \n\r or \r or \n 
in returned string).

The error in the tell() return value depends on how long the file is.  
Changing the 'more\n'*10 line in the example code will cause different 
incorrect return values.

Previous bug reports have mentioned problems with tell/seek when 
using file iterators, the file.next() method and the "for line in file:" 
construct.  This bug is different and just involves readline() and tell() 
with mode 'r'.

Example code tellbug.py follows:

wf = open('testdata', 'wb')
wf.write('01234\n56789\n'+ 'more\n'*10)
wf.close()

f = open('testdata', 'r')
f.readline()
t = f.tell()
print t
f.seek(t)

-------

Running gives:

>>> execfile('tellbug.py')
-5

Traceback (most recent call last):
  File "<pyshell#14>", line 1, in -toplevel-
    execfile('tellbug.py')
  File "tellbug.py", line 9, in -toplevel-
    f.seek(t)
IOError: [Errno 22] Invalid argument

----------------------------------------------------------------------

Comment By: Peter van Kampen (pterk)
Date: 2006-01-16 04:24

Message:
Logged In: YES 
user_id=174455

I have done some additional research on this.

It appears the bug is in ftell of the windows c-library and
it seems to be a 'known problem'. See:

http://www.mathworks.com/access/helpdesk/help/techdoc/ref/ftell.html

More here:
http://www.cygwin.com/faq/faq.api.html
http://gnuwin32.sourceforge.net/compile.html (search for
ftell in both cases)

Also see:

http://sourceforge.net/tracker/index.php?func=detail&aid=1381717&group_id=5470&atid=105470

So even though the file isn't opened with 'rt' I gather from
Tim Peter's remark about 't' on windows that the default is
't' anyway unless you really try. 

So this example is *exactly* what the matlab-people found.
That is nice to know I guess but it doesn't really help to
solve the problem.

It seems to me there are three solutions.

1. ftell could be fixed for unix-files on windows.* Not very
likely to happen soon. 
2. Python could special-case this for this specific
scenario. Way beyond my capabilities and probably too much
hassle in any case especially since the 'fix' is so easy
(just open with 'rb').
3. Leave it as is but document it. I have submitted a
(doc-)patch 1407021
(http://sourceforge.net/tracker/index.php?func=detail&aid=1407021&group_id=5470&atid=305470)
for the library reference (libstdtypes.tex) that reads:

Please note that tell() can return illegal values (after an
fgets()) on Windows when reading files with UNIX-style
line-endings. Use 'rb'-mode to circumvent this
problem.

* MS provides this note in the compatibility section of the
c-runtime libraries: "Note: In this version of Visual C++,
UNIX compatibility information has been removed from the
function descriptions."

In case your wondering what 'this version' is: VS.NET and VS
6.0. Apperantly there were no other 'previous versions'. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1396471&group_id=5470