[ python-Bugs-1735418 ] file.read() truncating strings under Windows

[SourceForge.net]

Bugs item #1735418, was opened at 2007-06-11 20:19
Message generated for change (Comment added) made by ilgiz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1735418&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cgkanchi (cgkanchi)
Assigned to: Nobody/Anonymous (nobody)
Summary: file.read() truncating strings under Windows

Initial Comment:
On Python 2.4.4 and 2.5.1 under Windows, file.read() fails to read a varying number of characters from the last line(s) of text files when asked to read more than 800 characters from near the end of the file. For example, if the last word of a 500kb file is "superlative", file.read() might output "erlative". The file pointer at this stage is very close (a few words at most) to the end of the file. I ran into this problem while writing a program to split .txt ebooks into smaller files so that my ancient iPod could handle them. The behaviour is identical on both 2.4.4 and 2.5.1 under Windows, but does not appear under Mac OS X. I was unable to test it under Linux. To test the bug, I used various books from http://gutenberg.org . The one primarily used was Pride and Prejudice by Jane Austen.

----------------------------------------------------------------------

Comment By: Ilguiz Latypov (ilgiz)
Date: 2007-06-12 11:51

Message:
Logged In: YES 
user_id=281701
Originator: NO

(e) To have tell() on the same level with read(), try the unbuffered mode
by specifying bufsize=0 in open(), 

  http://docs.python.org/lib/built-in-funcs.html


----------------------------------------------------------------------

Comment By: Ilguiz Latypov (ilgiz)
Date: 2007-06-12 11:47

Message:
Logged In: YES 
user_id=281701
Originator: NO


This is your coding bug.

(a) I would not trust tell().  Calculate the absolute position and use
seek().

(b) Just from the documentation to Python's file-like objects I can assume
that read() and tell() belong to different levels of API.  The read()
function has this in its documentation:

"Note that this method may call the underlying C function fread() more
than once in an effort to acquire as close to size bytes as possible".

http://docs.python.org/lib/bltin-file-objects.html

The tell() function's documentation refers to stdio's ftell().  This hints
that tell() will return the position of the fread() buffer's end, not the
read()'s end.

(c) It also appears that by adding 1 to the "current position - unget
size" you are skipping the space character itself.

(d) The rfind() might return -1 if the search fails.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1735418&group_id=5470