[Python-bugs-list] [ python-Bugs-645594 ] for lin in file: file.tell() tells wrong

[SourceForge.net]

Bugs item #645594, was opened at 2002-11-29 12:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=645594&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Dmitry Vasiliev (hdima)
Assigned to: Nobody/Anonymous (nobody)
Summary: for lin in file: file.tell() tells wrong

Initial Comment:
Consider following piece of code:

f = file("test.txt", "rb")
pos = f.tell()
for line in f:
	print "%u: '%s'" % (pos, line)
	pos = f.tell()

During the code execution we have following result:

0 'Line 1'
63 'Line 2'
63 'Line 3'
...
63 'Line 9'

However, following piece of code works fine:

f = file("test.txt", "rb")
while True:
	pos = f.tell()
	line = f.readline()
	if line == "":
		break
	print "%u: '%s'" % (pos, line)

It prints:

0 'Line 1'
7 'Line 2'
14 'Line 3'
...
56 'Line 9'

It seems a file iterator makes file.tell() to tell
positions of some internal blocks used by the iterator.

----------------------------------------------------------------------

>Comment By: Dmitry Vasiliev (hdima)
Date: 2003-01-08 13:17

Message:
Logged In: YES 
user_id=388573

I agree. Unfortunately a small patch doesn't work here. :-(
But I suggest to recategorize it as a documentation bug.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 00:25

Message:
Logged In: YES 
user_id=92689

This bug is very similar to #524804, which was closed as
"won't fix". Unless it's recategorized as a documentation
bug, I suggest to close this one as a duplicate.

----------------------------------------------------------------------

Comment By: Dmitry Vasiliev (hdima)
Date: 2002-12-04 14:24

Message:
Logged In: YES 
user_id=388573

File.next() uses readahead buffer (8192 bytes for now).
Calling file.seek() drops the buffer, but other operations
don't. I think it's possible for file.tell() to return right
result (I'll try to make a patch). But all these quirks
should be documented.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-11-30 01:07

Message:
Logged In: YES 
user_id=45365

Actually, all file operations behave the same (well, all: I tried one: readline():-): they behave as if the whole file has been read. I.e. file.readline() within a "for line in file" will return an empty string.

If a file iterator behaves as if it has read the complete file at once on instantiation (never mind what it actually does: if it *behaves* as if this happens) I think this should just be documented and that's it.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-29 19:57

Message:
Logged In: YES 
user_id=31435

Possibly.  Maybe something fancier could be done too, to 
give "expected" results. although I don't know how to 
approach that on Windows for text-mode files (byte 
arithmetic on seek/tell results doesn't work at all for 
those).  I'd take it to Python-Dev.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-29 19:01

Message:
Logged In: YES 
user_id=21627

Shouldn't tell raise an exception then?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-29 18:42

Message:
Logged In: YES 
user_id=31435

"for x in file" does do its own chunking under the covers, for 
speed, and seek() and tell() are indeed not to be used on a 
file being iterated over in this way.

----------------------------------------------------------------------

Comment By: Dmitry Vasiliev (hdima)
Date: 2002-11-29 17:46

Message:
Logged In: YES 
user_id=388573

I'll try to dig in.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-29 17:30

Message:
Logged In: YES 
user_id=21627

Would you like to investigate this further? There is no need 
to, but if you find a solution and a patch, there is a better 
chance that this is fixed before 2.3.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=645594&group_id=5470