About open file for Read

Peter Otten __peter__ at web.de
Mon Dec 10 18:40:15 CET 2012

Dave Angel wrote:

> On 12/10/2012 11:36 AM, moonhkt wrote:
>> Hi All
>> I am new in Python. When using open and then for line in f .
>> Does it read all the data into f object ? or read line by line ?
>>   f=open(file, 'r')
>>            for line in f:
>>               if userstring in line:
>>                  print "file: " + os.path.join(root,file)
>>                  break
>>            f.close()
>> moonhk
> open() does not read the whole file into any object.  There is buffering
> that goes on in the C libraries that open() calls, but that should be
> transparent to you for regular files.
> When you ask for a line, it'll read enough to fulfill that request, and
> maybe some extra that'll get held somewhere in the C runtime library.
> You should look into the 'with' statement, to avoid that f.close().
> That way the file will be closed, regardless of whether you get an
> exception or not.
> http://docs.python.org/2/reference/compound_stmts.html#index-15
>     with open(file,. "r") as f:
>         for line in f:
>              etc.
> BTW, since you're in version 2.x, you should avoid hiding the builtin
> file object.  Call it something like file_name, or infile_name.

Python does a bit of buffering on its own (which is why you cannot mix file 
iteration and .readline() calls):

>>> with open("tmp.txt", "w") as f: f.writelines("%s\n" % i for i in 
>>> f = open("tmp.txt")
>>> f.readline()
>>> f.tell()
>>> f.readline()
>>> f.tell()
>>> next(f) # a for-loop does this implicitly
>>> f.tell()
8196 # after a next() call or the first loop iteration
     # part of the file is now in a buffer.
>>> f.readline()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Mixing iteration and read methods would lose data
>>> f.seek(0, 2)
>>> f.tell()

This is Python 2, in Python 3 f.tell() would fail after a next(f) call, but 
f.readline() continues to work.

More information about the Python-list mailing list