[issue13510] Clarify that readlines() is not needed to iterate over a file
New submission from Peter Otten <__peter__@web.de>: I've been looking at code on the tutor mailing list for some time, and for line in file.readlines(): ... is a common idiom there. I suppose this is because the readlines() method is easily discoverable while the proper way (iterate over the file object directly) is not. A note added to the readlines() documentation might help: """ You don't need the readlines() method to loop over the lines of a file. for line in file: process(line) consumes less memory and is often faster. """ ---------- assignee: docs@python components: Documentation messages: 148679 nosy: docs@python, potten priority: normal severity: normal status: open title: Clarify that readlines() is not needed to iterate over a file type: feature request _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +eric.araujo, ezio.melotti stage: -> needs patch versions: +Python 2.7, Python 3.2, Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: This current line is "Read and return a list of lines from the stream. hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint." I would like to see added "Since a file is a line iterator, file.readlines() == list(file). To save time and space, iterate over lines of a file with for line in file: process(line)." (with code markup for the two snippets). ---------- nosy: +terry.reedy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Meador Inge <meadori@gmail.com> added the comment: I am skeptical that such a note will help. The iterator behavior is clearly pointed out in the Python Tutorial [1] and in the IOBase documentation [2]. I suspect this bad code pattern is just being copied and pasted from other sources without folks ever even looking at the Python documentation. [1] http://docs.python.org/dev/tutorial/inputoutput.html?highlight=readlines#met... [2] http://docs.python.org/dev/library/io.html#io.IOBase ---------- nosy: +meador.inge _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Ezio Melotti <ezio.melotti@gmail.com> added the comment: FWIW I've seen several persons using "for line in file.readlines(): ..." or even "while 1: line = file.readline()". IMHO it's a good idea to document that "without sizehint, it's equivalent to list(file)" and that "for line in file: ..." can be used directly. Even if some people don't read the doc, the ones who do will benefit from this. The same note might also be added to the docstring (I think it's somewhat common to learn about readlines() through dir(file) + help(file.readlines)). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Chris Rebert added the comment: file.readlines() (and perhaps dare I say even file.readline()) should not even be mentioned in the tutorial, IMO. It is difficult to imagine a use case where just iterating over the file object isn't superior. I cannot remember the last time that I have used either of these methods. They ought to be relegated to the library docs. Presenting `for line in a_file:` as merely "An alternative approach" in the official tutorial is practically archaic. ---------- nosy: +cvrebert _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Éric Araujo added the comment: readlines might be discouraged; readline has its use cases (two days ago I used it to get one line to pass to csv.Sniffer.sniff), but next(file) works too, so it could be de-emphasized. There may be a difference with respect to the trailing newline however; I don’t remember if __iter__ or readline keep it or not. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- keywords: +easy versions: +Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Changes by Mike Hoy <mhoy09@gmail.com>: ---------- nosy: +mikehoy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Changes by Ashwini Chaudhary <monty.sinngh@gmail.com>: ---------- nosy: +montysinngh _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- versions: -Python 3.2 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Kushal Das added the comment: Working on a patch for this. ---------- nosy: +kushaldas _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Dan Riti added the comment: After reading the comments, I generated a patch that does the following: - Reorganize to present `for line in f:` as the first approach for reading lines. I refrained from saying it's the *preferred* approach, however I can add that if desired. - Reorganize to present `f.readlines()` as the alternative approach. Any feedback is more then welcome! Thanks. ---------- keywords: +patch nosy: +dan.riti Added file: http://bugs.python.org/file29822/demote-readlines.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Antoine Pitrou added the comment: In 3.x I think we'll want to drop the following sentence: "Since the two approaches manage line buffering differently, they should not be mixed". But it can wait for another issue. ---------- nosy: +pitrou _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Ezio Melotti added the comment: I would actually remove the whole section about readlines() or possibly just mention it briefly (something like "If you want to read all the lines of a file in a list you can also use f.readlines().") The sizehint arg is rarely used, so I don't see the point of going in such details about it in the tutorial. In Lib/, there are only a couple of places where it's actually used: Lib/fileinput.py:358: self._buffer = self._file.readlines(self._bufsize) Lib/idlelib/GrepDialog.py:90: block = f.readlines(100000) ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Antoine Pitrou added the comment:
I would actually remove the whole section about readlines() or possibly just mention it briefly (something like "If you want to read all the lines of a file in a list you can also use f.readlines().") The sizehint arg is rarely used, so I don't see the point of going in such details about it in the tutorial.
You are right, Ezio. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Terry J. Reedy added the comment: I agree with removing .readlines section. If .readlines did not exist, I do not think we would add it now. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Dan Riti added the comment: Added a new version of the patch to incorporate Ezio's comment! ---------- Added file: http://bugs.python.org/file29859/demote-readlines-v2.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Ezio Melotti added the comment: Patch LGTM. I think it would also be better to say something like "Note that it's already possible to iterate on file objects using ``for line in file: ...`` without calling file.readlines()." in Doc/library/io.rst:readlines, as suggested in msg148703. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Dan Riti added the comment: Agreed Ezio, I've updated the patch to include the change to Doc/library/io.rst:readlines. ---------- Added file: http://bugs.python.org/file29868/demote-readlines-v3.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Roundup Robot added the comment: New changeset 1e8be05a4039 by Ezio Melotti in branch '3.3': #13510: clarify that f.readlines() is note necessary to iterate over a file. Patch by Dan Riti. http://hg.python.org/cpython/rev/1e8be05a4039 New changeset 7f4325dc4256 by Ezio Melotti in branch 'default': #13510: merge with 3.3. http://hg.python.org/cpython/rev/7f4325dc4256 New changeset 6a4746b0afaf by Ezio Melotti in branch '2.7': #13510: clarify that f.readlines() is note necessary to iterate over a file. Patch by Dan Riti. http://hg.python.org/cpython/rev/6a4746b0afaf ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
Ezio Melotti added the comment: Fixed, thanks for the patch! I also realized I missed Terry suggestion about file.readlines() == list(file), so I added that too. ---------- assignee: docs@python -> ezio.melotti resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13510> _______________________________________
participants (12)
-
Antoine Pitrou -
Ashwini Chaudhary
-
Chris Rebert
-
Dan Riti
-
Ezio Melotti
-
Kushal Das
-
Meador Inge
-
Mike Hoy
-
Peter Otten
-
Roundup Robot
-
Terry J. Reedy
-
Éric Araujo