[issue10461] Use with statement throughout the docs
New submission from Éric Araujo <merwok@netwok.org>: The docs contain numerous examples that would trigger resource warnings under 3.2 (for example “open(...).read()”). They should be changed to use (and thus promote) the with statement. Not adding the “easy” keyword, since grepping for those things is not easy. Not sure we’ll backport that to 3.1 and 2.7. ---------- assignee: docs@python components: Documentation messages: 121545 nosy: docs@python, eric.araujo priority: normal severity: normal stage: needs patch status: open title: Use with statement throughout the docs versions: Python 3.2 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Alexander Belopolsky <belopolsky@users.sourceforge.net> added the comment: +1 BTW, I've updated examples in Unicode HOWTO to use with. ---------- nosy: +belopolsky _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: +1 I have not yet had occasion to use 'with' yet, but in reading the Unicode HOWTO diff, I noticed that I liked replacing 'open,read,close' with 'with open, read' just for reading purposes since it turns 3 steps into 1 compound transaction. Perhaps something should also be added to the doc style guide (along with using 'attributes' instead of 'members'). ---------- nosy: +terry.reedy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +ezio.melotti _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: with also replaces open-try-do stuff-finally-close, the correct idiom for ensuring file handles are always closes in all VMs. I don’t think the doc style guide is the right place, since this is a code issue. with is advertised in http://docs.python.org/dev/tutorial/inputoutput#methods-of-file-objects and http://docs.python.org/dev/howto/doanddont#exceptions ; http://docs.python.org/dev/library/functions#open says nothing about closing, and http://docs.python.org/dev/library/io tells about the with statement without recommending it strongly. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: Éric, although grepping for all such references may be tricky, could you specify the places where you did see them? I guess a few fixed places is better than none at all. ---------- nosy: +eli.bendersky _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Certainly. Here is my secret grep: $ cd py3k/Doc $ grep -n --color=auto -d skip -I --exclude-dir .svn --exclude-dir .hg -R open\(.*\).read *.rst c-api distutils documenting extending faq howto library reference tutorial using List without false positives: faq/library.rst library/pkgutil.rst library/atexit.rst library/pipes.rst library/cmd.rst library/logging.rst library/difflib.rst library/collections.rst tutorial/interpreter.rst tutorial/stdlib2.rst List for open(.*).write: faq/library.rst library/ftplib.rst library/atexit.rst There is probably a better regex that would catch “open(.*).[valid method name]”, but I hate regexes. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: Eric, I'm attaching a provisional fix for library/atexit.rst just to be sure this is what you mean. ---------- Added file: http://bugs.python.org/file19659/atexit.rst _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Eli Bendersky <eliben@gmail.com>: ---------- keywords: +patch Added file: http://bugs.python.org/file19660/issue10461.1.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Eli Bendersky <eliben@gmail.com>: Removed file: http://bugs.python.org/file19659/atexit.rst _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Yes, that’s it. Please don’t change whitespace in your patches. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Georg Brandl <georg@python.org> added the comment: FWIW, this doesn't belong in the style guide; it is obvious that the docs should exhibit "best practice" Python code, and using the with statement when opening resources is certainly such a best practice now. ---------- nosy: +georg.brandl _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: Eric, which whitespace change do you refer to. I changed to 4-spaces indentation in the code sample to conform to PEP-8. Shouldn't I have? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: Here is the patch for Docs/library/cmd.rst ---------- nosy: +SilentGhost Added file: http://bugs.python.org/file19663/cmd.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: Here is the patch for Doc/library/difflib.rst ---------- Added file: http://bugs.python.org/file19664/difflib.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: I'm trying to port the example in tutorial/stdlib2.rst, but the sample in "working with binary data record layouts" fails (before my porting to 'with'...) struct.error: unpack requires a bytes argument of length 16 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: Patch for Doc/library/collections.rst ---------- Added file: http://bugs.python.org/file19665/collections.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: Attaching patches for library/atexit.rst and for tutorial/stdlib2.rst ---------- Added file: http://bugs.python.org/file19667/issue10461.atexit.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Eli Bendersky <eliben@gmail.com>: Added file: http://bugs.python.org/file19668/issue10461.stdlib2.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Eli Bendersky <eliben@gmail.com>: Removed file: http://bugs.python.org/file19660/issue10461.1.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: SilentGhost, Your patches look fine. I have a doubt re collections.rst, however - about the Python prompt. The same issue is in faq/library.rst and I didn't want to touch it because I thought that on the prompt personally I probably wouldn't use 'with' - why write 2 lines when you can write 1? After all, it's just playing on the prompt - I don't care about safe closing of the file, etc. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: patch for Doc/library/logging.rst Also, note the the change of the mode from `'r'` to `'rb'`. `data_to_send` is further send through socket and therefore requires to be bytes. I expressed my opinion in irc, but I can repeat here that I think only the most trivial code such as in Doc/library/pipes.rst isn't worth converting. ---------- Added file: http://bugs.python.org/file19671/logging.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: patch for Doc/library/logging.rst Also, note the the change of the mode from `'r'` to `'rb'`. `data_to_send` is further send through socket and therefore requires to be bytes. I expressed my opinion in irc, but I can repeat here that I think only the most trivial code such as in Doc/library/pipes.rst isn't worth converting. ---------- Added file: http://bugs.python.org/file19672/logging.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by SilentGhost <michael.mischurow+bpo@gmail.com>: Removed file: http://bugs.python.org/file19672/logging.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment: Do not change the examples in collections.rst. That would interfere with the clarify of what is being demonstrated. For example, the hamlet.txt example is not about file reading, it about counting words. ---------- nosy: +rhettinger _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: None of the changes are about file reading, they only about using with statement throughout. May be nofix then? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment: I think the other (non collections patches) are fine. The change doesn't break up the flow of the text. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Alexander Belopolsky <belopolsky@users.sourceforge.net> added the comment: FWIW, I find the "with" variant much easier to read than
words = re.findall('\w+', open('hamlet.txt').read().lower())
Too many operations in one line. It would be even better with
words = re.findall('\w+', hamlet_text.lower())
---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment: Separate note for Éric: the try/finally examples do not need to change. Those are valid python. Users need to learn both try/finally and the with-statement. ---------- assignee: docs@python -> rhettinger _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Alexander Belopolsky <belopolsky@users.sourceforge.net>: ---------- Removed message: http://bugs.python.org/msg121711 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Eli, SilentGhost: Please open other bug reports for doc errors like the struct one in stdlib2 or the r/rb one. SilentGhost: with statement used with open *is* about file reading :) Raymond: I agree about try/finally. I agree with Alexander about collections.rst. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: On Sat, Nov 20, 2010 at 20:44, Éric Araujo <report@bugs.python.org> wrote:
Éric Araujo <merwok@netwok.org> added the comment:
Eli, SilentGhost: Please open other bug reports for doc errors like the struct one in stdlib2 or the r/rb one.
SilentGhost: with statement used with open *is* about file reading :)
Raymond: I agree about try/finally. I agree with Alexander about collections.rst.
Éric, It turned out to be a non-issue, never mind. ---------- Added file: http://bugs.python.org/file19696/unnamed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment: Éric, please apply most of these. In the atexit patch, the first change is wrong. Change it to a try/except/finally or skip it altogether. In the collections patch, only include the change for the tail example; the other two should remain unchanged. Skip the difflib patch entirely. It unnecessarily makes the example confusing and hard to follow. Change the cmd patch to: + with open(arg) as f: + self.cmdqueue.extend(f.read().splitlines()) ---------- assignee: rhettinger -> eric.araujo resolution: -> accepted stage: needs patch -> commit review _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by Éric Araujo <merwok@netwok.org>: Removed file: http://bugs.python.org/file19696/unnamed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Raymond: I made a single diff for easier review. I’m not 100% sure I should change 'r' to 'rb' in logging: It’s unrelated to with, and the rest of the file has not been checked for similar errors. I prefer to do commits that don’t mix things. Eli:
which whitespace change do you refer to. I changed to 4-spaces indentation in the code sample to conform to PEP-8. Shouldn't I have? PEP 8 only applies to Python code. http://docs.python.org/dev/documenting/style applies to the docs.
---------- Added file: http://bugs.python.org/file19765/with-in-docs-1.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Eli Bendersky <eliben@gmail.com> added the comment: Éric, Yes, in a consequent patch I fixed this - kept the formatted code indented at 3 spaces, while adhering to PEP-8 internally. If there's anything else, let me know. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Raymond Hettinger <rhettinger@users.sourceforge.net> added the comment:
I’m not 100% sure I should change 'r' to 'rb' in logging: It’s unrelated to with, and the rest of the file has not been checked for similar errors.
Good catch. This should only be a with-statement transformation. I caught other accidental semantic changes, so be continue to exercise care. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Changes by SilentGhost <michael.mischurow+bpo@gmail.com>: Removed file: http://bugs.python.org/file19671/logging.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <michael.mischurow+bpo@gmail.com> added the comment: following re-organization of the logging docs, I'm attaching updated patch. ---------- nosy: +vinay.sajip Added file: http://bugs.python.org/file20112/logging-cookbook.rst.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: Vinay, you should look at the logging-cookbook patch. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Vinay Sajip <vinay_sajip@yahoo.co.uk> added the comment: I've already made the change, Terry, just holding off committing it because Georg has frozen the py3k branch until 3.2b2 is released. There are a few other changes I'm making, will commit these soon after 3.2b2 is released. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment:
Perhaps something should also be added to the doc style guide (along with using 'attributes' instead of 'members').
Terry, could you open another report for that, or maybe just commit the change directly? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Roundup Robot <devnull@devnull> added the comment: New changeset c3d28c84a372 by Éric Araujo in branch 'default': Use with statement where it improves the documentation (closes #10461) http://hg.python.org/cpython/rev/c3d28c84a372 ---------- nosy: +python-dev resolution: accepted -> fixed status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
SilentGhost <ghost.adh@gmail.com> added the comment: Éric, I'm not sure about the first atexit.rst hunk. I thought the purpose of that code was to except IOError when no file is found. I'm not entirely sure why in the simplest case infile.read() would raise IOError. I'd think that eli's patch (file19667) would suit better. ---------- status: closed -> pending _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Roundup Robot <devnull@devnull> added the comment: New changeset cebaf1bdaf78 by Éric Araujo in branch 'default': Fix example in atexit doc: Both open and read could raise the IOError (#10461 follow-up). http://hg.python.org/cpython/rev/cebaf1bdaf78 ---------- status: pending -> open _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Thanks for the catch! I was so used to the idiom where opening the file doesn’t fail but you protect the read or write that I didn’t think about the issue here. ---------- stage: commit review -> committed/rejected status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue10461> _______________________________________
participants (10)
-
Alexander Belopolsky
-
Eli Bendersky
-
Ezio Melotti
-
Georg Brandl
-
Raymond Hettinger
-
Roundup Robot
-
SilentGhost
-
Terry J. Reedy
-
Vinay Sajip
-
Éric Araujo