"try with" syntactic sugar

Around two-thirds of the time, whenever I use the wonderful new "with" construct, it's enclosed in a "try-except" block, like this: try: with something as f: many lines of code except some_error: handle error The "with" statement is great, but it results in the bulk of the code being indented twice. I'd like to propose a little syntactic sugar, the "try with": try with something as f: many lines of code except some_error: handle error It saves one line of vertical space, and gets rid of an indentation level for the bulk of the code that rests within the "with" statement. Thoughts? Here's a short script to count how many uses of "with" within your code are immediately preceded by "try": #!/usr/bin/python import re, sys re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with ') try_with = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.findall(data): if match: try_with += 1 total += 1 print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)' Usage: Cashew:~$ /tmp/count_try_with.py *.py try-with: 17 out of: 25 ( 68.0 %) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

+0 But... Do we also add try for:, try if:, try while:, etc.? Not terrible, but potentially terrible. On Thu, Feb 26, 2009 at 9:36 AM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

On Thu, Feb 26, 2009 at 8:44 AM, Calvin Spealman <ironfroggy@gmail.com>wrote:
But... Do we also add try for:, try if:, try while:, etc.? Not terrible, but potentially terrible.
No, because they're rarely useful. Modifying my script a little, less than 1% of my "if" and "for" statements are immediately preceded by "try", compared to 68% of my "with" statements. "with" blocks very frequently define the scope of an object that might generate a particular type of exception (e.g., file object may throw IOErrors, SQL transactions might through SQL exceptions, etc.). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote: ...
Not commenting on how useful the proposal is (I'm mostly stuck in Python 2.4), but couldn't this be: with something as f: many lines of code except some_error: handle error That is, you don't really need the "try", since the above syntax is currently illegal? Eric.

On Thu, Feb 26, 2009 at 8:36 AM, Daniel Stutzbach < daniel@stutzbachenterprises.com> wrote:
re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with ')
Here's an updated regular expression that does a better job of ignoring comments and strings: re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with .*:') -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
#!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent try: (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P<with> (?P=indent)[ \t]+ # at higher indent level with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) )? # end (?P<with>) ''', re.MULTILINE | re.VERBOSE) try_with = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('with'): try_with += 1 total += 1 print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)' When I run this on a project of mine, I get: try-with: 1 out of: 87 ( 1.14942528736 %) The pattern that I find myself using is with/for, which can be counted by this program: #!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P<for> (?P=indent)[ \t]+ # at higher indent level for\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # for .*: (ignoring comments # and newlines before :) )? # end (?P<for>) ''', re.MULTILINE | re.VERBOSE) with_for = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('for'): with_for += 1 total += 1 print 'with-for:', with_for, 'out of:', total, '(', with_for*100.0/total,'%)' On my code, I get: with-for: 38 out of: 47 ( 80.8510638298 %) A few days ago, I proposed that a __throw__ method be added to context managers so that context managers could be used to capture common try/except usage patterns along with it's current ability to capture common try/finally patterns. I am curious whether you see common try/except patterns that could be captured in a context manager so that you could simply write: with my_except_handling(something) as x: many lines of code rather than: try: with something as f: many lines of code except some_error: handle error and be able to replace several occurrences of try/except/handle error with the same my_except_handling context manager? -bruce frederiksen

Hi Bruce, I bow to your superior regular expression knowledge. :) However, your version counts the number of "try"s that are followed by "with". Mine counts the number of "with"s that are preceded by "try" (which I think the more relevant metric?). Any chance you could alter your script? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
Sorry, my mistake. How about this? #!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent # (might be try, might be with) (?P<try> try: (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P=indent)[ \t]+ # kick out the indent level )? with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) ''', re.MULTILINE | re.VERBOSE) try_with = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('try'): try_with += 1 total += 1 print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)' On my code, I now get: try-with: 1 out of: 47 ( 2.12765957447 %) On my last response, I mentioned a suggestion to add __throw__ to context managers. But then I remembered that the __exit__ method is already given the exception information if an exception is raised. So you can already do what I was suggesting now. I'm still curious as to how often you could share try/except cases by writing your own context managers. -bruce frederiksen

On Thu, Feb 26, 2009 at 7:28 PM, Bruce Frederiksen <dangyogi@gmail.com>wrote:
How about this?
I tried it, but it matches quite a few comments and strings. Try putting a "print repr(match.group(0))" in the innermost loop to debug it.
Not particularly often. Much of the time, the exception handler has to clean up the mess when a file is unexpectedly unreadable or SQL explodes, and the clean-up bit is tied to the immediately surrounding code. YMMV. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

En Fri, 27 Feb 2009 00:00:59 -0200, Daniel Stutzbach <daniel-k6jgQluAQ1RLuhoIm4CQ5lNBC6VQM+k9@public.gmane.org> escribió:
I think a r.e. cannot handle this query well enough. This script uses the tokenize module and should be immune to those false positives (but it's much slower) <code> import sys, os from tokenize import generate_tokens from token import NAME def process(path): nfiles = nwith = ntrywith = 0 for base, dirs, files in os.walk(path): print base print '%d "try+with" out of %d "with" (%.1f%%) in %d files (partial)' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) print for fn in files: if fn[-3:]!='.py': continue if 'CVS' in dirs: dirs.remove('CVS') if '.svn' in dirs: dirs.remove('.svn') fullfn = os.path.join(base, fn) #print fn nfiles += 1 with open(fullfn) as f: try: was_try = False for toknum, tokval, _, _, _ in generate_tokens(f.readline): if toknum==NAME: is_with = tokval == 'with' if is_with: nwith += 1 if was_try: ntrywith += 1 was_try = tokval == 'try' except Exception, e: print e print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) process(sys.argv[1]) </code> I got 2/25 on my code (1500 files, but many are rather old to use "with") -- Gabriel Genellina

On Fri, Feb 27, 2009 at 5:14 AM, Gabriel Genellina <gagsl-py2@yahoo.com.ar>wrote:
Thanks, Gabriel. Using the tokenize module is, indeed, much better. I think the performance problems were caused by the O(n**2) cost of reading through the directories and then removing them selectively. I modified it to have an O(n) cost and it's quite snappy. #!/usr/bin/python from __future__ import with_statement import sys, os from tokenize import generate_tokens from token import NAME def process(paths): nfiles = nwith = ntrywith = 0 for path in paths: for base, dirs, files in os.walk(path): if nfiles: print '%d "try+with" out of %d "with" (%.1f%%) in %d files (so far)\ '% (ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) print base newdirs = [] for d in list(dirs): if d == 'CVS' or d == '_darcs' or d[0] == '.': continue newdirs.append(d) dirs[:] = newdirs for fn in files: if fn[-3:]!='.py': continue fullfn = os.path.join(base, fn) #print fn nfiles += 1 with open(fullfn) as f: try: was_try = False for toknum, tokval, _, _, _ in generate_tokens(f.readline): if toknum==NAME: is_with = tokval == 'with' if is_with: nwith += 1 if was_try: ntrywith += 1 was_try = tokval == 'try' except Exception, e: print e print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) process(sys.argv[1:]) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
<another way to get to possibly relevant code>
It seems to me the question is, among the single-statement try blocks, what kinds of (non-function call, non-assignment) statements are the inner statement, and what frequency does this occur. On Python5, 25, and 30, I get quite similar numbers: 2.5 2.6 3.0 10 15 14 with 15 11 13 yield 25 26 22 class 32 23 *8 print # print is a function in 3.0, called 8 times 37 33 24 while 44 35 *26 exec # exec is a function in 3.0, called 26 times 9 42 41 pass 49 44 64 raise 67 45 29 del 95 61 40 for 142 68 42 if 182 105 63 from 230 163 107 import 588 307 150 return These numbers are pulled from the output of the attached imperfect program, and hand-editted together. The numbers seem to me to say that try-for and try-while substantially exceed try-with statements. --Scott David Daniels @Acm.Org import os import collections __version__ = '0.5' def hunt_try(*roots): singles = multi = count = 0 starting = collections.defaultdict(int) for root in roots: for b,d,f in os.walk(os.path.expanduser(root)): d.sort() for name in sorted(nm for nm in f if nm.endswith(('.py', '.pyw'))): with open(os.path.join(b, name)) as src: gen = enumerate(src) for n, line in gen: text = line.lstrip() if text.startswith('try:'): outer = line[: - len(text)] count += 1 for n1, line1 in gen: lst = line1.lstrip() if lst and not lst.startswith('#'): break else: if name is not None: print src.name name = None print 'Broken at %s: %s..%s' % (n, n1) continue indent = line1[: -len(lst)] if indent == outer and lst.startswith('except' ) or lst.startswith('finally:'): singles += 1 kind = line.split(None, 2)[1] starting[kind] += 1 continue for n2, line2 in gen: lst = line2.lstrip() if lst and not lst.startswith('#'): if line2.startswith(indent): if not line2[len(indent)].isspace(): multi += 1 break # multi-try elif line2.startswith(outer) and not line2[ len(outer)].isspace(): kind = line1.split(None, 1)[0] starting[kind] += 1 singles += 1 break else: if name is not None: print src.name name = None print 'Broken at %s..%s..%s' % ( n, n1, n2) break else: if name is not None: print src.name name = None print 'final try broken: %s..%s..%s' % ( n, n1, n2) return singles, multi, count, starting if __name__ == '__main__': import sys singles, multi, count, sng = hunt_try(*sys.argv[1:] or [r'C:\Python30']) print 'v%s, %s singles + %s multis + %s confused = %s "try:"s' % ( __version__, singles, multi, count - singles - multi, count) for pair in sorted((n, nm) for nm, n in sng.iteritems() if n > 9): print '%4d %s' % pair

2009/2/26 Daniel Stutzbach <daniel@stutzbachenterprises.com>:
Every compound statement could be made into an implicit try, i.e. <compound statement>: <suite1> except: <suite2> would mean: try: <compound statement>: <suite1> except: <suite2> So you could write: with something as f: many lines of code except some_error: handle error -- Arnaud

On Fri, Feb 27, 2009 at 2:49 AM, Arnaud Delobelle <arnodel@googlemail.com> wrote:
Every compound statement could be made into an implicit try, i.e.
That way lies madness. What distinguishes "with" from other compound statements is that it's already about resource management in the face of possible exceptions. -- Curt Hagenlocher curt@hagenlocher.org

On Fri, Feb 27, 2009 at 4:49 AM, Curt Hagenlocher <curt@hagenlocher.org> wrote:
Still, a firm -1 from me. Once we have "try try" I'm sure people are going to clamor for "try if", "try while", "try for", even (oh horror :-) "try try". I don't think we should complicate the syntax just to save one level of indentation occasionally. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, Feb 27, 2009 at 1:20 PM, Guido van Rossum <guido@python.org> wrote:
In addition to reasons outlined by Curt, "with" is unique because it's short-hand for a "try" block with a "finally" clause. Unfortunately, "with" doesn't allow for other clauses and so I often end up using both "try" and "with". Also, "try if", "try while", and "try for" wouldn't work because they already have a meaning for the "else" clause. "with" does not. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On Sun, Mar 1, 2009 at 12:49 PM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
Sorry, but my gut keeps telling me that "try with" is not taking the language into a direction I am comfortable with. Programming language design is not a rational science. Most reasoning about is is at best rationalization of gut feelings, and at worst plain wrong. So, sorry, but I'm going with my gut feelings, so it's still -1. (Or if you wish, -1000.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sun, Mar 1, 2009 at 4:31 PM, Guido van Rossum <guido@python.org> wrote:
I thought that might be the case based on your first response, but figured I'd give it one more shot. ;-) I respect your opinion. Consider it dropped. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On Sun, Mar 1, 2009 at 2:53 PM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
Thanks. I've learned the hard way to trust my gut instincts in this area. If anyone can figure out how to explain my gut's responses to people wanting rational answers I'd love to hear about it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Le Sun, 1 Mar 2009 14:31:58 -0800, Guido van Rossum <guido@python.org> s'exprima ainsi:
! ;-) Denis ------ la vita e estrany PS: Time for starting a "Quotes" section at http://en.wikipedia.org/wiki/Guido_van_Rossum ?

On Sun, Mar 1, 2009 at 11:16 PM, spir <denis.spir@free.fr> wrote:
+1 QOTW! - Chris -- Shameless self-promotion: http://blog.rebertia.com

On Sun, Mar 1, 2009 at 11:18 PM, Chris Rebert <pyideas@rebertia.com> wrote:
Let me add that this formulation was at least in part inspired by reading "How we decide" by Jonah Lehrer. http://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0618620117 -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Daniel Stutzbach wrote
Your proposal sounds like a very good idea. +1 from me. I like to push your proposal even further and add the full set of exept, else, finally blocks to the with statement. Additionally I'd like to have multiple arguments, too. Example ------- log.info("Starting copy") with somelock, open(infile, 'r') as fin, open(outfile, 'w') as fout: fout.write(fin.read()) except Excption: log.exception("An error has occured") else: log.info("Copy successful") finally: log.info("All done") Equivalent ---------- log.info("Starting copy") try: with somelock: with open(infile, 'r') as fin: with open(outfile, 'w') as fout: fout.write(fin.read()) except Excption: log.exception("An error has occured") else: log.info("Copy successful") finally: log.info("All done") Christian

Christian Heimes schrieb:
I'd assumed this is already implicit in the proposal.
Additionally I'd like to have multiple arguments, too.
+1 to that (it should have been there in the beginning).
I still like it better with the "try" before the "with". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

+0 But... Do we also add try for:, try if:, try while:, etc.? Not terrible, but potentially terrible. On Thu, Feb 26, 2009 at 9:36 AM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

On Thu, Feb 26, 2009 at 8:44 AM, Calvin Spealman <ironfroggy@gmail.com>wrote:
But... Do we also add try for:, try if:, try while:, etc.? Not terrible, but potentially terrible.
No, because they're rarely useful. Modifying my script a little, less than 1% of my "if" and "for" statements are immediately preceded by "try", compared to 68% of my "with" statements. "with" blocks very frequently define the scope of an object that might generate a particular type of exception (e.g., file object may throw IOErrors, SQL transactions might through SQL exceptions, etc.). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote: ...
Not commenting on how useful the proposal is (I'm mostly stuck in Python 2.4), but couldn't this be: with something as f: many lines of code except some_error: handle error That is, you don't really need the "try", since the above syntax is currently illegal? Eric.

On Thu, Feb 26, 2009 at 8:36 AM, Daniel Stutzbach < daniel@stutzbachenterprises.com> wrote:
re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with ')
Here's an updated regular expression that does a better job of ignoring comments and strings: re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with .*:') -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
#!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent try: (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P<with> (?P=indent)[ \t]+ # at higher indent level with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) )? # end (?P<with>) ''', re.MULTILINE | re.VERBOSE) try_with = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('with'): try_with += 1 total += 1 print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)' When I run this on a project of mine, I get: try-with: 1 out of: 87 ( 1.14942528736 %) The pattern that I find myself using is with/for, which can be counted by this program: #!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P<for> (?P=indent)[ \t]+ # at higher indent level for\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # for .*: (ignoring comments # and newlines before :) )? # end (?P<for>) ''', re.MULTILINE | re.VERBOSE) with_for = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('for'): with_for += 1 total += 1 print 'with-for:', with_for, 'out of:', total, '(', with_for*100.0/total,'%)' On my code, I get: with-for: 38 out of: 47 ( 80.8510638298 %) A few days ago, I proposed that a __throw__ method be added to context managers so that context managers could be used to capture common try/except usage patterns along with it's current ability to capture common try/finally patterns. I am curious whether you see common try/except patterns that could be captured in a context manager so that you could simply write: with my_except_handling(something) as x: many lines of code rather than: try: with something as f: many lines of code except some_error: handle error and be able to replace several occurrences of try/except/handle error with the same my_except_handling context manager? -bruce frederiksen

Hi Bruce, I bow to your superior regular expression knowledge. :) However, your version counts the number of "try"s that are followed by "with". Mine counts the number of "with"s that are preceded by "try" (which I think the more relevant metric?). Any chance you could alter your script? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
Sorry, my mistake. How about this? #!/usr/bin/python import re, sys re_with = re.compile(r''' ^(?P<indent>\s*) # capture the indent # (might be try, might be with) (?P<try> try: (?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments) (?P=indent)[ \t]+ # kick out the indent level )? with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments # and newlines before :) ''', re.MULTILINE | re.VERBOSE) try_with = 0 total = 0 for fname in sys.argv[1:]: data = open(fname).read() for match in re_with.finditer(data): if match.group('try'): try_with += 1 total += 1 print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)' On my code, I now get: try-with: 1 out of: 47 ( 2.12765957447 %) On my last response, I mentioned a suggestion to add __throw__ to context managers. But then I remembered that the __exit__ method is already given the exception information if an exception is raised. So you can already do what I was suggesting now. I'm still curious as to how often you could share try/except cases by writing your own context managers. -bruce frederiksen

On Thu, Feb 26, 2009 at 7:28 PM, Bruce Frederiksen <dangyogi@gmail.com>wrote:
How about this?
I tried it, but it matches quite a few comments and strings. Try putting a "print repr(match.group(0))" in the innermost loop to debug it.
Not particularly often. Much of the time, the exception handler has to clean up the mess when a file is unexpectedly unreadable or SQL explodes, and the clean-up bit is tied to the immediately surrounding code. YMMV. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

En Fri, 27 Feb 2009 00:00:59 -0200, Daniel Stutzbach <daniel-k6jgQluAQ1RLuhoIm4CQ5lNBC6VQM+k9@public.gmane.org> escribió:
I think a r.e. cannot handle this query well enough. This script uses the tokenize module and should be immune to those false positives (but it's much slower) <code> import sys, os from tokenize import generate_tokens from token import NAME def process(path): nfiles = nwith = ntrywith = 0 for base, dirs, files in os.walk(path): print base print '%d "try+with" out of %d "with" (%.1f%%) in %d files (partial)' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) print for fn in files: if fn[-3:]!='.py': continue if 'CVS' in dirs: dirs.remove('CVS') if '.svn' in dirs: dirs.remove('.svn') fullfn = os.path.join(base, fn) #print fn nfiles += 1 with open(fullfn) as f: try: was_try = False for toknum, tokval, _, _, _ in generate_tokens(f.readline): if toknum==NAME: is_with = tokval == 'with' if is_with: nwith += 1 if was_try: ntrywith += 1 was_try = tokval == 'try' except Exception, e: print e print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) process(sys.argv[1]) </code> I got 2/25 on my code (1500 files, but many are rather old to use "with") -- Gabriel Genellina

On Fri, Feb 27, 2009 at 5:14 AM, Gabriel Genellina <gagsl-py2@yahoo.com.ar>wrote:
Thanks, Gabriel. Using the tokenize module is, indeed, much better. I think the performance problems were caused by the O(n**2) cost of reading through the directories and then removing them selectively. I modified it to have an O(n) cost and it's quite snappy. #!/usr/bin/python from __future__ import with_statement import sys, os from tokenize import generate_tokens from token import NAME def process(paths): nfiles = nwith = ntrywith = 0 for path in paths: for base, dirs, files in os.walk(path): if nfiles: print '%d "try+with" out of %d "with" (%.1f%%) in %d files (so far)\ '% (ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) print base newdirs = [] for d in list(dirs): if d == 'CVS' or d == '_darcs' or d[0] == '.': continue newdirs.append(d) dirs[:] = newdirs for fn in files: if fn[-3:]!='.py': continue fullfn = os.path.join(base, fn) #print fn nfiles += 1 with open(fullfn) as f: try: was_try = False for toknum, tokval, _, _, _ in generate_tokens(f.readline): if toknum==NAME: is_with = tokval == 'with' if is_with: nwith += 1 if was_try: ntrywith += 1 was_try = tokval == 'try' except Exception, e: print e print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % ( ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles) process(sys.argv[1:]) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

Daniel Stutzbach wrote:
<another way to get to possibly relevant code>
It seems to me the question is, among the single-statement try blocks, what kinds of (non-function call, non-assignment) statements are the inner statement, and what frequency does this occur. On Python5, 25, and 30, I get quite similar numbers: 2.5 2.6 3.0 10 15 14 with 15 11 13 yield 25 26 22 class 32 23 *8 print # print is a function in 3.0, called 8 times 37 33 24 while 44 35 *26 exec # exec is a function in 3.0, called 26 times 9 42 41 pass 49 44 64 raise 67 45 29 del 95 61 40 for 142 68 42 if 182 105 63 from 230 163 107 import 588 307 150 return These numbers are pulled from the output of the attached imperfect program, and hand-editted together. The numbers seem to me to say that try-for and try-while substantially exceed try-with statements. --Scott David Daniels @Acm.Org import os import collections __version__ = '0.5' def hunt_try(*roots): singles = multi = count = 0 starting = collections.defaultdict(int) for root in roots: for b,d,f in os.walk(os.path.expanduser(root)): d.sort() for name in sorted(nm for nm in f if nm.endswith(('.py', '.pyw'))): with open(os.path.join(b, name)) as src: gen = enumerate(src) for n, line in gen: text = line.lstrip() if text.startswith('try:'): outer = line[: - len(text)] count += 1 for n1, line1 in gen: lst = line1.lstrip() if lst and not lst.startswith('#'): break else: if name is not None: print src.name name = None print 'Broken at %s: %s..%s' % (n, n1) continue indent = line1[: -len(lst)] if indent == outer and lst.startswith('except' ) or lst.startswith('finally:'): singles += 1 kind = line.split(None, 2)[1] starting[kind] += 1 continue for n2, line2 in gen: lst = line2.lstrip() if lst and not lst.startswith('#'): if line2.startswith(indent): if not line2[len(indent)].isspace(): multi += 1 break # multi-try elif line2.startswith(outer) and not line2[ len(outer)].isspace(): kind = line1.split(None, 1)[0] starting[kind] += 1 singles += 1 break else: if name is not None: print src.name name = None print 'Broken at %s..%s..%s' % ( n, n1, n2) break else: if name is not None: print src.name name = None print 'final try broken: %s..%s..%s' % ( n, n1, n2) return singles, multi, count, starting if __name__ == '__main__': import sys singles, multi, count, sng = hunt_try(*sys.argv[1:] or [r'C:\Python30']) print 'v%s, %s singles + %s multis + %s confused = %s "try:"s' % ( __version__, singles, multi, count - singles - multi, count) for pair in sorted((n, nm) for nm, n in sng.iteritems() if n > 9): print '%4d %s' % pair

2009/2/26 Daniel Stutzbach <daniel@stutzbachenterprises.com>:
Every compound statement could be made into an implicit try, i.e. <compound statement>: <suite1> except: <suite2> would mean: try: <compound statement>: <suite1> except: <suite2> So you could write: with something as f: many lines of code except some_error: handle error -- Arnaud

On Fri, Feb 27, 2009 at 2:49 AM, Arnaud Delobelle <arnodel@googlemail.com> wrote:
Every compound statement could be made into an implicit try, i.e.
That way lies madness. What distinguishes "with" from other compound statements is that it's already about resource management in the face of possible exceptions. -- Curt Hagenlocher curt@hagenlocher.org

On Fri, Feb 27, 2009 at 4:49 AM, Curt Hagenlocher <curt@hagenlocher.org> wrote:
Still, a firm -1 from me. Once we have "try try" I'm sure people are going to clamor for "try if", "try while", "try for", even (oh horror :-) "try try". I don't think we should complicate the syntax just to save one level of indentation occasionally. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, Feb 27, 2009 at 1:20 PM, Guido van Rossum <guido@python.org> wrote:
In addition to reasons outlined by Curt, "with" is unique because it's short-hand for a "try" block with a "finally" clause. Unfortunately, "with" doesn't allow for other clauses and so I often end up using both "try" and "with". Also, "try if", "try while", and "try for" wouldn't work because they already have a meaning for the "else" clause. "with" does not. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On Sun, Mar 1, 2009 at 12:49 PM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
Sorry, but my gut keeps telling me that "try with" is not taking the language into a direction I am comfortable with. Programming language design is not a rational science. Most reasoning about is is at best rationalization of gut feelings, and at worst plain wrong. So, sorry, but I'm going with my gut feelings, so it's still -1. (Or if you wish, -1000.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sun, Mar 1, 2009 at 4:31 PM, Guido van Rossum <guido@python.org> wrote:
I thought that might be the case based on your first response, but figured I'd give it one more shot. ;-) I respect your opinion. Consider it dropped. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On Sun, Mar 1, 2009 at 2:53 PM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
Thanks. I've learned the hard way to trust my gut instincts in this area. If anyone can figure out how to explain my gut's responses to people wanting rational answers I'd love to hear about it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Le Sun, 1 Mar 2009 14:31:58 -0800, Guido van Rossum <guido@python.org> s'exprima ainsi:
! ;-) Denis ------ la vita e estrany PS: Time for starting a "Quotes" section at http://en.wikipedia.org/wiki/Guido_van_Rossum ?

On Sun, Mar 1, 2009 at 11:16 PM, spir <denis.spir@free.fr> wrote:
+1 QOTW! - Chris -- Shameless self-promotion: http://blog.rebertia.com

On Sun, Mar 1, 2009 at 11:18 PM, Chris Rebert <pyideas@rebertia.com> wrote:
Let me add that this formulation was at least in part inspired by reading "How we decide" by Jonah Lehrer. http://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0618620117 -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Daniel Stutzbach wrote
Your proposal sounds like a very good idea. +1 from me. I like to push your proposal even further and add the full set of exept, else, finally blocks to the with statement. Additionally I'd like to have multiple arguments, too. Example ------- log.info("Starting copy") with somelock, open(infile, 'r') as fin, open(outfile, 'w') as fout: fout.write(fin.read()) except Excption: log.exception("An error has occured") else: log.info("Copy successful") finally: log.info("All done") Equivalent ---------- log.info("Starting copy") try: with somelock: with open(infile, 'r') as fin: with open(outfile, 'w') as fout: fout.write(fin.read()) except Excption: log.exception("An error has occured") else: log.info("Copy successful") finally: log.info("All done") Christian

Christian Heimes schrieb:
I'd assumed this is already implicit in the proposal.
Additionally I'd like to have multiple arguments, too.
+1 to that (it should have been there in the beginning).
I still like it better with the "try" before the "with". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
participants (15)
-
Alec Thomas
-
Arnaud Delobelle
-
Bruce Frederiksen
-
Calvin Spealman
-
Cesare Di Mauro
-
Chris Rebert
-
Christian Heimes
-
Curt Hagenlocher
-
Daniel Stutzbach
-
Eric Smith
-
Gabriel Genellina
-
Georg Brandl
-
Guido van Rossum
-
Scott David Daniels
-
spir