exception based conditional expression, similar to if-else conditional expression
I would like to propose an expression, similar to the if-else expression, that responds to exceptions. I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting. _*Abstract: *_Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions. _*Motivation: *_An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions. _*Very Simple Example - type coercion: *_Current approach: try: x = float(string) except: x = float('nan') Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan') _*Simple Example - type coercion in a list comprehension: *_Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings) Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings) _*Discussion: *_In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.) Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.) I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys) Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?). In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller. I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language. Jeff McAninch -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
I like it. I'm not fully competent to comment on the details, but a big +1 for something like this, if not this. -T On Thu, Aug 20, 2009 at 9:07 AM, Jeff McAninch <mcaninch@lanl.gov> wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions.
I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting.
*Abstract: *Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions.
*Motivation: *An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions.
*Very Simple Example - type coercion: *Current approach: try: x = float(string) except: x = float('nan')
Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan')
*Simple Example - type coercion in a list comprehension: *Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings)
*Discussion: *In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.)
Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.)
I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?).
In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression
Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller.
I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language.
Jeff McAninch
-- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"
-1 on colons in the expression like that. I like the idea of being able to handle an exception in generator expressions and the like, but I've never seen a syntax I liked. I think I've favored the idea of something like `float(x) except float('nan') if ValueError` thinking it reads more naturally as an expression, puts the real logic ("convert x to a float or get a NaN float") together, which I think makes sense. I fear being able to "express" too much. That is, we're going to remove all that indentation gives us if eventually everything is given an expression form. Let's add an import expression, next. Maybe this? sys.exit(1) from sys Joke, obviously. On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch<mcaninch@lanl.gov> wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions.
I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting.
Abstract: Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions.
Motivation: An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions.
Very Simple Example - type coercion: Current approach: try: x = float(string) except: x = float('nan')
Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan')
Simple Example - type coercion in a list comprehension: Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings)
Discussion: In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.)
Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.)
I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?).
In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression
Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller.
I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language.
Jeff McAninch
-- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch<mcaninch@lanl.gov> wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions.
This is a pretty good idea. I can definitely see where I would use it in my own code. The only thing I would caution is in your simple example. I would choose to create a module full of conversion functions. It violates DRY to have the 'try float otherwise Nan' statement all over the place. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek
Jeff McAninch writes:
_*Very Simple Example - type coercion: *_Current approach: try: x = float(string) except: x = float('nan')
Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan')
-1 I really don't like the colon in the middle of the expression. It just looks like broken syntax because everywhere else in Python colon introduces a suite. I wonder if you really need it? I generally don't like a general conditional expression. For reasons presented below, I don't think it's as flexible as think it can be. OTOH, I fund it much less readable than the existing statement-based construction.
_*Simple Example - type coercion in a list comprehension: *_Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
I don't have a problem with the above. But I can see that for
In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x":
an exception-handling clause in comprehensions (and generator expressions?) might be useful, to make the exceptions handled explicit. For the cases I can imagine using myself, I would generally prefer the approach of defining functions to handle the exceptions, because they'd all be similar (eg, coercing exceptions to float("nan") as above), though.
try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression
But it seems to me the ez-read interpretation of x = expression0 except exception1: expression1 except: default_expression is x = expression0 except exception1: (expression1 except: default_expression) Ie, your parser resolves the "dangling except" ambiguity in the opposite way to the conventional resolution of the "dangling else" ambiguity. And what if excepts are mixed with conditional expressions? x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4 Does the bare except bind to exp0, the if expression, or exp3? I'm sure you can define rules to disambiguate. However, I suspect that it will be hard to get agreement that any given set of rules is the appropriate way to resolve the ambiguity.
Stephen J. Turnbull wrote:
Ie, your parser resolves the "dangling except" ambiguity in the opposite way to the conventional resolution of the "dangling else" ambiguity. And what if excepts are mixed with conditional expressions?
x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4
Does the bare except bind to exp0, the if expression, or exp3? I'm sure you can define rules to disambiguate. However, I suspect that it will be hard to get agreement that any given set of rules is the appropriate way to resolve the ambiguity.
Following up on my previous post, I think the general syntax would be something like: exception_expression :== nominal_value {except exception_tuple exception_value}* {except default_value} Hopefully this disambiguates the issue, at least from the parser point of view. Requiring an explicit tuple of exceptions (ie., requiring the parens around the tuple of exceptions) makes sense too, and I thought I had seen other discussions of requiring explicit tuples where currently parens could be implied. With the definition above, I think the combinations of exception expressions with if-else conditional expressions is probably also unambiguous (though not necessarily easily read). I haven't sat down to verify this mathematically yet though. Allowing parens around the except clauses might help to make it more readable? x = float(string) (except (ValueError,) float('nan')) (except (SomeOtherException,) someOtherValue) Okay, maybe not? (BTW: thanks for the cool new verb -- disambiguate -- though my friends and family may not be so happy when I start using it at every opportunity!) -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
I would like to propose an expression, similar to the if-else expression, that responds to exceptions. -1, if this goes through it will be extended to all other compound statements over time and we'll end up with 2 or 3 ways to do every
On 20 Aug 2009, at 01:07 , Jeff McAninch wrote: thing. A much better idea would be to find a way to make all compound statements into expressions, future-proofing the decision and avoiding the redundancy between "compound statement statement" and "compound statement expression".
On Thu, 20 Aug 2009 09:07:54 am Jeff McAninch wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions. ... _*Simple Example - type coercion in a list comprehension: *_Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
Personally, I think the above is likely to be the approach in a serious application. For lightweight applications, an expression-based solution would be acceptable, but it has the disadvantage of not being able to be documented or tested independently.
Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings)
Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression: EXPR except EXCEPT-LIST EXCEPT-EXPR As far as I can tell, that would be unprecedented in Python syntax. ...
Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ]
Ouch! That last one is better written as: xs = list(xs)
This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets.
Chaining iterators together is a powerful technique, and you don't have to be limited to generator expressions, you can use regular generators too: def operate(xs): for x in xs: try: yield operation_C(x) except ValueError: yield operation_D(x) xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = operate(xs) # finally convert to a list xs = list(xs)
One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
So far this is an argument for iterator-based processing, not for exception-based conditionals :) You can use exceptions in generator exceptions, you just have to put them into a function first.
I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
Is that real code? I don't think it works the way you think. Given a reasonable definition for y (an instance of a class with no special tricks) the if test will never succeed:
class MyObject(object): ... def evaluate(self, arg): ... return arg ... y = MyObject() y.evaluate(42) 42 'evaluate' in y.__dict__ False
A better way of writing that would be: x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y which have the bonus of working correctly even if y gets its evaluate() method via inheritance. Another way, two lines instead of one: try: x = y.evaluate(locals()) except AttributeError: x = y A third method: make sure all objects you pass have an evaluate() method, even if they are a null-operation that just return self.
This seems not very "Pythonic", similar to using type-testing instead of try-except.
You're talking about the Look Before You Leap technique rather than Easier To Ask Forgiveness Than Permission. Both techniques are valid, and both have advantages. ETAFTP is advantageous when exceptions are rare and hard to predict; LBYL is better if exceptions are common and easy to predict.
In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression
I would argue strongly that allowing a bare except is a SERIOUS mistake. Bare excepts have very few legitimate uses, and now that string exceptions are deprecated and very rare, it's no hardship to write: except Exception: if you want to catch everything except KeyboardInterrupt and one or two other things, or: except BaseException: for those rare times you actually do want to catch everything. -- Steven D'Aprano
On 20 Aug 2009, at 16:25 , Steven D'Aprano wrote:
A better way of writing that would be:
x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y
which have the bonus of working correctly even if y gets its evaluate() method via inheritance.
Another way, two lines instead of one:
try: x = y.evaluate(locals()) except AttributeError: x = y
A third method: make sure all objects you pass have an evaluate() method, even if they are a null-operation that just return self.
Fourth method: x = getattr(y, 'evaluate', lambda _: y)(locals())
Steven D'Aprano wrote:
Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression:
EXPR except EXCEPT-LIST EXCEPT-EXPR
I agree. Even if it can be parsed unambiguously, it's going to seem weird and confusing to a human. So far I haven't seen anything I like better than <expr> except <value> if <exception> despite the fact that it uses the words 'except' and 'if' in a different way than elsewhere. -- Greg
First, I'd like to thank you for taking time to write this as a very concise post. Your idea sounds interesting. But I don't think I saw any reasonable (= readable) syntax. Anything with bare semicolon is clearly confusing because bare semicolon has a well-defined meaning as ending a line and starting new block. Anything with bare except clause isn't good because catching all exceptions is a wrong style. Anything that encourages the use of \ line continuation character clearly contradicts the well-established guideline that this character should be used as a last resort. -1 on either of above suggestions. But there are deeper problems even in the "simple example". To see them, let me show how this example should look in programming style I strive for: def safe_float(string: 'any string', invalid = float('nan')) -> 'float': '''Wrapper around float() that converts all invalid values to given param. Use with care --- this will also catch invalid inputs and other cases where notifying the user with exception is a better idea. ''' try: return float(string) except ValueError: return invalid ... class tests: def testSafeFloat(self): assert safe_float('5.7') == float('5.7') assert safe_float('invalid') == float('nan') ... # Here string comes from an experiment and I know it's either empty or should # be valid. xs1 = (float(string or 'nan') for string in strings) # Here string comes from a user input. Tell him if he types something wrong. for string in strings: try: xs2[i] = float(string) catch ValueError: string = input('Please retype string #{}'.format(i)) ... # Here strings are a result of series of experiments. Unfortunately we don't # know what format the researchers used for 'no data' or 'invalid data'. So # for now we'll just change all of these values to 'nan'. xs3 = (safe_float(string) for string in strings) Your proposed solution using exception-based conditional expression: # Here strings are a result of series of experiments. Unfortunately we don't # know what format the researchers used for 'no data' or 'invalid data'. So # for now we'll just change all of these values to 'nan'. # But, unfortunately, if in the future we'll decide to use some different # function to process data, we'll have to rewrite this whole cycle... xs = ((float(string) except ValueError: float('nan')) for string in strings) I think from my examples you get what I'm trying to say -- when you're thinking about easily catching exceptions, you're either ok with something simpler or you're actually trying to apply a function. So why not do it explicitely? Ilya. On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch<mcaninch@lanl.gov> wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions.
I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting.
Abstract: Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions.
Motivation: An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions.
Very Simple Example - type coercion: Current approach: try: x = float(string) except: x = float('nan')
Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan')
Simple Example - type coercion in a list comprehension: Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings)
Discussion: In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.)
Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.)
I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?).
In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression
Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller.
I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language.
Jeff McAninch
-- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Greg Ewing wrote:
Steven D'Aprano wrote:
Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression:
EXPR except EXCEPT-LIST EXCEPT-EXPR
I agree. Even if it can be parsed unambiguously, it's going to seem weird and confusing to a human.
So far I haven't seen anything I like better than
<expr> except <value> if <exception>
despite the fact that it uses the words 'except' and 'if' in a different way than elsewhere.
Possibly the only way to do this within those constraints is a purely syntactic. value = (expr1 :exception: expr2) Sense most people (and editors) recognize exceptions names easily, there really isn't a strong need to use the key word "except" in an expression like this. The colons in this case can be considered to be similar to slice syntax. And if the colons are still considered bad, possibly some other symbol could be used. Ron
On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote:
Greg Ewing wrote:
Steven D'Aprano wrote:
Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression:
EXPR except EXCEPT-LIST EXCEPT-EXPR
I agree. Even if it can be parsed unambiguously, it's going to seem weird and confusing to a human.
So far I haven't seen anything I like better than
<expr> except <value> if <exception>
despite the fact that it uses the words 'except' and 'if' in a different way than elsewhere.
Possibly the only way to do this within those constraints
Which constraints? Are you referring to the fact that any proposed syntax must be capable of being unambiguously parsed?
is a purely syntactic.
I'm not sure I understand. Can you give an example of a proposal which *isn't* syntactic?
value = (expr1 :exception: expr2)
Are the brackets intended to be part of the syntax, or are they there just for grouping? In other words, would this be allowed? value = function(expr1 :exception: expr2) Or would we have to write this? value = function((expr1 :exception: expr2))
Sense most people (and editors) recognize exceptions names easily, there really isn't a strong need to use the key word "except" in an expression like this.
The word "except" cues in the reader that they are seeing an exception-expression, rather than some sort of data structure like {key:value} or (a, tuple). For the same reason, list comprehensions use "for": [x+1 for x in alist] rather than mathematical notation: {x+1 ∀ x ∈ alist} (if that doesn't display for you, the first odd character is an upside down A, "for all", and the second is a curved E, "element of"). If you don't like unicode-based source code, we could replace the "for all" and "element of" symbols with colons: [x+1 : x : alist] but we don't because the words "for" and "in" give the reader hints as to what is being done, which a colon would not. Words are just symbols, so anytime you have syntax including a keyword, you could replace the keyword with a symbol. That way leads to APL. As a general rule, keywords are more readable, punctuation is more concise. -- Steven D'Aprano
I like your use case, but it appears to me a processing task that repeats a lot deserves its own function, so: xs = apply(operation_N, {exceptionN: operation_Nprime}, xs) which also happens to be shorter than xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) Similarly, multiline try/except as in your last example x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression is certainly useful. But we're lucky, nearly identical syntax already exists out of the box! try: _ = expression0 except exception1: _ = expression1 except exception2: _ = expression2 ... except exceptionI: _ = expressionI ... except: _ = default_expression x = _ What's so good about the your snippet that is bad about mine? Anyway, instead of either of those I would prefer # In most cases expression0 returns well-formed foo. try: _ = expression0 except exception1: # Take care of the case of bar being not frobnicable. _ = expression1 except exception2: # This looks like magic, but is isn't. See documentation for # exception2 at http:// some_url for explanation. _ = expression2 ... except exceptionI: _ = expressionI ... except exceptionLast: _ = expressionLast # All cases I could think of are exhausted! Of course, anything can # happen so any other exception will propagate up. x = _ On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch<mcaninch@lanl.gov> wrote:
I would like to propose an expression, similar to the if-else expression, that responds to exceptions.
I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting.
Abstract: Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions.
Motivation: An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions.
Very Simple Example - type coercion: Current approach: try: x = float(string) except: x = float('nan')
Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan')
Simple Example - type coercion in a list comprehension: Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings)
Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings)
Discussion: In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.)
Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.)
I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?).
In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression
Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller.
I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language.
Jeff McAninch
-- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch@lanl.gov ==========================
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Steven D'Aprano wrote:
On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote:
Greg Ewing wrote:
Steven D'Aprano wrote:
Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression:
EXPR except EXCEPT-LIST EXCEPT-EXPR I agree. Even if it can be parsed unambiguously, it's going to seem weird and confusing to a human.
So far I haven't seen anything I like better than
<expr> except <value> if <exception>
despite the fact that it uses the words 'except' and 'if' in a different way than elsewhere. Possibly the only way to do this within those constraints
Which constraints? Are you referring to the fact that any proposed syntax must be capable of being unambiguously parsed?
And also to not using 'except' and 'if' in a different way than elsewhere.
is a purely syntactic.
I'm not sure I understand. Can you give an example of a proposal which *isn't* syntactic?
Technically you are correct, but I was trying to refer to patterns that do not use keywords to identify the operation. For example slice operations use a pattern rather than a unique symbol to identify the operation. So this is an example of a pattern that could work.
value = (expr1 :exception: expr2)
Are the brackets intended to be part of the syntax, or are they there just for grouping? In other words, would this be allowed?
Yes, I intended the brackets to be part of the pattern.
value = function(expr1 :exception: expr2)
Or would we have to write this?
value = function((expr1 :exception: expr2))
I supose it could work like generator expressions do. Then both of these could be equivalent.
Sense most people (and editors) recognize exceptions names easily, there really isn't a strong need to use the key word "except" in an expression like this.
The word "except" cues in the reader that they are seeing an exception-expression, rather than some sort of data structure like {key:value} or (a, tuple).
True, it needs to be unique enough to be easily identifiable or it would not be worth it.
For the same reason, list comprehensions use "for":
[x+1 for x in alist]
rather than mathematical notation:
{x+1 ∀ x ∈ alist}
It displayed fine. :-)
(if that doesn't display for you, the first odd character is an upside down A, "for all", and the second is a curved E, "element of"). If you don't like unicode-based source code, we could replace the "for all" and "element of" symbols with colons:
[x+1 : x : alist]
I think having the square brackets with the colons would be a bit too close to slice notation here. ie... it's not unique enough to be easily identifiable.
but we don't because the words "for" and "in" give the reader hints as to what is being done, which a colon would not.
This is especially true for more complex operations such as looping and repeating.
Words are just symbols, so anytime you have syntax including a keyword, you could replace the keyword with a symbol. That way leads to APL. As a general rule, keywords are more readable, punctuation is more concise.
Yes, I agree, but python isn't that strict about everything. To me a well defined exception expression that All the previous suggestions view the term as needing three values, so you need two separators along with unique symbols (or keywords) to come up with something that fits the need in a clear and concise way. I was thinking what if we could use the exception object itself as an operator, then the term becomes even simpler and more concise. But the only way I can think to do that in a clear way is to use syntax to identify the exception, such as putting colons, or some other symbol around the exception. <shrug> Ron
Ron Adam wrote:
Steven D'Aprano wrote:
Words are just symbols, so anytime you have syntax including a keyword, you could replace the keyword with a symbol. That way leads to APL. As a general rule, keywords are more readable, punctuation is more concise.
Yes, I agree, but python isn't that strict about everything. To me a well defined exception expression that
Editing fragment... Disregard the incomplete sentence.
All the previous suggestions view the term as needing three values, so you need two separators along with unique symbols (or keywords) to come up with something that fits the need in a clear and concise way. I was thinking what if we could use the exception object itself as an operator, then the term becomes even simpler and more concise. But the only way I can think to do that in a clear way is to use syntax to identify the exception, such as putting colons, or some other symbol around the exception. <shrug>
An additional note, the when you view an exception object in the console it is bracketed by the less than and and greater than signs. Maybe there is a way to use those in a manner that is consistent with the string repr() returns. Ron
Ron Adam wrote:
An additional note, the when you view an exception object in the console it is bracketed by the less than and and greater than signs. Maybe there is a way to use those in a manner that is consistent with the string repr() returns.
There won't be unless you rule out using "x > y" as an expression that gives the default value. But doing that will make Python's grammar way too complicated, and you're still left with the problem of "x, y = a < b, b > c" which is legal today. It's my understanding that Ruby has run into a lot of ambiguity with its use of | as both a bitwise operator and a part of a block definition. This seems worse.
On Sun, 23 Aug 2009 05:53:33 am Ron Adam wrote:
An additional note, the when you view an exception object in the console it is bracketed by the less than and and greater than signs. Maybe there is a way to use those in a manner that is consistent with the string repr() returns.
Lots of things have a repr() which includes < and > signs, including repr() itself.
print repr <built-in function repr>
Also new style classes and instances, types, functions, methods, etc. Exceptions are printed with < > signs because exceptions are subclasses of object. There's nothing specifically "exception-like" about the use of < and > in the repr of exceptions. -- Steven D'Aprano
participants (11)
-
Calvin Spealman -
Carl Johnson -
David Stanek -
Greg Ewing -
ilya -
Jeff McAninch -
Masklinn -
Ron Adam -
Stephen J. Turnbull -
Steven D'Aprano -
Tennessee Leeuwenburg