Should decimal.InvalidOperation subclass ValueError?
On the tutor mailing list, somebody asked a question about decimal.InvalidOperation. They were converting strings to Decimal using something like this: try: d = Decimal(the_string) except ValueError: handle_error() and were perplexed by the error that they got: decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>] This got me thinking. Normally, such a conversion error should raise ValueError, as ints, floats and Fractions do. Presumably decimal does something different as it must match the General Decimal Arithmetic Specification. But, is there any reason why InvalidOperation couldn't be a subclass of ValueError? Exception class hierarchies are not part of the GDAS, so this should be allowed. Looking at the docs, there are nine examples given of things which can raise InvalidOperation (if not trapped, in which case they return a NAN): Infinity - Infinity 0 * Infinity Infinity / Infinity x % 0 Infinity % x sqrt(-x) and x > 0 0 ** 0 x ** (non-integer) x ** Infinity plus invalid string conversion. To my mind, ValueError would be an acceptable error for all of these things. E.g. the root of a negative value raises ValueError in Python 2:
(-2.0)**0.5 Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: negative number cannot be raised to a fractional power
(In Python 3, it returns a complex number.) So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types. The only tricky bit is that division by zero doesn't raise ValueError, but DivideByZeroError instead. But that's no worse than the current situation: # current - people expect Decimal(1)/0 to raise DivideByZeroError, but it raises InvalidOperation; # proposal - people expect Decimal(1)/0 to raise DivideByZero, but it raises InvalidOperation (subclass of ValueError). Oh, a further data point: if you pass an invalid list or tuple to Decimal, you get a ValueError: py> decimal.Decimal([]) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: argument must be a sequence of length 3 Thoughts? -- Steve
On Sun, May 22, 2016 at 6:45 PM, Steven D'Aprano <steve@pearwood.info> wrote:
Looking at the docs, there are nine examples given of things which can raise InvalidOperation (if not trapped, in which case they return a NAN):
Infinity - Infinity 0 * Infinity Infinity / Infinity x % 0 Infinity % x sqrt(-x) and x > 0 0 ** 0 x ** (non-integer) x ** Infinity
plus invalid string conversion. To my mind, ValueError would be an acceptable error for all of these things.
Hmm. Out of interest, I probed fractions.Fraction for comparable behaviours. AFAIK, there's no way to represent infinity as a Fraction; using a floating-point infinity just converts the Fraction to a float and continues as per float; and mixing Fraction and Decimal either instantly raises TypeError, or (in the case of x**infinity) converts to float and *then* raises TypeError. 0 ** 0 simply returns 1. I can't get a ValueError out of any of the given operations. Why is x ** (non-integer) an InvalidOperation? Is that specifically for a negative value for x? The example you quoted [1] isn't clear there. (I also don't know why it describes the square root example that way, and not as "sqrt(x) and x < 0", but presumably there's a good mathematical reason for that.)
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
+1. ChrisA [1] https://docs.python.org/3/library/decimal.html#decimal.InvalidOperation
Steven D'Aprano <steve@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
One problem is that often a *combination* of values triggers InvalidOperation. ValueError does not really fit here. In this scenario other modules use TypeError, but I'm not a fan of that use either:
Decimal("inf") * Decimal(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
Decimal("inf") * Decimal("inf") Decimal('Infinity')
Additionally, InvalidOperation is part of Decimal's custom exception handling via the context, so it would be a bit odd if ValueError were part of that.
py> decimal.Decimal([]) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: argument must be a sequence of length 3
That is a ValueError because in principle list and tuple types are accepted due to the somewhat arcane from_tuple() conversion:
Decimal([0, [], 'F']) Decimal('Infinity')
Since this is a Python specific extension, ValueError looks appropriate here. Count me -1 on any changes here. Stefan Krah
On Sun, May 22, 2016 at 10:15:16AM +0000, Stefan Krah wrote:
Steven D'Aprano <steve@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
One problem is that often a *combination* of values triggers InvalidOperation. ValueError does not really fit here.
I don't see why it matters whether it is one value or a combination of values, it's still a value error.
In this scenario other modules use TypeError, but I'm not a fan of that use either:
Decimal("inf") * Decimal(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
I'm not sure what you mean by this example, and what it has to do with TypeError. The error shown is related to the *value* of the arguments, namely Decimal inf and Decimal zero, not their types. Both arguments are Decimal.
Decimal("inf") * Decimal("inf") Decimal('Infinity')
Again, I don't see the relevance of this example. Multiplying Decimal infinity by another Decimal infinity is well-defined, and not an error.
Additionally, InvalidOperation is part of Decimal's custom exception handling via the context, so it would be a bit odd if ValueError were part of that.
Perhaps I am mistaken, but none of Decimal's custom exception handling would need to change. That's the beauty of multiple inheritence. If we add ValueError as a base class, InvalidOperation is a subclass of both DecimalException and ValueError. The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied. -- Steve
Steven D'Aprano <steve@...> writes: [snipping excessively, since gmane.org started to lecture on this.]
The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied.
They won't be if they do setcontext(ExtendedContext). Stefan Krah
On 23 May 2016 at 00:35, Stefan Krah <stefan@bytereef.org> wrote:
Steven D'Aprano <steve@...> writes: [snipping excessively, since gmane.org started to lecture on this.]
The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied.
They won't be if they do setcontext(ExtendedContext).
At that point they've explicitly asked for unknown strings to be converted to "NaN", and the behaviour won't be changed by Steven's proposal. However, while I can see Steven's point (folks expect to be able to catch malformed strings with ValueError), I don't believe the InvalidOperation would be the right place to adjust the hierarchy, as in terms of base classes, the decimal exceptions already inherit from ArithmeticError, which is the common base class of ZeroDivisionError, OverflowError, and FloatingPointError (which I would consider the closest analogy to decimal.InvalidOperation for binary floating point values), and those have the same behaviour of bypassing except clauses that catch ValueError:
def to_float(arg): ... try: ... return float(arg) ... except ValueError: ... return float("nan") ... to_float(10*100) 1000.0 to_float("spam") nan to_float(10**10000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float
To address *just* the case of string conversion, it would suffice to introduce a dedicated decimal.ParsingError exception inheriting from both InvalidOperation and ValueError, specifically for failures converting from a string to a decimal value. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan <ncoghlan@...> writes:
On 23 May 2016 at 00:35, Stefan Krah <stefan@...> wrote:
Steven D'Aprano <steve <at> ...> writes: [snipping excessively, since gmane.org started to lecture on this.]
The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied.
They won't be if they do setcontext(ExtendedContext).
At that point they've explicitly asked for unknown strings to be converted to "NaN", and the behaviour won't be changed by Steven's proposal.
Of course, but this thread was motivated by a question on the tutor mailing list. Another question "Why isn't ValueError raised if ..." may come up. The answer to all that is of course to read the documentation. :) Stefan Krah
Steven D'Aprano <steve@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError? If it did, it would fix this issue with Decimal and possibly others as well. I ran into this a while ago with a function like def convert(x, t): try: return t(x) except ValueError: ... and found that not all numeric types raise something derived from ValueError from their constructors. -- Greg
On 23 May 2016 at 08:34, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Steven D'Aprano <steve@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError?
If it did, it would fix this issue with Decimal and possibly others as well.
I ran into this a while ago with a function like
def convert(x, t): try: return t(x) except ValueError: ...
and found that not all numeric types raise something derived from ValueError from their constructors.
I considered suggesting that, but got stuck on the question of how we quantify the compatibility implications - such a shift would change the behaviour of code that catches both, but handles them differently. That is, this code would be fine: try: ... except (ArithmeticError, ValueError): ... As would this: try: ... except ArithmeticError: ... except ValueError: ... But this would change to executing the first except clause rather than the second if the inheritance hierarchy changed: try: ... except ValueError: ... except ArithmeticError: ... While I can honestly say I've never seen an except clause catching ArithmeticError directly in the wild, I see more of a risk in the fact we'd potentially be silently changing the way ZeroDivisionError and OverflowError are handled. That said, this feels like the *right* answer to me, it's just the compatibility risk that worries me. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, May 23, 2016 at 12:56:40PM +1000, Nick Coghlan wrote:
On 23 May 2016 at 08:34, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError? [...]
I considered suggesting that, but got stuck on the question of how we quantify the compatibility implications - such a shift would change the behaviour of code that catches both, but handles them differently.
The same applies to decimal.InvalidOperation. Code like: try: ... except ValueError: ... except InvalidOperation: ... would change under my suggestion.
While I can honestly say I've never seen an except clause catching ArithmeticError directly in the wild, I see more of a risk in the fact we'd potentially be silently changing the way ZeroDivisionError and OverflowError are handled.
That said, this feels like the *right* answer to me, it's just the compatibility risk that worries me.
Indeed. -- Steve
On Tue, May 24, 2016 at 08:51:46AM -0700, Guido van Rossum wrote:
I've ignored this thread. Do you need help?
It might help if you ruled the change out completely, said it needs a PEP, or that you didn't care what we did so long as we don't break the module :-) I'd also want to see Raymond's and Tim's comments before any changes were made. But if you rule it out, no need to bother them. -- Steve
If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement. On Tue, May 24, 2016 at 9:19 AM, Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, May 24, 2016 at 08:51:46AM -0700, Guido van Rossum wrote:
I've ignored this thread. Do you need help?
It might help if you ruled the change out completely, said it needs a PEP, or that you didn't care what we did so long as we don't break the module :-)
I'd also want to see Raymond's and Tim's comments before any changes were made. But if you rule it out, no need to bother them.
-- Steve
-- --Guido van Rossum (python.org/~guido)
On 25 May 2016 at 02:56, Guido van Rossum <gvanrossum@gmail.com> wrote:
If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement.
Recapping the thread, the specific problem Steven raised was that catching ValueError will handle "float('not-a-valid-float')", but not "Decimal('not-a-valid-float')", as the latter raises decimal.InvalidOperation, which inherits from ArithmeticError rather than ValueError. There are 3 possible ways of changing that: - make ArithmeticError a subclass of ValueError - make decimal.InvalidOperation inherit from both ArithmeticError & ValueError - raise a subclass of InvalidOperation that also inherits from ValueError for decimal string conversions The last one is clearly the lowest risk and lowest impact way to allow ValueError to be used to catch decimal string conversion problems, so in the absence of any other considerations, I'd just say "Let's do that". However, what came up in the course of the discussion is that those of us participating in the thread don't actually know the original rationale for ArithmeticError being entirely distinct from ValueError, rather than being a subclass of it, which means there are other cases where folks may be expecting ValueError to catch all conversion errors, but it may in fact be missing some. As a toy example:
def to_float(x): ... try: ... return float(x) ... except ValueError: ... return float("nan") ... to_float("not-a-float") nan to_float(10**1000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float from fractions import Fraction class LazyFraction: ... def __float__(self): ... return float(Fraction(1, 0)) ... to_float(LazyFraction()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float File "<stdin>", line 3, in __float__ File "/usr/lib64/python3.5/fractions.py", line 186, in __new__ raise ZeroDivisionError('Fraction(%s, 0)' % numerator) ZeroDivisionError: Fraction(1, 0)
Before this thread, I would have said that the above "to_float()" function would reliably return either the converted value or NaN in the absence of coding bugs in a __float__ method implementation, but the OverflowError and ZeroDivisionError cases above show that I would have been wrong about that, and a more comprehensive exception clause would be "except (ValueError, ArithmeticError):". If you rule out turning ArithmeticError into a ValueError subclass in 3.6+, then we can go back to looking specifically at the decimal string conversion behaviour. However, if you thought the builtin exception hierarchy change was an idea worth considering further, then it would address decimal string conversions as a side effect (since those are already an ArithmeticError subclass). Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Is there really something here that needs to be fixed? It's very common for modules to define their own root exception class. And usually people who care about these kinds of exceptions just catch whatever exception is raised by the combination of argument they care about (e.g. calling float() with a string argument). The Decimal module already doesn't play all that much by the same rules as other modules (e.g. it doesn't participate in the numeric tower) so people trying to write algorithms independent from whether the numbers they're processing are floats, fractions or Decimals are already in a lot of pain, I imagine -- and the best solution is probably to just stick with Decimal (numerical specialists would cringe at attempts to write code without knowing how the arithmetic is done in detail anyways). If there's anything in this area that bugs me it would be that I'd worry that some obscure invalid input string to float() might not raise ValueError but something else (e.g. OverflowError -- while I cannot reproduce this, I've seen lots of code that catches that). But in Decimal I would expect this all to be specified by the standard, so I see no need to lose sleep there. On Tue, May 24, 2016 at 5:26 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 25 May 2016 at 02:56, Guido van Rossum <gvanrossum@gmail.com> wrote:
If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement.
Recapping the thread, the specific problem Steven raised was that catching ValueError will handle "float('not-a-valid-float')", but not "Decimal('not-a-valid-float')", as the latter raises decimal.InvalidOperation, which inherits from ArithmeticError rather than ValueError.
There are 3 possible ways of changing that:
- make ArithmeticError a subclass of ValueError - make decimal.InvalidOperation inherit from both ArithmeticError & ValueError - raise a subclass of InvalidOperation that also inherits from ValueError for decimal string conversions
The last one is clearly the lowest risk and lowest impact way to allow ValueError to be used to catch decimal string conversion problems, so in the absence of any other considerations, I'd just say "Let's do that".
However, what came up in the course of the discussion is that those of us participating in the thread don't actually know the original rationale for ArithmeticError being entirely distinct from ValueError, rather than being a subclass of it, which means there are other cases where folks may be expecting ValueError to catch all conversion errors, but it may in fact be missing some. As a toy example:
def to_float(x): ... try: ... return float(x) ... except ValueError: ... return float("nan") ... to_float("not-a-float") nan to_float(10**1000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float from fractions import Fraction class LazyFraction: ... def __float__(self): ... return float(Fraction(1, 0)) ... to_float(LazyFraction()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float File "<stdin>", line 3, in __float__ File "/usr/lib64/python3.5/fractions.py", line 186, in __new__ raise ZeroDivisionError('Fraction(%s, 0)' % numerator) ZeroDivisionError: Fraction(1, 0)
Before this thread, I would have said that the above "to_float()" function would reliably return either the converted value or NaN in the absence of coding bugs in a __float__ method implementation, but the OverflowError and ZeroDivisionError cases above show that I would have been wrong about that, and a more comprehensive exception clause would be "except (ValueError, ArithmeticError):".
If you rule out turning ArithmeticError into a ValueError subclass in 3.6+, then we can go back to looking specifically at the decimal string conversion behaviour. However, if you thought the builtin exception hierarchy change was an idea worth considering further, then it would address decimal string conversions as a side effect (since those are already an ArithmeticError subclass).
Regards, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-- --Guido van Rossum (python.org/~guido)
Guido van Rossum wrote:
It's very common for modules to define their own root exception class.
And it's a nuisance when their exceptions only belong to their own private hierarchy and don't participate in any the standard classifications. I don't think Decimal is directly to blame here, because deriving its exceptions from ArithmeticError seems like a reasonable thing to do. What *doesn't* seem reasonable to me is that ArithmeticError doesn't derive from ValueError. So far nobody has explained why that's a good idea. -- Greg
Well, ZeroDivisionError doesn't derive from ValueError, does it? So it may not be a good idea, but it's certainly a tradition. On Tuesday, May 24, 2016, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Guido van Rossum wrote:
It's very common for modules to define their own root exception class.
And it's a nuisance when their exceptions only belong to their own private hierarchy and don't participate in any the standard classifications.
I don't think Decimal is directly to blame here, because deriving its exceptions from ArithmeticError seems like a reasonable thing to do. What *doesn't* seem reasonable to me is that ArithmeticError doesn't derive from ValueError. So far nobody has explained why that's a good idea.
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido (mobile)
Or more generally, any operation on multiple values, where each value is valid individually, but the result of the operation is not:
p = Decimal('1E999999999')
q = Decimal('10')
try:
... p*q ... except ArithmeticError: ... print 'ArithmeticError' ... ArithmeticError On 25/05/2016 06:48, Guido van Rossum wrote:
Well, ZeroDivisionError doesn't derive from ValueError, does it? So it may not be a good idea, but it's certainly a tradition.
On Tuesday, May 24, 2016, Greg Ewing <greg.ewing@canterbury.ac.nz <mailto:greg.ewing@canterbury.ac.nz>> wrote:
Guido van Rossum wrote:
It's very common for modules to define their own root exception class.
And it's a nuisance when their exceptions only belong to their own private hierarchy and don't participate in any the standard classifications.
I don't think Decimal is directly to blame here, because deriving its exceptions from ArithmeticError seems like a reasonable thing to do. What *doesn't* seem reasonable to me is that ArithmeticError doesn't derive from ValueError. So far nobody has explained why that's a good idea.
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido (mobile)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
What I was trying to say was: I think of ValueError as applying to a single ("bad") value whereas ArithmeticError can arise from trying to combine multiple ("good") values. It wouldn't be appropriate for ArithmeticError to be a subclass of ValueError, because there is no single erroneous value that you can point to. On 25/05/2016 09:08, Rob Cliffe wrote:
Or more generally, any operation on multiple values, where each value is valid individually, but the result of the operation is not:
p = Decimal('1E999999999')
q = Decimal('10')
try:
... p*q
... except ArithmeticError:
... print 'ArithmeticError'
...
ArithmeticError
On 25/05/2016 06:48, Guido van Rossum wrote:
Well, ZeroDivisionError doesn't derive from ValueError, does it? So it may not be a good idea, but it's certainly a tradition.
On Tuesday, May 24, 2016, Greg Ewing <greg.ewing@canterbury.ac.nz <mailto:greg.ewing@canterbury.ac.nz>> wrote:
Guido van Rossum wrote:
It's very common for modules to define their own root exception class.
And it's a nuisance when their exceptions only belong to their own private hierarchy and don't participate in any the standard classifications.
I don't think Decimal is directly to blame here, because deriving its exceptions from ArithmeticError seems like a reasonable thing to do. What *doesn't* seem reasonable to me is that ArithmeticError doesn't derive from ValueError. So far nobody has explained why that's a good idea.
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido (mobile)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct:http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Rob Cliffe writes:
What I was trying to say was: I think of
ValueError as applying to a single ("bad") value
whereas ArithmeticError can arise from trying to combine multiple ("good") values.
I thought about that, and concluded that (1) a function taking multiple arguments is semantically equivalent to a function taking a single tuple as argument, and (2) offhand, I don't know of any errors of computation that require knowledge of more than one argument to recognize (and it's always the same argument). I realize that there are alternative ways of interpreting those facts (eg, in the implementation you will take the tuple apart and operate on the components as "multiple entities"). However, I found myself looping between those two, one or the other covered everything that wasn't a TypeError in the first place.
It wouldn't be appropriate for ArithmeticError to be a subclass of ValueError, because there is no single erroneous value that you can point to.
Well, I *can* point to the *tuple*. You need to argue why I shouldn't, in view of the fact that the subclassing is correct (not only is the tuple as value invalid, the subclassing tells me that it's not just one of the components, but an interaction of more than one). Examples would help. ZeroDivisionError tells you that it's exactly one of the arguments by its name. Feeding a complex as either argument to atan2 gives a TypeError. Feeding a zero as either argument to atan2 gives a (correct!) answer (atan2(x,y) is not the same as atan(y/x), which raises ZeroDivisionError before atan is actually called). Feeding two zeros to atan2 gives an answer (but I'm way too far from 9th grade to remember if that's correct -- perhaps that "should" raise an ArithmeticError on your interpretation). Steve
Guido van Rossum writes:
Well, ZeroDivisionError doesn't derive from ValueError, does it? So it may not be a good idea, but it's certainly a tradition.
I think it's deeper than merely tradition (though that doesn't make it a good idea, either). One way to look at these exceptions is that you shouldn't apply that operation to that value (ArithmeticError), and another is that you shouldn't pass that value to that operation (ValueError). A distinction without a difference, I suppose, after thinking about how I would apply it to Python. Eg (and most tellingly, I think) as Python implements division, first you ask the numerator if it knows how to divide itself by something. It says "sure, here's my method," the method asks "what number?", and *then* when you tell it "zero", it says, "sorry, I don't do zero!" and that's a ValueError. The example above is sort of an artifact of division being a binary operation, so even with an OO implementation you end up passing a value to function that then decides it doesn't like it. In a language like Haskell, where a function always takes one argument, and returns a new function if the program wants to handle more than one argument, the distinction might make more sense. But I don't see a reason to pursue that line of thought here, so I won't even try.
On Tue, May 24, 2016 at 07:04:46PM -0700, Guido van Rossum wrote:
Is there really something here that needs to be fixed?
I'm not sure. As the OP, I was hoping that there would be a simple and obvious "yes, we can change this". But it's not that simple. It seems to me that the annoyance of the current inconsistent and idiosyncratic exceptions is not obviously greater than the pain of "fixing" the problem. Even if we agreed on what the fix should be. So I am prepared to let this go as a "won't fix": - people who try to guess what exceptions are raised are on shaky ground, especially beginners, and should read the docs or try things out in the REPL; - the current exception hierarchy may, or may not, be illogical, but it would be more painful to change it than live with it. -- Steve
On Tue, May 24, 2016 at 5:26 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 25 May 2016 at 02:56, Guido van Rossum <gvanrossum@gmail.com> wrote:
If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement.
Recapping the thread, the specific problem Steven raised was that catching ValueError will handle "float('not-a-valid-float')", but not "Decimal('not-a-valid-float')", as the latter raises decimal.InvalidOperation, which inherits from ArithmeticError rather than ValueError.
There are 3 possible ways of changing that:
- make ArithmeticError a subclass of ValueError - make decimal.InvalidOperation inherit from both ArithmeticError & ValueError - raise a subclass of InvalidOperation that also inherits from ValueError for decimal string conversions
FWIW, here is the exception hierarchy that I use in gmpy2. I've ignored the standard base classes, etc. gmpy2.InvalidOperationError - gmpyError - ArithmeticError - ValueError gmpy2.DivisionByZeroError - gmpyError - ZeroDivisionError - ArithmeticError gmpy2.InexactResultError - gmpyError - ArithmeticError gmpy2.OverflowResultError - InexactResultError - gmpyError - ArithmeticError gmpy2.UnderflowResultError - InexactResultError - gmpyError - ArithmeticError gmpy2.RangeError - gmpyError - ArithmeticError
The last one is clearly the lowest risk and lowest impact way to allow ValueError to be used to catch decimal string conversion problems, so in the absence of any other considerations, I'd just say "Let's do that".
Of the 3 options proposed by Nick, the first option would change gmpy2.RangeError. The second option corresponds to the current approach used by gmpy2.InvalidOperation. The third option introduces a new hierarchy.
However, what came up in the course of the discussion is that those of us participating in the thread don't actually know the original rationale for ArithmeticError being entirely distinct from ValueError, rather than being a subclass of it, which means there are other cases where folks may be expecting ValueError to catch all conversion errors, but it may in fact be missing some. As a toy example:
def to_float(x): ... try: ... return float(x) ... except ValueError: ... return float("nan") ... to_float("not-a-float") nan to_float(10**1000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float from fractions import Fraction class LazyFraction: ... def __float__(self): ... return float(Fraction(1, 0)) ... to_float(LazyFraction()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float File "<stdin>", line 3, in __float__ File "/usr/lib64/python3.5/fractions.py", line 186, in __new__ raise ZeroDivisionError('Fraction(%s, 0)' % numerator) ZeroDivisionError: Fraction(1, 0)
Before this thread, I would have said that the above "to_float()" function would reliably return either the converted value or NaN in the absence of coding bugs in a __float__ method implementation, but the OverflowError and ZeroDivisionError cases above show that I would have been wrong about that, and a more comprehensive exception clause would be "except (ValueError, ArithmeticError):".
If you rule out turning ArithmeticError into a ValueError subclass in 3.6+, then we can go back to looking specifically at the decimal string conversion behaviour. However, if you thought the builtin exception hierarchy change was an idea worth considering further, then it would address decimal string conversions as a side effect (since those are already an ArithmeticError subclass).
I've used the following guidelines for exceptions: 1) A TypeError is raised when there is no possible way to associate a meaning to an instance of that type. 2) A ValueError is raised when at least some instances of the given type can have a valid meaning but the particular value passed has no possible valid interpretation. 3) An ArithmeticError (or subclass) is raised when arithmetic operations on a valid input fail for a particular reason. Some examples: int(float) raises a TypeError because there is no possible way to assign a numeric value to a type. int(float("nan")) raises a ValueError because there is no way to perform arithmetic operations on NaN. int(float("inf")) raises OverflowError because while you can perform arithmetic operation on Infinity, it will eventually overflow. Following my guidelines, it makes sense that decimal.Decimal("not-valid") would raise a (subclass of) ValueError. casevh
Regards, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Greg Ewing <greg.ewing@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError?
For Decimal at least, Overflow, Underflow, Clamped etc. don't really map to ValueError. This includes InvalidOperation, which unfortunately *already* has a subclass ConversionSyntax for the use case that came up here. ConversionSyntax, however, is designated by the standard to be a "condition" rather than a "signal", so ConversionSyntax isn't raised -- it just exists, and in the C version you can see the "condition" in the list next to the exception that was raised. Adding a ValueError to all that just complicates things further. I wouldn't mind raising ConversionSyntax directly, but then again no one knows about that either and it's a small deviation from the standard. Stefan Krah
On Mon, May 23, 2016 at 08:11:10AM +0000, Stefan Krah wrote:
Greg Ewing <greg.ewing@...> writes:
So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.
Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError?
For Decimal at least, Overflow, Underflow, Clamped etc. don't really map to ValueError.
I think they do. Take Overflow: py> Decimal("1e500000")**2 Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.Overflow: [<class 'decimal.Overflow'>] Obviously that's a problem with the *value*. It's not a type error. If the value were smaller, the calculation wouldn't have overflowed.
This includes InvalidOperation, which unfortunately *already* has a subclass ConversionSyntax for the use case that came up here.
ConversionSyntax, however, is designated by the standard to be a "condition" rather than a "signal", so ConversionSyntax isn't raised -- it just exists, and in the C version you can see the "condition" in the list next to the exception that was raised.
Adding a ValueError to all that just complicates things further. I wouldn't mind raising ConversionSyntax directly, but then again no one knows about that either and it's a small deviation from the standard.
I don't believe the standard says anything about class hierarchies for traps. It explicitly says: "This specification does not define the means by which flags and traps are reset or altered, respectively, or the means by which traps are effected." http://speleotrove.com/decimal/damodel.html#backref.11 As far as the standard itself cares, making InvalidOperation a subclass of ValueError is no better or worse than making InvalidOperation a subclass of Exception. -- Steve
participants (10)
-
Case Van Horsen -
Chris Angelico -
Greg Ewing -
Guido van Rossum -
Guido van Rossum -
Nick Coghlan -
Rob Cliffe -
Stefan Krah -
Stephen J. Turnbull -
Steven D'Aprano