On Sun, May 22, 2016 at 6:45 PM, Steven D'Aprano wrote:

Looking at the docs, there are nine examples given of things which can raise InvalidOperation (if not trapped, in which case they return a NAN):

Infinity - Infinity 0 * Infinity Infinity / Infinity x % 0 Infinity % x sqrt(-x) and x > 0 0 ** 0 x ** (non-integer) x ** Infinity

plus invalid string conversion. To my mind, ValueError would be an acceptable error for all of these things.

Hmm. Out of interest, I probed fractions.Fraction for comparable behaviours. AFAIK, there's no way to represent infinity as a Fraction; using a floating-point infinity just converts the Fraction to a float and continues as per float; and mixing Fraction and Decimal either instantly raises TypeError, or (in the case of x**infinity) converts to float and *then* raises TypeError. 0 ** 0 simply returns 1. I can't get a ValueError out of any of the given operations. Why is x ** (non-integer) an InvalidOperation? Is that specifically for a negative value for x? The example you quoted [1] isn't clear there. (I also don't know why it describes the square root example that way, and not as "sqrt(x) and x < 0", but presumably there's a good mathematical reason for that.)

So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.

+1. ChrisA [1] https://docs.python.org/3/library/decimal.html#decimal.InvalidOperation

On Sun, May 22, 2016 at 10:15:16AM +0000, Stefan Krah wrote:

Steven D'Aprano writes:

...

So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.

One problem is that often a *combination* of values triggers InvalidOperation. ValueError does not really fit here.

I don't see why it matters whether it is one value or a combination of values, it's still a value error.

In this scenario other modules use TypeError, but I'm not a fan of that use either:

...

...

...

Decimal("inf") * Decimal(0) Traceback (most recent call last): File "<stdin>", line 1, in <module> decimal.InvalidOperation: []

I'm not sure what you mean by this example, and what it has to do with TypeError. The error shown is related to the *value* of the arguments, namely Decimal inf and Decimal zero, not their types. Both arguments are Decimal.

...

...

...

Decimal("inf") * Decimal("inf") Decimal('Infinity')

Again, I don't see the relevance of this example. Multiplying Decimal infinity by another Decimal infinity is well-defined, and not an error.

Additionally, InvalidOperation is part of Decimal's custom exception handling via the context, so it would be a bit odd if ValueError were part of that.

Perhaps I am mistaken, but none of Decimal's custom exception handling would need to change. That's the beauty of multiple inheritence. If we add ValueError as a base class, InvalidOperation is a subclass of both DecimalException and ValueError. The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied. -- Steve

On 23 May 2016 at 00:35, Stefan Krah wrote:

Steven D'Aprano writes: [snipping excessively, since gmane.org started to lecture on this.]

...

The custom exception handling will continue to work the same way it does now, and people who expect Decimal("spam") to raise ValueError will also be satisfied.

They won't be if they do setcontext(ExtendedContext).

At that point they've explicitly asked for unknown strings to be converted to "NaN", and the behaviour won't be changed by Steven's proposal. However, while I can see Steven's point (folks expect to be able to catch malformed strings with ValueError), I don't believe the InvalidOperation would be the right place to adjust the hierarchy, as in terms of base classes, the decimal exceptions already inherit from ArithmeticError, which is the common base class of ZeroDivisionError, OverflowError, and FloatingPointError (which I would consider the closest analogy to decimal.InvalidOperation for binary floating point values), and those have the same behaviour of bypassing except clauses that catch ValueError:

...

...

def to_float(arg): ... try: ... return float(arg) ... except ValueError: ... return float("nan") ... to_float(10*100) 1000.0 to_float("spam") nan to_float(10**10000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float

To address *just* the case of string conversion, it would suffice to introduce a dedicated decimal.ParsingError exception inheriting from both InvalidOperation and ValueError, specifically for failures converting from a string to a decimal value. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

...

Steven D'Aprano writes:

...

So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.

Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError? If it did, it would fix this issue with Decimal and possibly others as well. I ran into this a while ago with a function like def convert(x, t): try: return t(x) except ValueError: ... and found that not all numeric types raise something derived from ValueError from their constructors. -- Greg

Greg Ewing writes:

...

...

...

So I propose that InvalidOperation be changed to inherit from ValueError, to match the expected behaviour from other numeric types.

Related to this, is there any good reason that ArithmeticError doesn't derive from ValueError?

For Decimal at least, Overflow, Underflow, Clamped etc. don't really map to ValueError. This includes InvalidOperation, which unfortunately *already* has a subclass ConversionSyntax for the use case that came up here. ConversionSyntax, however, is designated by the standard to be a "condition" rather than a "signal", so ConversionSyntax isn't raised -- it just exists, and in the C version you can see the "condition" in the list next to the exception that was raised. Adding a ValueError to all that just complicates things further. I wouldn't mind raising ConversionSyntax directly, but then again no one knows about that either and it's a small deviation from the standard. Stefan Krah

I've ignored this thread. Do you need help? --Guido (mobile)

On Tue, May 24, 2016 at 08:51:46AM -0700, Guido van Rossum wrote:

I've ignored this thread. Do you need help?

It might help if you ruled the change out completely, said it needs a PEP, or that you didn't care what we did so long as we don't break the module :-) I'd also want to see Raymond's and Tim's comments before any changes were made. But if you rule it out, no need to bother them. -- Steve

On 25 May 2016 at 02:56, Guido van Rossum wrote:

If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement.

Recapping the thread, the specific problem Steven raised was that catching ValueError will handle "float('not-a-valid-float')", but not "Decimal('not-a-valid-float')", as the latter raises decimal.InvalidOperation, which inherits from ArithmeticError rather than ValueError. There are 3 possible ways of changing that: - make ArithmeticError a subclass of ValueError - make decimal.InvalidOperation inherit from both ArithmeticError & ValueError - raise a subclass of InvalidOperation that also inherits from ValueError for decimal string conversions The last one is clearly the lowest risk and lowest impact way to allow ValueError to be used to catch decimal string conversion problems, so in the absence of any other considerations, I'd just say "Let's do that". However, what came up in the course of the discussion is that those of us participating in the thread don't actually know the original rationale for ArithmeticError being entirely distinct from ValueError, rather than being a subclass of it, which means there are other cases where folks may be expecting ValueError to catch all conversion errors, but it may in fact be missing some. As a toy example:

...

...

def to_float(x): ... try: ... return float(x) ... except ValueError: ... return float("nan") ... to_float("not-a-float") nan to_float(10**1000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float from fractions import Fraction class LazyFraction: ... def __float__(self): ... return float(Fraction(1, 0)) ... to_float(LazyFraction()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float File "<stdin>", line 3, in __float__ File "/usr/lib64/python3.5/fractions.py", line 186, in __new__ raise ZeroDivisionError('Fraction(%s, 0)' % numerator) ZeroDivisionError: Fraction(1, 0)

Before this thread, I would have said that the above "to_float()" function would reliably return either the converted value or NaN in the absence of coding bugs in a __float__ method implementation, but the OverflowError and ZeroDivisionError cases above show that I would have been wrong about that, and a more comprehensive exception clause would be "except (ValueError, ArithmeticError):". If you rule out turning ArithmeticError into a ValueError subclass in 3.6+, then we can go back to looking specifically at the decimal string conversion behaviour. However, if you thought the builtin exception hierarchy change was an idea worth considering further, then it would address decimal string conversions as a side effect (since those are already an ArithmeticError subclass). Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Guido van Rossum wrote:

It's very common for modules to define their own root exception class.

And it's a nuisance when their exceptions only belong to their own private hierarchy and don't participate in any the standard classifications. I don't think Decimal is directly to blame here, because deriving its exceptions from ArithmeticError seems like a reasonable thing to do. What *doesn't* seem reasonable to me is that ArithmeticError doesn't derive from ValueError. So far nobody has explained why that's a good idea. -- Greg

On Tue, May 24, 2016 at 5:26 PM, Nick Coghlan wrote:

On 25 May 2016 at 02:56, Guido van Rossum wrote:

...

If it's the right thing to do we should do it, and consider how to do it without breaking too much code too soon (or without clear indication of failure). I don't think a PEP is needed unless there's a lot of discussion about alternatives or disagreement.

Recapping the thread, the specific problem Steven raised was that catching ValueError will handle "float('not-a-valid-float')", but not "Decimal('not-a-valid-float')", as the latter raises decimal.InvalidOperation, which inherits from ArithmeticError rather than ValueError.

There are 3 possible ways of changing that:

- make ArithmeticError a subclass of ValueError - make decimal.InvalidOperation inherit from both ArithmeticError & ValueError - raise a subclass of InvalidOperation that also inherits from ValueError for decimal string conversions

FWIW, here is the exception hierarchy that I use in gmpy2. I've ignored the standard base classes, etc. gmpy2.InvalidOperationError - gmpyError - ArithmeticError - ValueError gmpy2.DivisionByZeroError - gmpyError - ZeroDivisionError - ArithmeticError gmpy2.InexactResultError - gmpyError - ArithmeticError gmpy2.OverflowResultError - InexactResultError - gmpyError - ArithmeticError gmpy2.UnderflowResultError - InexactResultError - gmpyError - ArithmeticError gmpy2.RangeError - gmpyError - ArithmeticError

The last one is clearly the lowest risk and lowest impact way to allow ValueError to be used to catch decimal string conversion problems, so in the absence of any other considerations, I'd just say "Let's do that".

Of the 3 options proposed by Nick, the first option would change gmpy2.RangeError. The second option corresponds to the current approach used by gmpy2.InvalidOperation. The third option introduces a new hierarchy.

However, what came up in the course of the discussion is that those of us participating in the thread don't actually know the original rationale for ArithmeticError being entirely distinct from ValueError, rather than being a subclass of it, which means there are other cases where folks may be expecting ValueError to catch all conversion errors, but it may in fact be missing some. As a toy example:

...

...

...

def to_float(x): ... try: ... return float(x) ... except ValueError: ... return float("nan") ... to_float("not-a-float") nan to_float(10**1000) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float OverflowError: int too large to convert to float from fractions import Fraction class LazyFraction: ... def __float__(self): ... return float(Fraction(1, 0)) ... to_float(LazyFraction()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in to_float File "<stdin>", line 3, in __float__ File "/usr/lib64/python3.5/fractions.py", line 186, in __new__ raise ZeroDivisionError('Fraction(%s, 0)' % numerator) ZeroDivisionError: Fraction(1, 0)

Before this thread, I would have said that the above "to_float()" function would reliably return either the converted value or NaN in the absence of coding bugs in a __float__ method implementation, but the OverflowError and ZeroDivisionError cases above show that I would have been wrong about that, and a more comprehensive exception clause would be "except (ValueError, ArithmeticError):".

If you rule out turning ArithmeticError into a ValueError subclass in 3.6+, then we can go back to looking specifically at the decimal string conversion behaviour. However, if you thought the builtin exception hierarchy change was an idea worth considering further, then it would address decimal string conversions as a side effect (since those are already an ArithmeticError subclass).

I've used the following guidelines for exceptions: 1) A TypeError is raised when there is no possible way to associate a meaning to an instance of that type. 2) A ValueError is raised when at least some instances of the given type can have a valid meaning but the particular value passed has no possible valid interpretation. 3) An ArithmeticError (or subclass) is raised when arithmetic operations on a valid input fail for a particular reason. Some examples: int(float) raises a TypeError because there is no possible way to assign a numeric value to a type. int(float("nan")) raises a ValueError because there is no way to perform arithmetic operations on NaN. int(float("inf")) raises OverflowError because while you can perform arithmetic operation on Infinity, it will eventually overflow. Following my guidelines, it makes sense that decimal.Decimal("not-valid") would raise a (subclass of) ValueError. casevh

Regards, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Guido van Rossum writes:

Well, ZeroDivisionError doesn't derive from ValueError, does it? So it may not be a good idea, but it's certainly a tradition.

I think it's deeper than merely tradition (though that doesn't make it a good idea, either). One way to look at these exceptions is that you shouldn't apply that operation to that value (ArithmeticError), and another is that you shouldn't pass that value to that operation (ValueError). A distinction without a difference, I suppose, after thinking about how I would apply it to Python. Eg (and most tellingly, I think) as Python implements division, first you ask the numerator if it knows how to divide itself by something. It says "sure, here's my method," the method asks "what number?", and *then* when you tell it "zero", it says, "sorry, I don't do zero!" and that's a ValueError. The example above is sort of an artifact of division being a binary operation, so even with an OO implementation you end up passing a value to function that then decides it doesn't like it. In a language like Haskell, where a function always takes one argument, and returns a new function if the program wants to handle more than one argument, the distinction might make more sense. But I don't see a reason to pursue that line of thought here, so I won't even try.

Rob Cliffe writes:

What I was trying to say was: I think of

ValueError as applying to a single ("bad") value

whereas ArithmeticError can arise from trying to combine multiple ("good") values.

I thought about that, and concluded that (1) a function taking multiple arguments is semantically equivalent to a function taking a single tuple as argument, and (2) offhand, I don't know of any errors of computation that require knowledge of more than one argument to recognize (and it's always the same argument). I realize that there are alternative ways of interpreting those facts (eg, in the implementation you will take the tuple apart and operate on the components as "multiple entities"). However, I found myself looping between those two, one or the other covered everything that wasn't a TypeError in the first place.

It wouldn't be appropriate for ArithmeticError to be a subclass of ValueError, because there is no single erroneous value that you can point to.

Well, I *can* point to the *tuple*. You need to argue why I shouldn't, in view of the fact that the subclassing is correct (not only is the tuple as value invalid, the subclassing tells me that it's not just one of the components, but an interaction of more than one). Examples would help. ZeroDivisionError tells you that it's exactly one of the arguments by its name. Feeding a complex as either argument to atan2 gives a TypeError. Feeding a zero as either argument to atan2 gives a (correct!) answer (atan2(x,y) is not the same as atan(y/x), which raises ZeroDivisionError before atan is actually called). Feeding two zeros to atan2 gives an answer (but I'm way too far from 9th grade to remember if that's correct -- perhaps that "should" raise an ArithmeticError on your interpretation). Steve