[issue20481] Clarify type coercion rules in statistics module

[Wolfgang Maier]

Wolfgang Maier added the comment:

Just to make sure that this discussion is not getting on the wrong track,
there are currently two strict requirements for any numeric type to be usable with statistics._sum:

(1) the type has to provide either
    - numerator/denominator properties or
    - an as_integer_ratio method or
    - an as_tuple method that mimicks the Decimal method of that name

    this requirement comes from statistics._exact_ratio.

This is where, for example, the sympy numeric types fail because they do not provide any of these interfaces.


****************
    For completeness, this is the code of this function:

def _exact_ratio(x):
    """Convert Real number x exactly to (numerator, denominator) pair.

    >>> _exact_ratio(0.25)
    (1, 4)

    x is expected to be an int, Fraction, Decimal or float.
    """
    try:
        try:
            # int, Fraction
            return (x.numerator, x.denominator)
        except AttributeError:
            # float
            try:
                return x.as_integer_ratio()
            except AttributeError:
                # Decimal
                try:
                    return _decimal_to_ratio(x)
                except AttributeError:
                    msg = "can't convert type '{}' to numerator/denominator"
                    raise TypeError(msg.format(type(x).__name__)) from None
    except (OverflowError, ValueError):
        # INF or NAN
        if __debug__:
            # Decimal signalling NANs cannot be converted to float :-(
            if isinstance(x, Decimal):
                assert not x.is_finite()
            else:
                assert not math.isfinite(x)
        return (x, None)

*****************

(2) Essentially, the numerator and the denominator returned by _exact_ratio have to be valid arguments for the Fraction constructor.
This is a consequence of this block of code in _sum:

    for d, n in sorted(partials.items()):
        total += Fraction(n, d)

Of note, Fraction(n, d) requires both arguments to be members of numbers.Rational and this is where, for example, the gmpy.mpq type fails.

(3) The type's constructor has to work with a Fraction argument.
    This is because _sum tries to

    return T(total) # where T is the coerced type and total the calculated sum as a Fraction
The gmpy.mpq type, for example, fails at this step again.


ACTUALLY: THIS SHOULD BE RAISED AS ITS OWN ISSUE HERE as soon as the coercion part is settled because it means that _sum may succeed with certain mixed input types even though some of the same types may fail, when they are the only type.


IMPORTANTLY, neither requirement has anything to do with the module's type coercion, which is the topic of this discussion.

IN ADDITION, the proposed patch involving _coerce_types adds the following (soft) requirement:
in order to be able to coerce a sequence of numbers of mixed numeric types without loss of precision all involved types need to be integrated into the hierarchy of numeric abstract base classes as defined in the numbers module (otherwise all types are coerced to float).

>From this it should be clear that the compatibility bottleneck is not in this patch, but in other parts of the module.

What is more, if a custom numeric type implements the numerator/denominator properties, then it is simple enough to register it as a virtual subclass of numbers.Rational or .Integral to enable lossless coercion in the presence of mixed types.

Example with gmpy2 numeric type:

>>> from gmpy2 import mpq # mpq is gmpy's Fraction equivalent
>>> import numbers

>>> numbers.Rational.register(type(mpq()))
>>> r = mpq(7,5)
>>> type(r)
<class 'mpq'>

>>> issubclass(type(r), numbers.Rational)
True

=> the mpq type could now be coerced correctly even in mixed input types situations, but remains incompatible with _sum for reasons (2) and (3).


SUMMARY: Making _sum work with custom types is very complicated, BUT:
this is not due to type coercion as it is discussed here.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20481>
_______________________________________