Mailman 3 May 2015 - Python-checkins

cpython (merge 3.5 -> default): minor fix of module order in whatsnew/3.5
by tal.einat May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/7f9421c800d1 changeset: 96429:7f9421c800d1 parent: 96427:5c8c123943cf parent: 96428:d8deabc8d39a user: Tal Einat <taleinat(a)gmail.com> date: Sun May 31 22:19:27 2015 +0300 summary: minor fix of module order in whatsnew/3.5 (merge 3.5) files: Doc/whatsnew/3.5.rst | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ … [View More]

1 0

cpython (3.5): minor fix of module order in whatsnew/3.5
by tal.einat May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/d8deabc8d39a changeset: 96428:d8deabc8d39a branch: 3.5 parent: 96426:0347f6e14ad6 user: Tal Einat <taleinat(a)gmail.com> date: Sun May 31 22:18:31 2015 +0300 summary: minor fix of module order in whatsnew/3.5 files: Doc/whatsnew/3.5.rst | 16 ++++++++-------- 1 files changed, 8 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -… [View More]

1 0

cpython (merge 3.5 -> default): Issue #19543: Implementation of isclose as per PEP 485
by tal.einat May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/5c8c123943cf changeset: 96427:5c8c123943cf parent: 96424:60f5091cbfbf parent: 96426:0347f6e14ad6 user: Tal Einat <taleinat(a)gmail.com> date: Sun May 31 22:15:11 2015 +0300 summary: Issue #19543: Implementation of isclose as per PEP 485 For details, see: PEP 0485 -- A Function for testing approximate equality Functions added: math.isclose() and cmath.isclose(). Original code by Chris Barker. Patch by Tal Einat. (merge 3.5) … [View More]files: Doc/library/cmath.rst | 32 +++++ Doc/library/math.rst | 32 +++++ Doc/whatsnew/3.5.rst | 21 +++ Lib/test/test_cmath.py | 42 +++++++ Lib/test/test_math.py | 121 +++++++++++++++++++++ Misc/NEWS | 3 + Modules/clinic/cmathmodule.c.h | 53 +++++++++- Modules/cmathmodule.c | 68 +++++++++++ Modules/mathmodule.c | 79 +++++++++++++ 9 files changed, 450 insertions(+), 1 deletions(-) diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst --- a/Doc/library/cmath.rst +++ b/Doc/library/cmath.rst @@ -207,6 +207,38 @@ and ``False`` otherwise. +.. function:: isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) + + Return ``True`` if the values *a* and *b* are close to each other and + ``False`` otherwise. + + Whether or not two values are considered close is determined according to + given absolute and relative tolerances. + + *rel_tol* is the relative tolerance -- it is the maximum allowed difference + between *a* and *b*, relative to the larger absolute value of *a* or *b*. + For example, to set a tolerance of 5%, pass ``rel_tol=0.05``. The default + tolerance is ``1e-09``, which assures that the two values are the same + within about 9 decimal digits. *rel_tol* must be greater than zero. + + *abs_tol* is the minimum absolute tolerance -- useful for comparisons near + zero. *abs_tol* must be at least zero. + + If no errors occur, the result will be: + ``abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)``. + + The IEEE 754 special values of ``NaN``, ``inf``, and ``-inf`` will be + handled according to IEEE rules. Specifically, ``NaN`` is not considered + close to any other value, including ``NaN``. ``inf`` and ``-inf`` are only + considered close to themselves. + + .. versionadded:: 3.5 + + .. seealso:: + + :pep:`485` -- A function for testing approximate equality + + Constants --------- diff --git a/Doc/library/math.rst b/Doc/library/math.rst --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -110,6 +110,38 @@ .. versionadded:: 3.5 +.. function:: isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) + + Return ``True`` if the values *a* and *b* are close to each other and + ``False`` otherwise. + + Whether or not two values are considered close is determined according to + given absolute and relative tolerances. + + *rel_tol* is the relative tolerance -- it is the maximum allowed difference + between *a* and *b*, relative to the larger absolute value of *a* or *b*. + For example, to set a tolerance of 5%, pass ``rel_tol=0.05``. The default + tolerance is ``1e-09``, which assures that the two values are the same + within about 9 decimal digits. *rel_tol* must be greater than zero. + + *abs_tol* is the minimum absolute tolerance -- useful for comparisons near + zero. *abs_tol* must be at least zero. + + If no errors occur, the result will be: + ``abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)``. + + The IEEE 754 special values of ``NaN``, ``inf``, and ``-inf`` will be + handled according to IEEE rules. Specifically, ``NaN`` is not considered + close to any other value, including ``NaN``. ``inf`` and ``-inf`` are only + considered close to themselves. + + .. versionadded:: 3.5 + + .. seealso:: + + :pep:`485` -- A function for testing approximate equality + + .. function:: isfinite(x) Return ``True`` if *x* is neither an infinity nor a NaN, and diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -285,6 +285,18 @@ :pep:`488` -- Multi-phase extension module initialization +PEP 485: A function for testing approximate equality +---------------------------------------------------- + +:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` +functions which tell whether two values are approximately equal or +"close" to each other. Whether or not two values are considered +close is determined according to given absolute and relative tolerances. + +.. seealso:: + + :pep:`485` -- A function for testing approximate equality + Other Language Changes ====================== @@ -346,6 +358,13 @@ * :class:`~cgi.FieldStorage` now supports the context management protocol. (Contributed by Berker Peksag in :issue:`20289`.) +cmath +----- + +* :func:`cmath.isclose` function added. + (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) + + code ---- @@ -578,6 +597,8 @@ * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark Dickinson in :issue:`23185`.) +* :func:`math.isclose` function added. + (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) shutil ------ diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py --- a/Lib/test/test_cmath.py +++ b/Lib/test/test_cmath.py @@ -1,5 +1,6 @@ from test.support import requires_IEEE_754 from test.test_math import parse_testfile, test_file +import test.test_math as test_math import unittest import cmath, math from cmath import phase, polar, rect, pi @@ -529,5 +530,46 @@ self.assertComplexIdentical(cmath.atanh(z), z) +class IsCloseTests(test_math.IsCloseTests): + isclose = cmath.isclose + + def test_reject_complex_tolerances(self): + with self.assertRaises(TypeError): + self.isclose(1j, 1j, rel_tol=1j) + + with self.assertRaises(TypeError): + self.isclose(1j, 1j, abs_tol=1j) + + with self.assertRaises(TypeError): + self.isclose(1j, 1j, rel_tol=1j, abs_tol=1j) + + def test_complex_values(self): + # test complex values that are close to within 12 decimal places + complex_examples = [(1.0+1.0j, 1.000000000001+1.0j), + (1.0+1.0j, 1.0+1.000000000001j), + (-1.0+1.0j, -1.000000000001+1.0j), + (1.0-1.0j, 1.0-0.999999999999j), + ] + + self.assertAllClose(complex_examples, rel_tol=1e-12) + self.assertAllNotClose(complex_examples, rel_tol=1e-13) + + def test_complex_near_zero(self): + # test values near zero that are near to within three decimal places + near_zero_examples = [(0.001j, 0), + (0.001, 0), + (0.001+0.001j, 0), + (-0.001+0.001j, 0), + (0.001-0.001j, 0), + (-0.001-0.001j, 0), + ] + + self.assertAllClose(near_zero_examples, abs_tol=1.5e-03) + self.assertAllNotClose(near_zero_examples, abs_tol=0.5e-03) + + self.assertIsClose(0.001-0.001j, 0.001+0.001j, abs_tol=2e-03) + self.assertIsNotClose(0.001-0.001j, 0.001+0.001j, abs_tol=1e-03) + + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -1166,10 +1166,131 @@ '\n '.join(failures)) +class IsCloseTests(unittest.TestCase): + isclose = math.isclose # sublcasses should override this + + def assertIsClose(self, a, b, *args, **kwargs): + self.assertTrue(self.isclose(a, b, *args, **kwargs), + msg="%s and %s should be close!" % (a, b)) + + def assertIsNotClose(self, a, b, *args, **kwargs): + self.assertFalse(self.isclose(a, b, *args, **kwargs), + msg="%s and %s should not be close!" % (a, b)) + + def assertAllClose(self, examples, *args, **kwargs): + for a, b in examples: + self.assertIsClose(a, b, *args, **kwargs) + + def assertAllNotClose(self, examples, *args, **kwargs): + for a, b in examples: + self.assertIsNotClose(a, b, *args, **kwargs) + + def test_negative_tolerances(self): + # ValueError should be raised if either tolerance is less than zero + with self.assertRaises(ValueError): + self.assertIsClose(1, 1, rel_tol=-1e-100) + with self.assertRaises(ValueError): + self.assertIsClose(1, 1, rel_tol=1e-100, abs_tol=-1e10) + + def test_identical(self): + # identical values must test as close + identical_examples = [(2.0, 2.0), + (0.1e200, 0.1e200), + (1.123e-300, 1.123e-300), + (12345, 12345.0), + (0.0, -0.0), + (345678, 345678)] + self.assertAllClose(identical_examples, rel_tol=0.0, abs_tol=0.0) + + def test_eight_decimal_places(self): + # examples that are close to 1e-8, but not 1e-9 + eight_decimal_places_examples = [(1e8, 1e8 + 1), + (-1e-8, -1.000000009e-8), + (1.12345678, 1.12345679)] + self.assertAllClose(eight_decimal_places_examples, rel_tol=1e-8) + self.assertAllNotClose(eight_decimal_places_examples, rel_tol=1e-9) + + def test_near_zero(self): + # values close to zero + near_zero_examples = [(1e-9, 0.0), + (-1e-9, 0.0), + (-1e-150, 0.0)] + # these should not be close to any rel_tol + self.assertAllNotClose(near_zero_examples, rel_tol=0.9) + # these should be close to abs_tol=1e-8 + self.assertAllClose(near_zero_examples, abs_tol=1e-8) + + def test_identical_infinite(self): + # these are close regardless of tolerance -- i.e. they are equal + self.assertIsClose(INF, INF) + self.assertIsClose(INF, INF, abs_tol=0.0) + self.assertIsClose(NINF, NINF) + self.assertIsClose(NINF, NINF, abs_tol=0.0) + + def test_inf_ninf_nan(self): + # these should never be close (following IEEE 754 rules for equality) + not_close_examples = [(NAN, NAN), + (NAN, 1e-100), + (1e-100, NAN), + (INF, NAN), + (NAN, INF), + (INF, NINF), + (INF, 1.0), + (1.0, INF), + (INF, 1e308), + (1e308, INF)] + # use largest reasonable tolerance + self.assertAllNotClose(not_close_examples, abs_tol=0.999999999999999) + + def test_zero_tolerance(self): + # test with zero tolerance + zero_tolerance_close_examples = [(1.0, 1.0), + (-3.4, -3.4), + (-1e-300, -1e-300)] + self.assertAllClose(zero_tolerance_close_examples, rel_tol=0.0) + + zero_tolerance_not_close_examples = [(1.0, 1.000000000000001), + (0.99999999999999, 1.0), + (1.0e200, .999999999999999e200)] + self.assertAllNotClose(zero_tolerance_not_close_examples, rel_tol=0.0) + + def test_assymetry(self): + # test the assymetry example from PEP 485 + self.assertAllClose([(9, 10), (10, 9)], rel_tol=0.1) + + def test_integers(self): + # test with integer values + integer_examples = [(100000001, 100000000), + (123456789, 123456788)] + + self.assertAllClose(integer_examples, rel_tol=1e-8) + self.assertAllNotClose(integer_examples, rel_tol=1e-9) + + def test_decimals(self): + # test with Decimal values + from decimal import Decimal + + decimal_examples = [(Decimal('1.00000001'), Decimal('1.0')), + (Decimal('1.00000001e-20'), Decimal('1.0e-20')), + (Decimal('1.00000001e-100'), Decimal('1.0e-100'))] + self.assertAllClose(decimal_examples, rel_tol=1e-8) + self.assertAllNotClose(decimal_examples, rel_tol=1e-9) + + def test_fractions(self): + # test with Fraction values + from fractions import Fraction + + # could use some more examples here! + fraction_examples = [(Fraction(1, 100000000) + 1, Fraction(1))] + self.assertAllClose(fraction_examples, rel_tol=1e-8) + self.assertAllNotClose(fraction_examples, rel_tol=1e-9) + + def test_main(): from doctest import DocFileSuite suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(MathTests)) + suite.addTest(unittest.makeSuite(IsCloseTests)) suite.addTest(DocFileSuite("ieee754.txt")) run_unittest(suite) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -288,6 +288,9 @@ - Issue #24298: Fix inspect.signature() to correctly unwrap wrappers around bound methods. +- Issue #24270: Add math.isclose() and cmath.isclose() functions as per PEP 485. + Contributed by Chris Barker and Tal Einat. + IDLE ---- diff --git a/Modules/clinic/cmathmodule.c.h b/Modules/clinic/cmathmodule.c.h --- a/Modules/clinic/cmathmodule.c.h +++ b/Modules/clinic/cmathmodule.c.h @@ -806,4 +806,55 @@ exit: return return_value; } -/*[clinic end generated code: output=274f59792cf4f418 input=a9049054013a1b77]*/ + +PyDoc_STRVAR(cmath_isclose__doc__, +"isclose($module, /, a, b, *, rel_tol=1e-09, abs_tol=0.0)\n" +"--\n" +"\n" +"Determine whether two complex numbers are close in value.\n" +"\n" +" rel_tol\n" +" maximum difference for being considered \"close\", relative to the\n" +" magnitude of the input values\n" +" abs_tol\n" +" maximum difference for being considered \"close\", regardless of the\n" +" magnitude of the input values\n" +"\n" +"Return True if a is close in value to b, and False otherwise.\n" +"\n" +"For the values to be considered close, the difference between them must be\n" +"smaller than at least one of the tolerances.\n" +"\n" +"-inf, inf and NaN behave similarly to the IEEE 754 Standard. That is, NaN is\n" +"not close to anything, even itself. inf and -inf are only close to themselves."); + +#define CMATH_ISCLOSE_METHODDEF \ + {"isclose", (PyCFunction)cmath_isclose, METH_VARARGS|METH_KEYWORDS, cmath_isclose__doc__}, + +static int +cmath_isclose_impl(PyModuleDef *module, Py_complex a, Py_complex b, + double rel_tol, double abs_tol); + +static PyObject * +cmath_isclose(PyModuleDef *module, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + Py_complex a; + Py_complex b; + double rel_tol = 1e-09; + double abs_tol = 0.0; + int _return_value; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "DD|$dd:isclose", _keywords, + &a, &b, &rel_tol, &abs_tol)) + goto exit; + _return_value = cmath_isclose_impl(module, a, b, rel_tol, abs_tol); + if ((_return_value == -1) && PyErr_Occurred()) + goto exit; + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} +/*[clinic end generated code: output=229e9c48c9d27362 input=a9049054013a1b77]*/ diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1114,6 +1114,73 @@ Py_IS_INFINITY(z.imag)); } +/*[clinic input] +cmath.isclose -> bool + + a: Py_complex + b: Py_complex + * + rel_tol: double = 1e-09 + maximum difference for being considered "close", relative to the + magnitude of the input values + abs_tol: double = 0.0 + maximum difference for being considered "close", regardless of the + magnitude of the input values + +Determine whether two complex numbers are close in value. + +Return True if a is close in value to b, and False otherwise. + +For the values to be considered close, the difference between them must be +smaller than at least one of the tolerances. + +-inf, inf and NaN behave similarly to the IEEE 754 Standard. That is, NaN is +not close to anything, even itself. inf and -inf are only close to themselves. +[clinic start generated code]*/ + +static int +cmath_isclose_impl(PyModuleDef *module, Py_complex a, Py_complex b, + double rel_tol, double abs_tol) +/*[clinic end generated code: output=da0c535fb54e2310 input=df9636d7de1d4ac3]*/ +{ + double diff; + + /* sanity check on the inputs */ + if (rel_tol < 0.0 || abs_tol < 0.0 ) { + PyErr_SetString(PyExc_ValueError, + "tolerances must be non-negative"); + return -1; + } + + if ( (a.real == b.real) && (a.imag == b.imag) ) { + /* short circuit exact equality -- needed to catch two infinities of + the same sign. And perhaps speeds things up a bit sometimes. + */ + return 1; + } + + /* This catches the case of two infinities of opposite sign, or + one infinity and one finite number. Two infinities of opposite + sign would otherwise have an infinite relative tolerance. + Two infinities of the same sign are caught by the equality check + above. + */ + + if (Py_IS_INFINITY(a.real) || Py_IS_INFINITY(a.imag) || + Py_IS_INFINITY(b.real) || Py_IS_INFINITY(b.imag)) { + return 0; + } + + /* now do the regular computation + this is essentially the "weak" test from the Boost library + */ + + diff = _Py_c_abs(_Py_c_diff(a, b)); + + return (((diff <= rel_tol * _Py_c_abs(b)) || + (diff <= rel_tol * _Py_c_abs(a))) || + (diff <= abs_tol)); +} PyDoc_STRVAR(module_doc, "This module is always available. It provides access to mathematical\n" @@ -1129,6 +1196,7 @@ CMATH_COS_METHODDEF CMATH_COSH_METHODDEF CMATH_EXP_METHODDEF + CMATH_ISCLOSE_METHODDEF CMATH_ISFINITE_METHODDEF CMATH_ISINF_METHODDEF CMATH_ISNAN_METHODDEF diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1990,6 +1990,83 @@ "isinf(x) -> bool\n\n\ Return True if x is a positive or negative infinity, and False otherwise."); +static PyObject * +math_isclose(PyObject *self, PyObject *args, PyObject *kwargs) +{ + double a, b; + double rel_tol = 1e-9; + double abs_tol = 0.0; + double diff = 0.0; + long result = 0; + + static char *keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "dd|$dd:isclose", + keywords, + &a, &b, &rel_tol, &abs_tol + )) + return NULL; + + /* sanity check on the inputs */ + if (rel_tol < 0.0 || abs_tol < 0.0 ) { + PyErr_SetString(PyExc_ValueError, + "tolerances must be non-negative"); + return NULL; + } + + if ( a == b ) { + /* short circuit exact equality -- needed to catch two infinities of + the same sign. And perhaps speeds things up a bit sometimes. + */ + Py_RETURN_TRUE; + } + + /* This catches the case of two infinities of opposite sign, or + one infinity and one finite number. Two infinities of opposite + sign would otherwise have an infinite relative tolerance. + Two infinities of the same sign are caught by the equality check + above. + */ + + if (Py_IS_INFINITY(a) || Py_IS_INFINITY(b)) { + Py_RETURN_FALSE; + } + + /* now do the regular computation + this is essentially the "weak" test from the Boost library + */ + + diff = fabs(b - a); + + result = (((diff <= fabs(rel_tol * b)) || + (diff <= fabs(rel_tol * a))) || + (diff <= abs_tol)); + + return PyBool_FromLong(result); +} + +PyDoc_STRVAR(math_isclose_doc, +"is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool\n" +"\n" +"Determine whether two floating point numbers are close in value.\n" +"\n" +" rel_tol\n" +" maximum difference for being considered \"close\", relative to the\n" +" magnitude of the input values\n" +" abs_tol\n" +" maximum difference for being considered \"close\", regardless of the\n" +" magnitude of the input values\n" +"\n" +"Return True if a is close in value to b, and False otherwise.\n" +"\n" +"For the values to be considered close, the difference between them\n" +"must be smaller than at least one of the tolerances.\n" +"\n" +"-inf, inf and NaN behave similarly to the IEEE 754 Standard. That\n" +"is, NaN is not close to anything, even itself. inf and -inf are\n" +"only close to themselves."); + static PyMethodDef math_methods[] = { {"acos", math_acos, METH_O, math_acos_doc}, {"acosh", math_acosh, METH_O, math_acosh_doc}, @@ -2016,6 +2093,8 @@ {"gamma", math_gamma, METH_O, math_gamma_doc}, {"gcd", math_gcd, METH_VARARGS, math_gcd_doc}, {"hypot", math_hypot, METH_VARARGS, math_hypot_doc}, + {"isclose", (PyCFunction) math_isclose, METH_VARARGS | METH_KEYWORDS, + math_isclose_doc}, {"isfinite", math_isfinite, METH_O, math_isfinite_doc}, {"isinf", math_isinf, METH_O, math_isinf_doc}, {"isnan", math_isnan, METH_O, math_isnan_doc}, -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

cpython (3.5): Issue #19543: Implementation of isclose as per PEP 485
by tal.einat May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/0347f6e14ad6 changeset: 96426:0347f6e14ad6 branch: 3.5 parent: 96423:8296f0119f20 user: Tal Einat <taleinat(a)gmail.com> date: Sun May 31 22:05:00 2015 +0300 summary: Issue #19543: Implementation of isclose as per PEP 485 For details, see: PEP 0485 -- A Function for testing approximate equality Functions added: math.isclose() and cmath.isclose(). Original code by Chris Barker. Patch by Tal Einat. files: Doc/library/cmath.… [View More]rst | 32 +++++ Doc/library/math.rst | 32 +++++ Doc/whatsnew/3.5.rst | 21 +++ Lib/test/test_cmath.py | 42 +++++++ Lib/test/test_math.py | 121 +++++++++++++++++++++ Misc/NEWS | 3 + Modules/clinic/cmathmodule.c.h | 53 +++++++++- Modules/cmathmodule.c | 68 +++++++++++ Modules/mathmodule.c | 79 +++++++++++++ 9 files changed, 450 insertions(+), 1 deletions(-) diff --git a/Doc/library/cmath.rst b/Doc/library/cmath.rst --- a/Doc/library/cmath.rst +++ b/Doc/library/cmath.rst @@ -207,6 +207,38 @@ and ``False`` otherwise. +.. function:: isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) + + Return ``True`` if the values *a* and *b* are close to each other and + ``False`` otherwise. + + Whether or not two values are considered close is determined according to + given absolute and relative tolerances. + + *rel_tol* is the relative tolerance -- it is the maximum allowed difference + between *a* and *b*, relative to the larger absolute value of *a* or *b*. + For example, to set a tolerance of 5%, pass ``rel_tol=0.05``. The default + tolerance is ``1e-09``, which assures that the two values are the same + within about 9 decimal digits. *rel_tol* must be greater than zero. + + *abs_tol* is the minimum absolute tolerance -- useful for comparisons near + zero. *abs_tol* must be at least zero. + + If no errors occur, the result will be: + ``abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)``. + + The IEEE 754 special values of ``NaN``, ``inf``, and ``-inf`` will be + handled according to IEEE rules. Specifically, ``NaN`` is not considered + close to any other value, including ``NaN``. ``inf`` and ``-inf`` are only + considered close to themselves. + + .. versionadded:: 3.5 + + .. seealso:: + + :pep:`485` -- A function for testing approximate equality + + Constants --------- diff --git a/Doc/library/math.rst b/Doc/library/math.rst --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -110,6 +110,38 @@ .. versionadded:: 3.5 +.. function:: isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) + + Return ``True`` if the values *a* and *b* are close to each other and + ``False`` otherwise. + + Whether or not two values are considered close is determined according to + given absolute and relative tolerances. + + *rel_tol* is the relative tolerance -- it is the maximum allowed difference + between *a* and *b*, relative to the larger absolute value of *a* or *b*. + For example, to set a tolerance of 5%, pass ``rel_tol=0.05``. The default + tolerance is ``1e-09``, which assures that the two values are the same + within about 9 decimal digits. *rel_tol* must be greater than zero. + + *abs_tol* is the minimum absolute tolerance -- useful for comparisons near + zero. *abs_tol* must be at least zero. + + If no errors occur, the result will be: + ``abs(a-b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)``. + + The IEEE 754 special values of ``NaN``, ``inf``, and ``-inf`` will be + handled according to IEEE rules. Specifically, ``NaN`` is not considered + close to any other value, including ``NaN``. ``inf`` and ``-inf`` are only + considered close to themselves. + + .. versionadded:: 3.5 + + .. seealso:: + + :pep:`485` -- A function for testing approximate equality + + .. function:: isfinite(x) Return ``True`` if *x* is neither an infinity nor a NaN, and diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -285,6 +285,18 @@ :pep:`488` -- Multi-phase extension module initialization +PEP 485: A function for testing approximate equality +---------------------------------------------------- + +:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` +functions which tell whether two values are approximately equal or +"close" to each other. Whether or not two values are considered +close is determined according to given absolute and relative tolerances. + +.. seealso:: + + :pep:`485` -- A function for testing approximate equality + Other Language Changes ====================== @@ -346,6 +358,13 @@ * :class:`~cgi.FieldStorage` now supports the context management protocol. (Contributed by Berker Peksag in :issue:`20289`.) +cmath +----- + +* :func:`cmath.isclose` function added. + (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) + + code ---- @@ -578,6 +597,8 @@ * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark Dickinson in :issue:`23185`.) +* :func:`math.isclose` function added. + (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) shutil ------ diff --git a/Lib/test/test_cmath.py b/Lib/test/test_cmath.py --- a/Lib/test/test_cmath.py +++ b/Lib/test/test_cmath.py @@ -1,5 +1,6 @@ from test.support import requires_IEEE_754 from test.test_math import parse_testfile, test_file +import test.test_math as test_math import unittest import cmath, math from cmath import phase, polar, rect, pi @@ -529,5 +530,46 @@ self.assertComplexIdentical(cmath.atanh(z), z) +class IsCloseTests(test_math.IsCloseTests): + isclose = cmath.isclose + + def test_reject_complex_tolerances(self): + with self.assertRaises(TypeError): + self.isclose(1j, 1j, rel_tol=1j) + + with self.assertRaises(TypeError): + self.isclose(1j, 1j, abs_tol=1j) + + with self.assertRaises(TypeError): + self.isclose(1j, 1j, rel_tol=1j, abs_tol=1j) + + def test_complex_values(self): + # test complex values that are close to within 12 decimal places + complex_examples = [(1.0+1.0j, 1.000000000001+1.0j), + (1.0+1.0j, 1.0+1.000000000001j), + (-1.0+1.0j, -1.000000000001+1.0j), + (1.0-1.0j, 1.0-0.999999999999j), + ] + + self.assertAllClose(complex_examples, rel_tol=1e-12) + self.assertAllNotClose(complex_examples, rel_tol=1e-13) + + def test_complex_near_zero(self): + # test values near zero that are near to within three decimal places + near_zero_examples = [(0.001j, 0), + (0.001, 0), + (0.001+0.001j, 0), + (-0.001+0.001j, 0), + (0.001-0.001j, 0), + (-0.001-0.001j, 0), + ] + + self.assertAllClose(near_zero_examples, abs_tol=1.5e-03) + self.assertAllNotClose(near_zero_examples, abs_tol=0.5e-03) + + self.assertIsClose(0.001-0.001j, 0.001+0.001j, abs_tol=2e-03) + self.assertIsNotClose(0.001-0.001j, 0.001+0.001j, abs_tol=1e-03) + + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -1166,10 +1166,131 @@ '\n '.join(failures)) +class IsCloseTests(unittest.TestCase): + isclose = math.isclose # sublcasses should override this + + def assertIsClose(self, a, b, *args, **kwargs): + self.assertTrue(self.isclose(a, b, *args, **kwargs), + msg="%s and %s should be close!" % (a, b)) + + def assertIsNotClose(self, a, b, *args, **kwargs): + self.assertFalse(self.isclose(a, b, *args, **kwargs), + msg="%s and %s should not be close!" % (a, b)) + + def assertAllClose(self, examples, *args, **kwargs): + for a, b in examples: + self.assertIsClose(a, b, *args, **kwargs) + + def assertAllNotClose(self, examples, *args, **kwargs): + for a, b in examples: + self.assertIsNotClose(a, b, *args, **kwargs) + + def test_negative_tolerances(self): + # ValueError should be raised if either tolerance is less than zero + with self.assertRaises(ValueError): + self.assertIsClose(1, 1, rel_tol=-1e-100) + with self.assertRaises(ValueError): + self.assertIsClose(1, 1, rel_tol=1e-100, abs_tol=-1e10) + + def test_identical(self): + # identical values must test as close + identical_examples = [(2.0, 2.0), + (0.1e200, 0.1e200), + (1.123e-300, 1.123e-300), + (12345, 12345.0), + (0.0, -0.0), + (345678, 345678)] + self.assertAllClose(identical_examples, rel_tol=0.0, abs_tol=0.0) + + def test_eight_decimal_places(self): + # examples that are close to 1e-8, but not 1e-9 + eight_decimal_places_examples = [(1e8, 1e8 + 1), + (-1e-8, -1.000000009e-8), + (1.12345678, 1.12345679)] + self.assertAllClose(eight_decimal_places_examples, rel_tol=1e-8) + self.assertAllNotClose(eight_decimal_places_examples, rel_tol=1e-9) + + def test_near_zero(self): + # values close to zero + near_zero_examples = [(1e-9, 0.0), + (-1e-9, 0.0), + (-1e-150, 0.0)] + # these should not be close to any rel_tol + self.assertAllNotClose(near_zero_examples, rel_tol=0.9) + # these should be close to abs_tol=1e-8 + self.assertAllClose(near_zero_examples, abs_tol=1e-8) + + def test_identical_infinite(self): + # these are close regardless of tolerance -- i.e. they are equal + self.assertIsClose(INF, INF) + self.assertIsClose(INF, INF, abs_tol=0.0) + self.assertIsClose(NINF, NINF) + self.assertIsClose(NINF, NINF, abs_tol=0.0) + + def test_inf_ninf_nan(self): + # these should never be close (following IEEE 754 rules for equality) + not_close_examples = [(NAN, NAN), + (NAN, 1e-100), + (1e-100, NAN), + (INF, NAN), + (NAN, INF), + (INF, NINF), + (INF, 1.0), + (1.0, INF), + (INF, 1e308), + (1e308, INF)] + # use largest reasonable tolerance + self.assertAllNotClose(not_close_examples, abs_tol=0.999999999999999) + + def test_zero_tolerance(self): + # test with zero tolerance + zero_tolerance_close_examples = [(1.0, 1.0), + (-3.4, -3.4), + (-1e-300, -1e-300)] + self.assertAllClose(zero_tolerance_close_examples, rel_tol=0.0) + + zero_tolerance_not_close_examples = [(1.0, 1.000000000000001), + (0.99999999999999, 1.0), + (1.0e200, .999999999999999e200)] + self.assertAllNotClose(zero_tolerance_not_close_examples, rel_tol=0.0) + + def test_assymetry(self): + # test the assymetry example from PEP 485 + self.assertAllClose([(9, 10), (10, 9)], rel_tol=0.1) + + def test_integers(self): + # test with integer values + integer_examples = [(100000001, 100000000), + (123456789, 123456788)] + + self.assertAllClose(integer_examples, rel_tol=1e-8) + self.assertAllNotClose(integer_examples, rel_tol=1e-9) + + def test_decimals(self): + # test with Decimal values + from decimal import Decimal + + decimal_examples = [(Decimal('1.00000001'), Decimal('1.0')), + (Decimal('1.00000001e-20'), Decimal('1.0e-20')), + (Decimal('1.00000001e-100'), Decimal('1.0e-100'))] + self.assertAllClose(decimal_examples, rel_tol=1e-8) + self.assertAllNotClose(decimal_examples, rel_tol=1e-9) + + def test_fractions(self): + # test with Fraction values + from fractions import Fraction + + # could use some more examples here! + fraction_examples = [(Fraction(1, 100000000) + 1, Fraction(1))] + self.assertAllClose(fraction_examples, rel_tol=1e-8) + self.assertAllNotClose(fraction_examples, rel_tol=1e-9) + + def test_main(): from doctest import DocFileSuite suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(MathTests)) + suite.addTest(unittest.makeSuite(IsCloseTests)) suite.addTest(DocFileSuite("ieee754.txt")) run_unittest(suite) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -273,6 +273,9 @@ - Issue #24298: Fix inspect.signature() to correctly unwrap wrappers around bound methods. +- Issue #24270: Add math.isclose() and cmath.isclose() functions as per PEP 485. + Contributed by Chris Barker and Tal Einat. + IDLE ---- diff --git a/Modules/clinic/cmathmodule.c.h b/Modules/clinic/cmathmodule.c.h --- a/Modules/clinic/cmathmodule.c.h +++ b/Modules/clinic/cmathmodule.c.h @@ -806,4 +806,55 @@ exit: return return_value; } -/*[clinic end generated code: output=274f59792cf4f418 input=a9049054013a1b77]*/ + +PyDoc_STRVAR(cmath_isclose__doc__, +"isclose($module, /, a, b, *, rel_tol=1e-09, abs_tol=0.0)\n" +"--\n" +"\n" +"Determine whether two complex numbers are close in value.\n" +"\n" +" rel_tol\n" +" maximum difference for being considered \"close\", relative to the\n" +" magnitude of the input values\n" +" abs_tol\n" +" maximum difference for being considered \"close\", regardless of the\n" +" magnitude of the input values\n" +"\n" +"Return True if a is close in value to b, and False otherwise.\n" +"\n" +"For the values to be considered close, the difference between them must be\n" +"smaller than at least one of the tolerances.\n" +"\n" +"-inf, inf and NaN behave similarly to the IEEE 754 Standard. That is, NaN is\n" +"not close to anything, even itself. inf and -inf are only close to themselves."); + +#define CMATH_ISCLOSE_METHODDEF \ + {"isclose", (PyCFunction)cmath_isclose, METH_VARARGS|METH_KEYWORDS, cmath_isclose__doc__}, + +static int +cmath_isclose_impl(PyModuleDef *module, Py_complex a, Py_complex b, + double rel_tol, double abs_tol); + +static PyObject * +cmath_isclose(PyModuleDef *module, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + static char *_keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + Py_complex a; + Py_complex b; + double rel_tol = 1e-09; + double abs_tol = 0.0; + int _return_value; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "DD|$dd:isclose", _keywords, + &a, &b, &rel_tol, &abs_tol)) + goto exit; + _return_value = cmath_isclose_impl(module, a, b, rel_tol, abs_tol); + if ((_return_value == -1) && PyErr_Occurred()) + goto exit; + return_value = PyBool_FromLong((long)_return_value); + +exit: + return return_value; +} +/*[clinic end generated code: output=229e9c48c9d27362 input=a9049054013a1b77]*/ diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1114,6 +1114,73 @@ Py_IS_INFINITY(z.imag)); } +/*[clinic input] +cmath.isclose -> bool + + a: Py_complex + b: Py_complex + * + rel_tol: double = 1e-09 + maximum difference for being considered "close", relative to the + magnitude of the input values + abs_tol: double = 0.0 + maximum difference for being considered "close", regardless of the + magnitude of the input values + +Determine whether two complex numbers are close in value. + +Return True if a is close in value to b, and False otherwise. + +For the values to be considered close, the difference between them must be +smaller than at least one of the tolerances. + +-inf, inf and NaN behave similarly to the IEEE 754 Standard. That is, NaN is +not close to anything, even itself. inf and -inf are only close to themselves. +[clinic start generated code]*/ + +static int +cmath_isclose_impl(PyModuleDef *module, Py_complex a, Py_complex b, + double rel_tol, double abs_tol) +/*[clinic end generated code: output=da0c535fb54e2310 input=df9636d7de1d4ac3]*/ +{ + double diff; + + /* sanity check on the inputs */ + if (rel_tol < 0.0 || abs_tol < 0.0 ) { + PyErr_SetString(PyExc_ValueError, + "tolerances must be non-negative"); + return -1; + } + + if ( (a.real == b.real) && (a.imag == b.imag) ) { + /* short circuit exact equality -- needed to catch two infinities of + the same sign. And perhaps speeds things up a bit sometimes. + */ + return 1; + } + + /* This catches the case of two infinities of opposite sign, or + one infinity and one finite number. Two infinities of opposite + sign would otherwise have an infinite relative tolerance. + Two infinities of the same sign are caught by the equality check + above. + */ + + if (Py_IS_INFINITY(a.real) || Py_IS_INFINITY(a.imag) || + Py_IS_INFINITY(b.real) || Py_IS_INFINITY(b.imag)) { + return 0; + } + + /* now do the regular computation + this is essentially the "weak" test from the Boost library + */ + + diff = _Py_c_abs(_Py_c_diff(a, b)); + + return (((diff <= rel_tol * _Py_c_abs(b)) || + (diff <= rel_tol * _Py_c_abs(a))) || + (diff <= abs_tol)); +} PyDoc_STRVAR(module_doc, "This module is always available. It provides access to mathematical\n" @@ -1129,6 +1196,7 @@ CMATH_COS_METHODDEF CMATH_COSH_METHODDEF CMATH_EXP_METHODDEF + CMATH_ISCLOSE_METHODDEF CMATH_ISFINITE_METHODDEF CMATH_ISINF_METHODDEF CMATH_ISNAN_METHODDEF diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1990,6 +1990,83 @@ "isinf(x) -> bool\n\n\ Return True if x is a positive or negative infinity, and False otherwise."); +static PyObject * +math_isclose(PyObject *self, PyObject *args, PyObject *kwargs) +{ + double a, b; + double rel_tol = 1e-9; + double abs_tol = 0.0; + double diff = 0.0; + long result = 0; + + static char *keywords[] = {"a", "b", "rel_tol", "abs_tol", NULL}; + + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "dd|$dd:isclose", + keywords, + &a, &b, &rel_tol, &abs_tol + )) + return NULL; + + /* sanity check on the inputs */ + if (rel_tol < 0.0 || abs_tol < 0.0 ) { + PyErr_SetString(PyExc_ValueError, + "tolerances must be non-negative"); + return NULL; + } + + if ( a == b ) { + /* short circuit exact equality -- needed to catch two infinities of + the same sign. And perhaps speeds things up a bit sometimes. + */ + Py_RETURN_TRUE; + } + + /* This catches the case of two infinities of opposite sign, or + one infinity and one finite number. Two infinities of opposite + sign would otherwise have an infinite relative tolerance. + Two infinities of the same sign are caught by the equality check + above. + */ + + if (Py_IS_INFINITY(a) || Py_IS_INFINITY(b)) { + Py_RETURN_FALSE; + } + + /* now do the regular computation + this is essentially the "weak" test from the Boost library + */ + + diff = fabs(b - a); + + result = (((diff <= fabs(rel_tol * b)) || + (diff <= fabs(rel_tol * a))) || + (diff <= abs_tol)); + + return PyBool_FromLong(result); +} + +PyDoc_STRVAR(math_isclose_doc, +"is_close(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool\n" +"\n" +"Determine whether two floating point numbers are close in value.\n" +"\n" +" rel_tol\n" +" maximum difference for being considered \"close\", relative to the\n" +" magnitude of the input values\n" +" abs_tol\n" +" maximum difference for being considered \"close\", regardless of the\n" +" magnitude of the input values\n" +"\n" +"Return True if a is close in value to b, and False otherwise.\n" +"\n" +"For the values to be considered close, the difference between them\n" +"must be smaller than at least one of the tolerances.\n" +"\n" +"-inf, inf and NaN behave similarly to the IEEE 754 Standard. That\n" +"is, NaN is not close to anything, even itself. inf and -inf are\n" +"only close to themselves."); + static PyMethodDef math_methods[] = { {"acos", math_acos, METH_O, math_acos_doc}, {"acosh", math_acosh, METH_O, math_acosh_doc}, @@ -2016,6 +2093,8 @@ {"gamma", math_gamma, METH_O, math_gamma_doc}, {"gcd", math_gcd, METH_VARARGS, math_gcd_doc}, {"hypot", math_hypot, METH_VARARGS, math_hypot_doc}, + {"isclose", (PyCFunction) math_isclose, METH_VARARGS | METH_KEYWORDS, + math_isclose_doc}, {"isfinite", math_isfinite, METH_O, math_isfinite_doc}, {"isinf", math_isinf, METH_O, math_isinf_doc}, {"isnan", math_isnan, METH_O, math_isnan_doc}, -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

cpython (2.7): Issue #19543: Emit deprecation warning for known non-text encodings.
by serhiy.storchaka May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/cf6e782a7f94 changeset: 96425:cf6e782a7f94 branch: 2.7 parent: 96422:7d6a6028b104 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Sun May 31 20:21:00 2015 +0300 summary: Issue #19543: Emit deprecation warning for known non-text encodings. Backported issues #19619: encode() and decode() methods and constructors of str, unicode and bytearray classes now emit deprecation warning for known non-text encodings when Python is … [View More]ran with the -3 option. Backported issues #20404: io.TextIOWrapper (and hence io.open()) now uses the internal codec marking system added to emit deprecation warning for known non-text encodings at stream construction time when Python is ran with the -3 option. files: Include/codecs.h | 45 ++++ Lib/_pyio.py | 6 + Lib/codecs.py | 14 +- Lib/encodings/base64_codec.py | 1 + Lib/encodings/bz2_codec.py | 1 + Lib/encodings/hex_codec.py | 1 + Lib/encodings/quopri_codec.py | 1 + Lib/encodings/rot_13.py | 1 + Lib/encodings/uu_codec.py | 1 + Lib/encodings/zlib_codec.py | 1 + Lib/json/decoder.py | 6 +- Lib/test/string_tests.py | 12 +- Lib/test/test_calendar.py | 4 +- Lib/test/test_codecs.py | 48 ++++- Lib/test/test_fileinput.py | 7 +- Lib/test/test_io.py | 43 +++- Misc/NEWS | 8 + Modules/_io/textio.c | 34 ++- Objects/bytearrayobject.c | 6 +- Objects/stringobject.c | 4 +- Objects/unicodeobject.c | 8 +- Python/codecs.c | 213 +++++++++++++++++++-- 22 files changed, 393 insertions(+), 72 deletions(-) diff --git a/Include/codecs.h b/Include/codecs.h --- a/Include/codecs.h +++ b/Include/codecs.h @@ -81,6 +81,51 @@ const char *errors ); +/* Text codec specific encoding and decoding API. + + Checks the encoding against a list of codecs which do not + implement a unicode<->bytes encoding before attempting the + operation. + + Please note that these APIs are internal and should not + be used in Python C extensions. + + XXX (ncoghlan): should we make these, or something like them, public + in Python 3.5+? + + */ +PyAPI_FUNC(PyObject *) _PyCodec_LookupTextEncoding( + const char *encoding, + const char *alternate_command + ); + +PyAPI_FUNC(PyObject *) _PyCodec_EncodeText( + PyObject *object, + const char *encoding, + const char *errors + ); + +PyAPI_FUNC(PyObject *) _PyCodec_DecodeText( + PyObject *object, + const char *encoding, + const char *errors + ); + +/* These two aren't actually text encoding specific, but _io.TextIOWrapper + * is the only current API consumer. + */ +PyAPI_FUNC(PyObject *) _PyCodecInfo_GetIncrementalDecoder( + PyObject *codec_info, + const char *errors + ); + +PyAPI_FUNC(PyObject *) _PyCodecInfo_GetIncrementalEncoder( + PyObject *codec_info, + const char *errors + ); + + + /* --- Codec Lookup APIs -------------------------------------------------- All APIs return a codec object with incremented refcount and are diff --git a/Lib/_pyio.py b/Lib/_pyio.py --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -7,6 +7,7 @@ import os import abc import codecs +import sys import warnings import errno # Import thread instead of threading to reduce startup cost @@ -1497,6 +1498,11 @@ if not isinstance(encoding, basestring): raise ValueError("invalid encoding: %r" % encoding) + if sys.py3kwarning and not codecs.lookup(encoding)._is_text_encoding: + msg = ("%r is not a text encoding; " + "use codecs.open() to handle arbitrary codecs") + warnings.warnpy3k(msg % encoding, stacklevel=2) + if errors is None: errors = "strict" else: diff --git a/Lib/codecs.py b/Lib/codecs.py --- a/Lib/codecs.py +++ b/Lib/codecs.py @@ -79,9 +79,19 @@ ### Codec base classes (defining the API) class CodecInfo(tuple): + """Codec details when looking up the codec registry""" + + # Private API to allow Python to blacklist the known non-Unicode + # codecs in the standard library. A more general mechanism to + # reliably distinguish test encodings from other codecs will hopefully + # be defined for Python 3.5 + # + # See http://bugs.python.org/issue19619 + _is_text_encoding = True # Assume codecs are text encodings by default def __new__(cls, encode, decode, streamreader=None, streamwriter=None, - incrementalencoder=None, incrementaldecoder=None, name=None): + incrementalencoder=None, incrementaldecoder=None, name=None, + _is_text_encoding=None): self = tuple.__new__(cls, (encode, decode, streamreader, streamwriter)) self.name = name self.encode = encode @@ -90,6 +100,8 @@ self.incrementaldecoder = incrementaldecoder self.streamwriter = streamwriter self.streamreader = streamreader + if _is_text_encoding is not None: + self._is_text_encoding = _is_text_encoding return self def __repr__(self): diff --git a/Lib/encodings/base64_codec.py b/Lib/encodings/base64_codec.py --- a/Lib/encodings/base64_codec.py +++ b/Lib/encodings/base64_codec.py @@ -76,4 +76,5 @@ incrementaldecoder=IncrementalDecoder, streamwriter=StreamWriter, streamreader=StreamReader, + _is_text_encoding=False, ) diff --git a/Lib/encodings/bz2_codec.py b/Lib/encodings/bz2_codec.py --- a/Lib/encodings/bz2_codec.py +++ b/Lib/encodings/bz2_codec.py @@ -99,4 +99,5 @@ incrementaldecoder=IncrementalDecoder, streamwriter=StreamWriter, streamreader=StreamReader, + _is_text_encoding=False, ) diff --git a/Lib/encodings/hex_codec.py b/Lib/encodings/hex_codec.py --- a/Lib/encodings/hex_codec.py +++ b/Lib/encodings/hex_codec.py @@ -76,4 +76,5 @@ incrementaldecoder=IncrementalDecoder, streamwriter=StreamWriter, streamreader=StreamReader, + _is_text_encoding=False, ) diff --git a/Lib/encodings/quopri_codec.py b/Lib/encodings/quopri_codec.py --- a/Lib/encodings/quopri_codec.py +++ b/Lib/encodings/quopri_codec.py @@ -72,4 +72,5 @@ incrementaldecoder=IncrementalDecoder, streamwriter=StreamWriter, streamreader=StreamReader, + _is_text_encoding=False, ) diff --git a/Lib/encodings/rot_13.py b/Lib/encodings/rot_13.py --- a/Lib/encodings/rot_13.py +++ b/Lib/encodings/rot_13.py @@ -44,6 +44,7 @@ incrementaldecoder=IncrementalDecoder, streamwriter=StreamWriter, streamreader=StreamReader, + _is_text_encoding=False, ) ### Decoding Map diff --git a/Lib/encodings/uu_codec.py b/Lib/encodings/uu_codec.py --- a/Lib/encodings/uu_codec.py +++ b/Lib/encodings/uu_codec.py @@ -126,4 +126,5 @@ incrementaldecoder=IncrementalDecoder, streamreader=StreamReader, streamwriter=StreamWriter, + _is_text_encoding=False, ) diff --git a/Lib/encodings/zlib_codec.py b/Lib/encodings/zlib_codec.py --- a/Lib/encodings/zlib_codec.py +++ b/Lib/encodings/zlib_codec.py @@ -99,4 +99,5 @@ incrementaldecoder=IncrementalDecoder, streamreader=StreamReader, streamwriter=StreamWriter, + _is_text_encoding=False, ) diff --git a/Lib/json/decoder.py b/Lib/json/decoder.py --- a/Lib/json/decoder.py +++ b/Lib/json/decoder.py @@ -15,10 +15,8 @@ FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL def _floatconstants(): - _BYTES = '7FF80000000000007FF0000000000000'.decode('hex') - if sys.byteorder != 'big': - _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1] - nan, inf = struct.unpack('dd', _BYTES) + nan, = struct.unpack('>d', b'\x7f\xf8\x00\x00\x00\x00\x00\x00') + inf, = struct.unpack('>d', b'\x7f\xf0\x00\x00\x00\x00\x00\x00') return nan, inf, -inf NaN, PosInf, NegInf = _floatconstants() diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py --- a/Lib/test/string_tests.py +++ b/Lib/test/string_tests.py @@ -1295,8 +1295,10 @@ ('hex', '68656c6c6f20776f726c64'), ('uu', 'begin 666 <data>\n+:&5L;&\\@=V]R;&0 \n \nend\n')] for encoding, data in codecs: - self.checkequal(data, 'hello world', 'encode', encoding) - self.checkequal('hello world', data, 'decode', encoding) + with test_support.check_py3k_warnings(): + self.checkequal(data, 'hello world', 'encode', encoding) + with test_support.check_py3k_warnings(): + self.checkequal('hello world', data, 'decode', encoding) # zlib is optional, so we make the test optional too... try: import zlib @@ -1304,8 +1306,10 @@ pass else: data = 'x\x9c\xcbH\xcd\xc9\xc9W(\xcf/\xcaI\x01\x00\x1a\x0b\x04]' - self.checkequal(data, 'hello world', 'encode', 'zlib') - self.checkequal('hello world', data, 'decode', 'zlib') + with test_support.check_py3k_warnings(): + self.checkequal(data, 'hello world', 'encode', 'zlib') + with test_support.check_py3k_warnings(): + self.checkequal('hello world', data, 'decode', 'zlib') self.checkraises(TypeError, 'xyz', 'decode', 42) self.checkraises(TypeError, 'xyz', 'encode', 42) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -513,8 +513,8 @@ def test_option_encoding(self): self.assertFailure('-e') self.assertFailure('--encoding') - stdout = self.run_ok('--encoding', 'rot-13', '2004') - self.assertEqual(stdout.strip(), conv(result_2004_text.encode('rot-13')).strip()) + stdout = self.run_ok('--encoding', 'utf-16-le', '2004') + self.assertEqual(stdout.strip(), conv(result_2004_text.encode('utf-16-le')).strip()) def test_option_locale(self): self.assertFailure('-L') diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -1395,14 +1395,14 @@ class Str2StrTest(unittest.TestCase): def test_read(self): - sin = "\x80".encode("base64_codec") + sin = codecs.encode("\x80", "base64_codec") reader = codecs.getreader("base64_codec")(StringIO.StringIO(sin)) sout = reader.read() self.assertEqual(sout, "\x80") self.assertIsInstance(sout, str) def test_readline(self): - sin = "\x80".encode("base64_codec") + sin = codecs.encode("\x80", "base64_codec") reader = codecs.getreader("base64_codec")(StringIO.StringIO(sin)) sout = reader.readline() self.assertEqual(sout, "\x80") @@ -1536,6 +1536,9 @@ ] broken_incremental_coders = broken_unicode_with_streams[:] +if sys.flags.py3k_warning: + broken_unicode_with_streams.append("rot_13") + # The following encodings only support "strict" mode only_strict_mode = [ "idna", @@ -2135,6 +2138,47 @@ # Missing "begin" line self.assertRaises(ValueError, codecs.decode, "", "uu-codec") + def test_text_to_binary_blacklists_binary_transforms(self): + # Check binary -> binary codecs give a good error for str input + bad_input = "bad input type" + for encoding in bytes_transform_encodings: + fmt = (r"{!r} is not a text encoding; " + r"use codecs.encode to handle arbitrary codecs") + msg = fmt.format(encoding) + with self.assertRaisesRegex(LookupError, msg) as failure: + bad_input.encode(encoding) + self.assertIsNone(failure.exception.__cause__) + + def test_text_to_binary_blacklists_text_transforms(self): + # Check str.encode gives a good error message for str -> str codecs + msg = (r"^'rot_13' is not a text encoding; " + r"use codecs.encode to handle arbitrary codecs") + with self.assertRaisesRegex(LookupError, msg): + "just an example message".encode("rot_13") + + def test_binary_to_text_blacklists_binary_transforms(self): + # Check bytes.decode and bytearray.decode give a good error + # message for binary -> binary codecs + data = b"encode first to ensure we meet any format restrictions" + for encoding in bytes_transform_encodings: + encoded_data = codecs.encode(data, encoding) + fmt = (r"{!r} is not a text encoding; " + r"use codecs.decode to handle arbitrary codecs") + msg = fmt.format(encoding) + with self.assertRaisesRegex(LookupError, msg): + encoded_data.decode(encoding) + with self.assertRaisesRegex(LookupError, msg): + bytearray(encoded_data).decode(encoding) + + def test_binary_to_text_blacklists_text_transforms(self): + # Check str -> str codec gives a good error for binary input + for bad_input in (b"immutable", bytearray(b"mutable")): + msg = (r"^'rot_13' is not a text encoding; " + r"use codecs.decode to handle arbitrary codecs") + with self.assertRaisesRegex(LookupError, msg) as failure: + bad_input.decode("rot_13") + self.assertIsNone(failure.exception.__cause__) + if __name__ == "__main__": test_main() diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py --- a/Lib/test/test_fileinput.py +++ b/Lib/test/test_fileinput.py @@ -211,10 +211,11 @@ except ValueError: pass try: - t1 = writeTmp(1, ["A\nB"], mode="wb") - fi = FileInput(files=t1, openhook=hook_encoded("rot13")) + # UTF-7 is a convenient, seldom used encoding + t1 = writeTmp(1, ['+AEE-\n+AEI-'], mode="wb") + fi = FileInput(files=t1, openhook=hook_encoded("utf-7")) lines = list(fi) - self.assertEqual(lines, ["N\n", "O"]) + self.assertEqual(lines, [u'A\n', u'B']) finally: remove_tempfiles(t1) diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -2001,6 +2001,15 @@ t.__init__(self.MockRawIO()) self.assertEqual(t.read(0), u'') + def test_non_text_encoding_codecs_are_rejected(self): + # Ensure the constructor complains if passed a codec that isn't + # marked as a text encoding + # http://bugs.python.org/issue20404 + r = self.BytesIO() + b = self.BufferedWriter(r) + with support.check_py3k_warnings(): + self.TextIOWrapper(b, encoding="hex_codec") + def test_detach(self): r = self.BytesIO() b = self.BufferedWriter(r) @@ -2617,19 +2626,39 @@ def test_illegal_decoder(self): # Issue #17106 + # Bypass the early encoding check added in issue 20404 + def _make_illegal_wrapper(): + quopri = codecs.lookup("quopri_codec") + quopri._is_text_encoding = True + try: + t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), + newline='\n', encoding="quopri_codec") + finally: + quopri._is_text_encoding = False + return t # Crash when decoder returns non-string - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + with support.check_py3k_warnings(): + t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', + encoding='quopri_codec') with self.maybeRaises(TypeError): t.read(1) - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + with support.check_py3k_warnings(): + t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', + encoding='quopri_codec') with self.maybeRaises(TypeError): t.readline() - t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', - encoding='quopri_codec') + with support.check_py3k_warnings(): + t = self.TextIOWrapper(self.BytesIO(b'aaaaaa'), newline='\n', + encoding='quopri_codec') with self.maybeRaises(TypeError): t.read() + #else: + #t = _make_illegal_wrapper() + #self.assertRaises(TypeError, t.read, 1) + #t = _make_illegal_wrapper() + #self.assertRaises(TypeError, t.readline) + #t = _make_illegal_wrapper() + #self.assertRaises(TypeError, t.read) class CTextIOWrapperTest(TextIOWrapperTest): @@ -3002,9 +3031,11 @@ class CMiscIOTest(MiscIOTest): io = io + shutdown_error = "RuntimeError: could not find io module state" class PyMiscIOTest(MiscIOTest): io = pyio + shutdown_error = "LookupError: unknown encoding: ascii" @unittest.skipIf(os.name == 'nt', 'POSIX signals required for this test.') diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,10 @@ Core and Builtins ----------------- +- Issue #19543: encode() and decode() methods and constructors of str, + unicode and bytearray classes now emit deprecation warning for known + non-text encodings when Python is ran with the -3 option. + - Issue #24115: Update uses of PyObject_IsTrue(), PyObject_Not(), PyObject_IsInstance(), PyObject_RichCompareBool() and _PyDict_Contains() to check for and handle errors correctly. @@ -26,6 +30,10 @@ Library ------- +- Issue #19543: io.TextIOWrapper (and hence io.open()) now uses the internal + codec marking system added to emit deprecation warning for known non-text + encodings at stream construction time when Python is ran with the -3 option. + - Issue #24264: Fixed buffer overflow in the imageop module. - Issue #5633: Fixed timeit when the statement is a string and the setup is not. diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -826,7 +826,7 @@ char *kwlist[] = {"buffer", "encoding", "errors", "newline", "line_buffering", NULL}; - PyObject *buffer, *raw; + PyObject *buffer, *raw, *codec_info = NULL; char *encoding = NULL; char *errors = NULL; char *newline = NULL; @@ -909,6 +909,17 @@ "could not determine default encoding"); } + /* Check we have been asked for a real text encoding */ + codec_info = _PyCodec_LookupTextEncoding(encoding, "codecs.open()"); + if (codec_info == NULL) { + Py_CLEAR(self->encoding); + goto error; + } + + /* XXX: Failures beyond this point have the potential to leak elements + * of the partially constructed object (like self->encoding) + */ + if (errors == NULL) errors = "strict"; self->errors = PyBytes_FromString(errors); @@ -922,7 +933,7 @@ if (newline) { self->readnl = PyString_FromString(newline); if (self->readnl == NULL) - return -1; + goto error; } self->writetranslate = (newline == NULL || newline[0] != '\0'); if (!self->readuniversal && self->writetranslate) { @@ -944,8 +955,8 @@ if (r == -1) goto error; if (r == 1) { - self->decoder = PyCodec_IncrementalDecoder( - encoding, errors); + self->decoder = _PyCodecInfo_GetIncrementalDecoder(codec_info, + errors); if (self->decoder == NULL) goto error; @@ -969,17 +980,12 @@ if (r == -1) goto error; if (r == 1) { - PyObject *ci; - self->encoder = PyCodec_IncrementalEncoder( - encoding, errors); + self->encoder = _PyCodecInfo_GetIncrementalEncoder(codec_info, + errors); if (self->encoder == NULL) goto error; /* Get the normalized named of the codec */ - ci = _PyCodec_Lookup(encoding); - if (ci == NULL) - goto error; - res = PyObject_GetAttrString(ci, "name"); - Py_DECREF(ci); + res = PyObject_GetAttrString(codec_info, "name"); if (res == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError)) PyErr_Clear(); @@ -999,6 +1005,9 @@ Py_XDECREF(res); } + /* Finished sorting out the codec details */ + Py_DECREF(codec_info); + self->buffer = buffer; Py_INCREF(buffer); @@ -1059,6 +1068,7 @@ return 0; error: + Py_XDECREF(codec_info); return -1; } diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -783,7 +783,7 @@ if (PyBytes_Check(arg)) { PyObject *new, *encoded; if (encoding != NULL) { - encoded = PyCodec_Encode(arg, encoding, errors); + encoded = _PyCodec_EncodeText(arg, encoding, errors); if (encoded == NULL) return -1; assert(PyBytes_Check(encoded)); @@ -809,7 +809,7 @@ "unicode argument without an encoding"); return -1; } - encoded = PyCodec_Encode(arg, encoding, errors); + encoded = _PyCodec_EncodeText(arg, encoding, errors); if (encoded == NULL) return -1; assert(PyBytes_Check(encoded)); @@ -2567,7 +2567,7 @@ return NULL; #endif } - return PyCodec_Decode(self, encoding, errors); + return _PyCodec_DecodeText(self, encoding, errors); } PyDoc_STRVAR(alloc_doc, diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -449,7 +449,7 @@ } /* Decode via the codec registry */ - v = PyCodec_Decode(str, encoding, errors); + v = _PyCodec_DecodeText(str, encoding, errors); if (v == NULL) goto onError; @@ -529,7 +529,7 @@ } /* Encode via the codec registry */ - v = PyCodec_Encode(str, encoding, errors); + v = _PyCodec_EncodeText(str, encoding, errors); if (v == NULL) goto onError; diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -1259,7 +1259,7 @@ buffer = PyBuffer_FromMemory((void *)s, size); if (buffer == NULL) goto onError; - unicode = PyCodec_Decode(buffer, encoding, errors); + unicode = _PyCodec_DecodeText(buffer, encoding, errors); if (unicode == NULL) goto onError; if (!PyUnicode_Check(unicode)) { @@ -1292,7 +1292,7 @@ encoding = PyUnicode_GetDefaultEncoding(); /* Decode via the codec registry */ - v = PyCodec_Decode(unicode, encoding, errors); + v = _PyCodec_DecodeText(unicode, encoding, errors); if (v == NULL) goto onError; return v; @@ -1331,7 +1331,7 @@ encoding = PyUnicode_GetDefaultEncoding(); /* Encode via the codec registry */ - v = PyCodec_Encode(unicode, encoding, errors); + v = _PyCodec_EncodeText(unicode, encoding, errors); if (v == NULL) goto onError; return v; @@ -1369,7 +1369,7 @@ } /* Encode via the codec registry */ - v = PyCodec_Encode(unicode, encoding, errors); + v = _PyCodec_EncodeText(unicode, encoding, errors); if (v == NULL) goto onError; if (!PyString_Check(v)) { diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -217,20 +217,15 @@ return v; } -/* Helper function to create an incremental codec. */ +/* Helper functions to create an incremental codec. */ +static +PyObject *codec_makeincrementalcodec(PyObject *codec_info, + const char *errors, + const char *attrname) +{ + PyObject *ret, *inccodec; -static -PyObject *codec_getincrementalcodec(const char *encoding, - const char *errors, - const char *attrname) -{ - PyObject *codecs, *ret, *inccodec; - - codecs = _PyCodec_Lookup(encoding); - if (codecs == NULL) - return NULL; - inccodec = PyObject_GetAttrString(codecs, attrname); - Py_DECREF(codecs); + inccodec = PyObject_GetAttrString(codec_info, attrname); if (inccodec == NULL) return NULL; if (errors) @@ -241,6 +236,21 @@ return ret; } +static +PyObject *codec_getincrementalcodec(const char *encoding, + const char *errors, + const char *attrname) +{ + PyObject *codec_info, *ret; + + codec_info = _PyCodec_Lookup(encoding); + if (codec_info == NULL) + return NULL; + ret = codec_makeincrementalcodec(codec_info, errors, attrname); + Py_DECREF(codec_info); + return ret; +} + /* Helper function to create a stream codec. */ static @@ -264,6 +274,24 @@ return streamcodec; } +/* Helpers to work with the result of _PyCodec_Lookup + + */ +PyObject *_PyCodecInfo_GetIncrementalDecoder(PyObject *codec_info, + const char *errors) +{ + return codec_makeincrementalcodec(codec_info, errors, + "incrementaldecoder"); +} + +PyObject *_PyCodecInfo_GetIncrementalEncoder(PyObject *codec_info, + const char *errors) +{ + return codec_makeincrementalcodec(codec_info, errors, + "incrementalencoder"); +} + + /* Convenience APIs to query the Codec registry. All APIs return a codec object with incremented refcount. @@ -311,18 +339,15 @@ errors is passed to the encoder factory as argument if non-NULL. */ -PyObject *PyCodec_Encode(PyObject *object, - const char *encoding, - const char *errors) +static PyObject * +_PyCodec_EncodeInternal(PyObject *object, + PyObject *encoder, + const char *encoding, + const char *errors) { - PyObject *encoder = NULL; PyObject *args = NULL, *result = NULL; PyObject *v; - encoder = PyCodec_Encoder(encoding); - if (encoder == NULL) - goto onError; - args = args_tuple(object, errors); if (args == NULL) goto onError; @@ -358,18 +383,15 @@ errors is passed to the decoder factory as argument if non-NULL. */ -PyObject *PyCodec_Decode(PyObject *object, - const char *encoding, - const char *errors) +static PyObject * +_PyCodec_DecodeInternal(PyObject *object, + PyObject *decoder, + const char *encoding, + const char *errors) { - PyObject *decoder = NULL; PyObject *args = NULL, *result = NULL; PyObject *v; - decoder = PyCodec_Decoder(encoding); - if (decoder == NULL) - goto onError; - args = args_tuple(object, errors); if (args == NULL) goto onError; @@ -399,6 +421,139 @@ return NULL; } +/* Generic encoding/decoding API */ +PyObject *PyCodec_Encode(PyObject *object, + const char *encoding, + const char *errors) +{ + PyObject *encoder; + + encoder = PyCodec_Encoder(encoding); + if (encoder == NULL) + return NULL; + + return _PyCodec_EncodeInternal(object, encoder, encoding, errors); +} + +PyObject *PyCodec_Decode(PyObject *object, + const char *encoding, + const char *errors) +{ + PyObject *decoder; + + decoder = PyCodec_Decoder(encoding); + if (decoder == NULL) + return NULL; + + return _PyCodec_DecodeInternal(object, decoder, encoding, errors); +} + +/* Text encoding/decoding API */ +PyObject * _PyCodec_LookupTextEncoding(const char *encoding, + const char *alternate_command) +{ + PyObject *codec; + PyObject *attr; + int is_text_codec; + + codec = _PyCodec_Lookup(encoding); + if (codec == NULL) + return NULL; + + /* Backwards compatibility: assume any raw tuple describes a text + * encoding, and the same for anything lacking the private + * attribute. + */ + if (Py_Py3kWarningFlag && !PyTuple_CheckExact(codec)) { + attr = PyObject_GetAttrString(codec, "_is_text_encoding"); + if (attr == NULL) { + if (!PyErr_ExceptionMatches(PyExc_AttributeError)) + goto onError; + PyErr_Clear(); + } else { + is_text_codec = PyObject_IsTrue(attr); + Py_DECREF(attr); + if (is_text_codec < 0) + goto onError; + if (!is_text_codec) { + PyObject *msg = PyString_FromFormat( + "'%.400s' is not a text encoding; " + "use %s to handle arbitrary codecs", + encoding, alternate_command); + if (msg == NULL) + goto onError; + if (PyErr_WarnPy3k(PyString_AS_STRING(msg), 1) < 0) { + Py_DECREF(msg); + goto onError; + } + Py_DECREF(msg); + } + } + } + + /* This appears to be a valid text encoding */ + return codec; + + onError: + Py_DECREF(codec); + return NULL; +} + + +static +PyObject *codec_getitem_checked(const char *encoding, + const char *alternate_command, + int index) +{ + PyObject *codec; + PyObject *v; + + codec = _PyCodec_LookupTextEncoding(encoding, alternate_command); + if (codec == NULL) + return NULL; + + v = PyTuple_GET_ITEM(codec, index); + Py_INCREF(v); + Py_DECREF(codec); + return v; +} + +static PyObject * _PyCodec_TextEncoder(const char *encoding) +{ + return codec_getitem_checked(encoding, "codecs.encode()", 0); +} + +static PyObject * _PyCodec_TextDecoder(const char *encoding) +{ + return codec_getitem_checked(encoding, "codecs.decode()", 1); +} + +PyObject *_PyCodec_EncodeText(PyObject *object, + const char *encoding, + const char *errors) +{ + PyObject *encoder; + + encoder = _PyCodec_TextEncoder(encoding); + if (encoder == NULL) + return NULL; + + return _PyCodec_EncodeInternal(object, encoder, encoding, errors); +} + +PyObject *_PyCodec_DecodeText(PyObject *object, + const char *encoding, + const char *errors) +{ + PyObject *decoder; + + decoder = _PyCodec_TextDecoder(encoding); + if (decoder == NULL) + return NULL; + + return _PyCodec_DecodeInternal(object, decoder, encoding, errors); +} + /* Register the error handling callback function error under the name name. This function will be called by the codec when it encounters an unencodable characters/undecodable bytes and doesn't know the -- Repository URL: https://hg.python.org/cpython [View Less]

1 0

cpython (merge 3.5 -> default): Issue 24004: Fix DeprecationWarning in a unittest
by yury.selivanov May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/60f5091cbfbf changeset: 96424:60f5091cbfbf parent: 96420:04162e06323f parent: 96423:8296f0119f20 user: Yury Selivanov <yselivanov(a)sprymix.com> date: Sun May 31 11:28:57 2015 -0400 summary: Issue 24004: Fix DeprecationWarning in a unittest (Merge from 3.5) files: Lib/test/test_asyncio/test_pep492.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_pep492.py b/Lib/test/… [View More]

1 0

cpython (3.5): Issue 24004: Fix DeprecationWarning in a unittest
by yury.selivanov May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/8296f0119f20 changeset: 96423:8296f0119f20 branch: 3.5 parent: 96419:a82498f424fe user: Yury Selivanov <yselivanov(a)sprymix.com> date: Sun May 31 11:28:35 2015 -0400 summary: Issue 24004: Fix DeprecationWarning in a unittest files: Lib/test/test_asyncio/test_pep492.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_asyncio/test_pep492.py b/Lib/test/test_asyncio/test_pep492.py --- a/… [View More]

1 0

Daily reference leaks (85c22fe713a3): sum=709
by solipsis＠pitrou.net May 31, 2015

May 31, 2015

results for 85c22fe713a3 on branch "default" -------------------------------------------- test_asyncio leaked [0, 0, 3] memory blocks, sum=3 test_collections leaked [122, 122, 122] references, sum=366 test_collections leaked [38, 38, 38] memory blocks, sum=114 test_functools leaked [0, 2, 2] memory blocks, sum=4 test_importlib leaked [50, 50, 50] references, sum=150 test_importlib leaked [24, 24, 24] memory blocks, sum=72 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/refloghRW0HO', '--timeout', '7200']

1 0

cpython (2.7): Fixed test_audioop for issue #24326.
by serhiy.storchaka May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/7d6a6028b104 changeset: 96422:7d6a6028b104 branch: 2.7 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Sun May 31 12:05:40 2015 +0300 summary: Fixed test_audioop for issue #24326. 24-bit samples are not supported in 2.7. files: Lib/test/test_audioop.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_audioop.py b/Lib/test/test_audioop.py --- a/Lib/test/test_audioop.py +++ b/Lib/test/… [View More]

1 0

cpython (2.7): Fixed the array module in unicode disabled build (regression of issue20014).
by serhiy.storchaka May 31, 2015

May 31, 2015

https://hg.python.org/cpython/rev/6ee122fac1e8 changeset: 96421:6ee122fac1e8 branch: 2.7 parent: 96418:de0ccaaf2e64 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Sun May 31 11:56:48 2015 +0300 summary: Fixed the array module in unicode disabled build (regression of issue20014). files: Lib/test/test_array.py | 5 ++++- Modules/arraymodule.c | 2 ++ 2 files changed, 6 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_array.py b/Lib/test/… [View More]

1 0