Mailman 3 November 2018 - Python-checkins

bpo-34532: Fixed exit code for py.exe list versions arg (GH-9039)
by Miss Islington (bot) 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/129642a1ff6a2f55933186b2303c307d58… commit: 129642a1ff6a2f55933186b2303c307d58b710eb branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T13:48:34-08:00 summary: bpo-34532: Fixed exit code for py.exe list versions arg (GH-9039) (cherry picked from commit c8fe9ccf7bfbcf9a2cb48e3b6abacf6ec0e5e58d) Co-authored-by: Brendan Gerrity <brerrity(a)gmail.com> files: A Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst M PC/launcher.c diff --git a/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst b/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst new file mode 100644 index 000000000000..812b47497c2d --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst @@ -0,0 +1 @@ +Fixes exit code of list version arguments for py.exe. diff --git a/PC/launcher.c b/PC/launcher.c index eb3433ab53fe..7d666aae4ab1 100644 --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1453,7 +1453,7 @@ show_python_list(wchar_t ** argv) fwprintf(stderr, L"\n\nCan't find a Default Python.\n\n"); else fwprintf(stderr, L"\n\n"); /* End with a blank line */ - return(FALSE); /* If this has been called we cannot continue */ + return FALSE; /* If this has been called we cannot continue */ } static int @@ -1600,11 +1600,12 @@ process(int argc, wchar_t ** argv) else { p = argv[1]; plen = wcslen(p); - if ((argc == 2) && - (!wcsncmp(p, L"-0", wcslen(L"-0")) || /* Starts with -0 or --list */ + if ((argc == 2) && // list version args + (!wcsncmp(p, L"-0", wcslen(L"-0")) || !wcsncmp(p, L"--list", wcslen(L"--list")))) { - valid = show_python_list(argv); /* Check for -0 or --list FIRST */ + show_python_list(argv); + return rc; } valid = valid && (*p == L'-') && validate_version(&p[1]); if (valid) {

1 0

bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) (GH-10619) (GH-10621)
by Victor Stinner 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/df3051b53fd7f2862a4087f5449e811d84… commit: df3051b53fd7f2862a4087f5449e811d8421347a branch: 3.6 author: Victor Stinner <vstinner(a)redhat.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T22:36:15+01:00 summary: bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) (GH-10619) (GH-10621) locale.localeconv() now sets temporarily the LC_CTYPE locale to the LC_MONETARY locale if the two locales are different and monetary strings are non-ASCII. This temporary change affects other threads. Changes: * locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode monetary fields. * Add LocaleInfo.grouping_buffer: copy localeconv() grouping string since it can be replaced anytime if a different thread calls localeconv(). (cherry picked from commit 02e6bf7f2025cddcbde6432f6b6396198ab313f4) (cherry picked from commit 6eff6b8eecd7a8eccad16419269fa18ec820922e) files: A Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst M Doc/library/locale.rst M Modules/_localemodule.c M Python/fileutils.c M Python/formatter_unicode.c diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index 9a0c570533a1..2addc0ea6622 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -148,10 +148,8 @@ The :mod:`locale` module defines the following exception and functions: +--------------+-----------------------------------------+ The function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` - locale to decode ``decimal_point`` and ``thousands_sep`` byte strings if - they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is - different than the ``LC_CTYPE`` locale. This temporary change affects other - threads. + locale or the ``LC_MONETARY`` locale if locales are different and numeric or + monetary strings are non-ASCII. This temporary change affects other threads. .. versionchanged:: 3.6.5 The function now sets temporarily the ``LC_CTYPE`` locale to the diff --git a/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst b/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst new file mode 100644 index 000000000000..289e484c35d6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst @@ -0,0 +1,3 @@ +:func:`locale.localeconv` now sets temporarily the ``LC_CTYPE`` locale to the +``LC_MONETARY`` locale if the two locales are different and monetary strings +are non-ASCII. This temporary change affects other threads. diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index 95b370b1ad08..f3421af65f3d 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -128,6 +128,82 @@ PyLocale_setlocale(PyObject* self, PyObject* args) return result_object; } +static int +locale_is_ascii(const char *str) +{ + return (strlen(str) == 1 && ((unsigned char)str[0]) <= 127); +} + +static int +locale_decode_monetary(PyObject *dict, struct lconv *lc) +{ + int change_locale; + change_locale = (!locale_is_ascii(lc->int_curr_symbol) + || !locale_is_ascii(lc->currency_symbol) + || !locale_is_ascii(lc->mon_decimal_point) + || !locale_is_ascii(lc->mon_thousands_sep)); + + /* Keep a copy of the LC_CTYPE locale */ + char *oldloc = NULL, *loc = NULL; + if (change_locale) { + oldloc = setlocale(LC_CTYPE, NULL); + if (!oldloc) { + PyErr_SetString(PyExc_RuntimeWarning, + "failed to get LC_CTYPE locale"); + return -1; + } + + oldloc = _PyMem_Strdup(oldloc); + if (!oldloc) { + PyErr_NoMemory(); + return -1; + } + + loc = setlocale(LC_MONETARY, NULL); + if (loc != NULL && strcmp(loc, oldloc) == 0) { + loc = NULL; + } + + if (loc != NULL) { + /* Only set the locale temporarily the LC_CTYPE locale + to the LC_MONETARY locale if the two locales are different and + at least one string is non-ASCII. */ + setlocale(LC_CTYPE, loc); + } + } + + int res = -1; + +#define RESULT_STRING(ATTR) \ + do { \ + PyObject *obj; \ + obj = PyUnicode_DecodeLocale(lc->ATTR, NULL); \ + if (obj == NULL) { \ + goto done; \ + } \ + if (PyDict_SetItemString(dict, Py_STRINGIFY(ATTR), obj) < 0) { \ + Py_DECREF(obj); \ + goto done; \ + } \ + Py_DECREF(obj); \ + } while (0) + + RESULT_STRING(int_curr_symbol); + RESULT_STRING(currency_symbol); + RESULT_STRING(mon_decimal_point); + RESULT_STRING(mon_thousands_sep); +#undef RESULT_STRING + + res = 0; + +done: + if (loc != NULL) { + setlocale(LC_CTYPE, oldloc); + } + PyMem_Free(oldloc); + return res; +} + PyDoc_STRVAR(localeconv__doc__, "() -> dict. Returns numeric and monetary locale-specific parameters."); @@ -171,11 +247,10 @@ PyLocale_localeconv(PyObject* self) RESULT(#i, x); \ } while (0) - /* Monetary information */ - RESULT_STRING(int_curr_symbol); - RESULT_STRING(currency_symbol); - RESULT_STRING(mon_decimal_point); - RESULT_STRING(mon_thousands_sep); + /* Monetary information: LC_MONETARY encoding */ + if (locale_decode_monetary(result, l) < 0) { + goto failed; + } x = copy_grouping(l->mon_grouping); RESULT("mon_grouping", x); @@ -190,7 +265,7 @@ PyLocale_localeconv(PyObject* self) RESULT_INT(p_sign_posn); RESULT_INT(n_sign_posn); - /* Numeric information */ + /* Numeric information: LC_NUMERIC encoding */ PyObject *decimal_point, *thousands_sep; const char *grouping; if (_Py_GetLocaleconvNumeric(&decimal_point, @@ -220,6 +295,10 @@ PyLocale_localeconv(PyObject* self) failed: Py_XDECREF(result); return NULL; + +#undef RESULT +#undef RESULT_STRING +#undef RESULT_INT } #if defined(HAVE_WCSCOLL) diff --git a/Python/fileutils.c b/Python/fileutils.c index 306838ecce44..d1a9ca03a134 100644 --- a/Python/fileutils.c +++ b/Python/fileutils.c @@ -1687,7 +1687,7 @@ _Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep, if (change_locale) { oldloc = setlocale(LC_CTYPE, NULL); if (!oldloc) { - PyErr_SetString(PyExc_RuntimeWarning, "faild to get LC_CTYPE locale"); + PyErr_SetString(PyExc_RuntimeWarning, "failed to get LC_CTYPE locale"); return -1; } @@ -1703,7 +1703,7 @@ _Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep, } if (loc != NULL) { - /* Only set the locale temporarilty the LC_CTYPE locale + /* Only set the locale temporarily the LC_CTYPE locale if LC_NUMERIC locale is different than LC_CTYPE locale and decimal_point and/or thousands_sep are non-ASCII or longer than 1 byte */ diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c index d6772c593877..c9a2c99dd257 100644 --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -397,9 +397,10 @@ typedef struct { PyObject *decimal_point; PyObject *thousands_sep; const char *grouping; + char *grouping_buffer; } LocaleInfo; -#define STATIC_LOCALE_INFO_INIT {0, 0, 0} +#define STATIC_LOCALE_INFO_INIT {0, 0, 0, 0} /* describes the layout for an integer, see the comment in calc_number_widths() for details */ @@ -708,11 +709,22 @@ get_locale_info(enum LocaleType type, LocaleInfo *locale_info) { switch (type) { case LT_CURRENT_LOCALE: { + const char *grouping; if (_Py_GetLocaleconvNumeric(&locale_info->decimal_point, &locale_info->thousands_sep, - &locale_info->grouping) < 0) { + &grouping) < 0) { return -1; } + + /* localeconv() grouping can become a dangling pointer or point + to a different string if another thread calls localeconv() during + the string formatting. Copy the string to avoid this risk. */ + locale_info->grouping_buffer = _PyMem_Strdup(grouping); + if (locale_info->grouping_buffer == NULL) { + PyErr_NoMemory(); + return -1; + } + locale_info->grouping = locale_info->grouping_buffer; break; } case LT_DEFAULT_LOCALE: @@ -746,6 +758,7 @@ free_locale_info(LocaleInfo *locale_info) { Py_XDECREF(locale_info->decimal_point); Py_XDECREF(locale_info->thousands_sep); + PyMem_Free(locale_info->grouping_buffer); } /************************************************************************/

1 0

bpo-34532: Fixed exit code for py.exe list versions arg (GH-9039)
by Steve Dower 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/c8fe9ccf7bfbcf9a2cb48e3b6abacf6ec0… commit: c8fe9ccf7bfbcf9a2cb48e3b6abacf6ec0e5e58d branch: master author: Brendan Gerrity <brerrity(a)gmail.com> committer: Steve Dower <steve.dower(a)microsoft.com> date: 2018-11-20T13:28:27-08:00 summary: bpo-34532: Fixed exit code for py.exe list versions arg (GH-9039) files: A Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst M PC/launcher.c diff --git a/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst b/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst new file mode 100644 index 000000000000..812b47497c2d --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2018-09-03-01-23-52.bpo-34532.N1HEbE.rst @@ -0,0 +1 @@ +Fixes exit code of list version arguments for py.exe. diff --git a/PC/launcher.c b/PC/launcher.c index 75a06c2310c8..2c2da76f6146 100644 --- a/PC/launcher.c +++ b/PC/launcher.c @@ -1454,7 +1454,7 @@ show_python_list(wchar_t ** argv) fwprintf(stderr, L"\n\nCan't find a Default Python.\n\n"); else fwprintf(stderr, L"\n\n"); /* End with a blank line */ - return(FALSE); /* If this has been called we cannot continue */ + return FALSE; /* If this has been called we cannot continue */ } static int @@ -1601,11 +1601,12 @@ process(int argc, wchar_t ** argv) else { p = argv[1]; plen = wcslen(p); - if ((argc == 2) && - (!wcsncmp(p, L"-0", wcslen(L"-0")) || /* Starts with -0 or --list */ + if ((argc == 2) && // list version args + (!wcsncmp(p, L"-0", wcslen(L"-0")) || !wcsncmp(p, L"--list", wcslen(L"--list")))) { - valid = show_python_list(argv); /* Check for -0 or --list FIRST */ + show_python_list(argv); + return rc; } valid = valid && (*p == L'-') && validate_version(&p[1]); if (valid) {

1 0

bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) (GH-10619)
by Victor Stinner 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/6eff6b8eecd7a8eccad16419269fa18ec8… commit: 6eff6b8eecd7a8eccad16419269fa18ec820922e branch: 3.7 author: Victor Stinner <vstinner(a)redhat.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T22:06:21+01:00 summary: bpo-28604: Fix localeconv() for different LC_MONETARY (GH-10606) (GH-10619) locale.localeconv() now sets temporarily the LC_CTYPE locale to the LC_MONETARY locale if the two locales are different and monetary strings are non-ASCII. This temporary change affects other threads. Changes: * locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode monetary fields. * Add LocaleInfo.grouping_buffer: copy localeconv() grouping string since it can be replaced anytime if a different thread calls localeconv(). (cherry picked from commit 02e6bf7f2025cddcbde6432f6b6396198ab313f4) files: A Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst M Doc/library/locale.rst M Modules/_localemodule.c M Python/fileutils.c M Python/formatter_unicode.c diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index 2fd44fe8e90a..bf57a0835591 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -148,10 +148,8 @@ The :mod:`locale` module defines the following exception and functions: +--------------+-----------------------------------------+ The function sets temporarily the ``LC_CTYPE`` locale to the ``LC_NUMERIC`` - locale to decode ``decimal_point`` and ``thousands_sep`` byte strings if - they are non-ASCII or longer than 1 byte, and the ``LC_NUMERIC`` locale is - different than the ``LC_CTYPE`` locale. This temporary change affects other - threads. + locale or the ``LC_MONETARY`` locale if locales are different and numeric or + monetary strings are non-ASCII. This temporary change affects other threads. .. versionchanged:: 3.7 The function now sets temporarily the ``LC_CTYPE`` locale to the diff --git a/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst b/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst new file mode 100644 index 000000000000..289e484c35d6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-11-20-13-34-01.bpo-28604.iiih5h.rst @@ -0,0 +1,3 @@ +:func:`locale.localeconv` now sets temporarily the ``LC_CTYPE`` locale to the +``LC_MONETARY`` locale if the two locales are different and monetary strings +are non-ASCII. This temporary change affects other threads. diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index f9eeeb72dd9e..716a7306d3dc 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -128,6 +128,82 @@ PyLocale_setlocale(PyObject* self, PyObject* args) return result_object; } +static int +locale_is_ascii(const char *str) +{ + return (strlen(str) == 1 && ((unsigned char)str[0]) <= 127); +} + +static int +locale_decode_monetary(PyObject *dict, struct lconv *lc) +{ + int change_locale; + change_locale = (!locale_is_ascii(lc->int_curr_symbol) + || !locale_is_ascii(lc->currency_symbol) + || !locale_is_ascii(lc->mon_decimal_point) + || !locale_is_ascii(lc->mon_thousands_sep)); + + /* Keep a copy of the LC_CTYPE locale */ + char *oldloc = NULL, *loc = NULL; + if (change_locale) { + oldloc = setlocale(LC_CTYPE, NULL); + if (!oldloc) { + PyErr_SetString(PyExc_RuntimeWarning, + "failed to get LC_CTYPE locale"); + return -1; + } + + oldloc = _PyMem_Strdup(oldloc); + if (!oldloc) { + PyErr_NoMemory(); + return -1; + } + + loc = setlocale(LC_MONETARY, NULL); + if (loc != NULL && strcmp(loc, oldloc) == 0) { + loc = NULL; + } + + if (loc != NULL) { + /* Only set the locale temporarily the LC_CTYPE locale + to the LC_MONETARY locale if the two locales are different and + at least one string is non-ASCII. */ + setlocale(LC_CTYPE, loc); + } + } + + int res = -1; + +#define RESULT_STRING(ATTR) \ + do { \ + PyObject *obj; \ + obj = PyUnicode_DecodeLocale(lc->ATTR, NULL); \ + if (obj == NULL) { \ + goto done; \ + } \ + if (PyDict_SetItemString(dict, Py_STRINGIFY(ATTR), obj) < 0) { \ + Py_DECREF(obj); \ + goto done; \ + } \ + Py_DECREF(obj); \ + } while (0) + + RESULT_STRING(int_curr_symbol); + RESULT_STRING(currency_symbol); + RESULT_STRING(mon_decimal_point); + RESULT_STRING(mon_thousands_sep); +#undef RESULT_STRING + + res = 0; + +done: + if (loc != NULL) { + setlocale(LC_CTYPE, oldloc); + } + PyMem_Free(oldloc); + return res; +} + PyDoc_STRVAR(localeconv__doc__, "() -> dict. Returns numeric and monetary locale-specific parameters."); @@ -172,11 +248,10 @@ PyLocale_localeconv(PyObject* self) RESULT(#i, x); \ } while (0) - /* Monetary information */ - RESULT_STRING(int_curr_symbol); - RESULT_STRING(currency_symbol); - RESULT_STRING(mon_decimal_point); - RESULT_STRING(mon_thousands_sep); + /* Monetary information: LC_MONETARY encoding */ + if (locale_decode_monetary(result, l) < 0) { + goto failed; + } x = copy_grouping(l->mon_grouping); RESULT("mon_grouping", x); @@ -191,7 +266,7 @@ PyLocale_localeconv(PyObject* self) RESULT_INT(p_sign_posn); RESULT_INT(n_sign_posn); - /* Numeric information */ + /* Numeric information: LC_NUMERIC encoding */ PyObject *decimal_point, *thousands_sep; const char *grouping; if (_Py_GetLocaleconvNumeric(&decimal_point, @@ -221,6 +296,10 @@ PyLocale_localeconv(PyObject* self) failed: Py_DECREF(result); return NULL; + +#undef RESULT +#undef RESULT_STRING +#undef RESULT_INT } #if defined(HAVE_WCSCOLL) diff --git a/Python/fileutils.c b/Python/fileutils.c index b3b7925073b2..b77e490ce236 100644 --- a/Python/fileutils.c +++ b/Python/fileutils.c @@ -1801,7 +1801,7 @@ _Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep, if (change_locale) { oldloc = setlocale(LC_CTYPE, NULL); if (!oldloc) { - PyErr_SetString(PyExc_RuntimeWarning, "faild to get LC_CTYPE locale"); + PyErr_SetString(PyExc_RuntimeWarning, "failed to get LC_CTYPE locale"); return -1; } @@ -1817,7 +1817,7 @@ _Py_GetLocaleconvNumeric(PyObject **decimal_point, PyObject **thousands_sep, } if (loc != NULL) { - /* Only set the locale temporarilty the LC_CTYPE locale + /* Only set the locale temporarily the LC_CTYPE locale if LC_NUMERIC locale is different than LC_CTYPE locale and decimal_point and/or thousands_sep are non-ASCII or longer than 1 byte */ diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c index ba09cc67becf..2cd3eb8eb750 100644 --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -396,9 +396,10 @@ typedef struct { PyObject *decimal_point; PyObject *thousands_sep; const char *grouping; + char *grouping_buffer; } LocaleInfo; -#define STATIC_LOCALE_INFO_INIT {0, 0, 0} +#define STATIC_LOCALE_INFO_INIT {0, 0, 0, 0} /* describes the layout for an integer, see the comment in calc_number_widths() for details */ @@ -705,11 +706,22 @@ get_locale_info(enum LocaleType type, LocaleInfo *locale_info) { switch (type) { case LT_CURRENT_LOCALE: { + const char *grouping; if (_Py_GetLocaleconvNumeric(&locale_info->decimal_point, &locale_info->thousands_sep, - &locale_info->grouping) < 0) { + &grouping) < 0) { return -1; } + + /* localeconv() grouping can become a dangling pointer or point + to a different string if another thread calls localeconv() during + the string formatting. Copy the string to avoid this risk. */ + locale_info->grouping_buffer = _PyMem_Strdup(grouping); + if (locale_info->grouping_buffer == NULL) { + PyErr_NoMemory(); + return -1; + } + locale_info->grouping = locale_info->grouping_buffer; break; } case LT_DEFAULT_LOCALE: @@ -743,6 +755,7 @@ free_locale_info(LocaleInfo *locale_info) { Py_XDECREF(locale_info->decimal_point); Py_XDECREF(locale_info->thousands_sep); + PyMem_Free(locale_info->grouping_buffer); } /************************************************************************/

1 0

[2.7] bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039) (GH-10617)
by Serhiy Storchaka 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/40fdf471931f029ea07e6190f0fe116e07… commit: 40fdf471931f029ea07e6190f0fe116e0735661b branch: 2.7 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T21:56:34+02:00 summary: [2.7] bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039) (GH-10617) Fixes assertion failures in _datetimemodule.c introduced in the previous fix (see bpo-31752). Rather of trying to handle an int subclass as exact int, let it to use overridden special methods, but check the result of divmod(). (cherry picked from commit 3ec0f495163da3b7a15deb2805cec48aed432f58) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: M Lib/test/test_datetime.py M Modules/datetimemodule.c diff --git a/Lib/test/test_datetime.py b/Lib/test/test_datetime.py index f56c3f5fe3de..2620fbb9fbf7 100644 --- a/Lib/test/test_datetime.py +++ b/Lib/test/test_datetime.py @@ -495,41 +495,64 @@ def as_hours(self): def test_issue31752(self): # The interpreter shouldn't crash because divmod() returns negative # remainder. - class BadInt(int): - def __mul__(self, other): - return Prod() - - class BadLong(long): - def __mul__(self, other): - return Prod() - - class Prod: - def __radd__(self, other): - return Sum() - - class Sum(int): - def __divmod__(self, other): - # negative remainder - return (0, -1) - - timedelta(microseconds=BadInt(1)) - timedelta(hours=BadInt(1)) - timedelta(weeks=BadInt(1)) - timedelta(microseconds=BadLong(1)) - timedelta(hours=BadLong(1)) - timedelta(weeks=BadLong(1)) - - class Sum(long): - def __divmod__(self, other): - # negative remainder - return (0, -1) - - timedelta(microseconds=BadInt(1)) - timedelta(hours=BadInt(1)) - timedelta(weeks=BadInt(1)) - timedelta(microseconds=BadLong(1)) - timedelta(hours=BadLong(1)) - timedelta(weeks=BadLong(1)) + for inttype in (int, long): + class BadInt(inttype): + def __mul__(self, other): + return Prod() + def __rmul__(self, other): + return Prod() + def __floordiv__(self, other): + return Prod() + def __rfloordiv__(self, other): + return Prod() + + class BadLong(long): + def __mul__(self, other): + return Prod() + def __rmul__(self, other): + return Prod() + def __floordiv__(self, other): + return Prod() + def __rfloordiv__(self, other): + return Prod() + + class Prod: + def __add__(self, other): + return Sum() + def __radd__(self, other): + return Sum() + + for inttype2 in (int, long): + class Sum(inttype2): + def __divmod__(self, other): + return divmodresult + + for divmodresult in [None, (), (0, 1, 2), (0, -1)]: + # The following examples should not crash. + try: + timedelta(microseconds=BadInt(1)) + except TypeError: + pass + try: + timedelta(hours=BadInt(1)) + except TypeError: + pass + try: + timedelta(weeks=BadInt(1)) + except (TypeError, ValueError): + pass + try: + timedelta(1) * BadInt(1) + except (TypeError, ValueError): + pass + try: + BadInt(1) * timedelta(1) + except TypeError: + pass + try: + timedelta(1) // BadInt(1) + except TypeError: + pass ############################################################################# diff --git a/Modules/datetimemodule.c b/Modules/datetimemodule.c index c0b7102de234..29492de5167c 100644 --- a/Modules/datetimemodule.c +++ b/Modules/datetimemodule.c @@ -1546,6 +1546,29 @@ delta_to_microseconds(PyDateTime_Delta *self) return result; } +static PyObject * +checked_divmod(PyObject *a, PyObject *b) +{ + PyObject *result = PyNumber_Divmod(a, b); + if (result != NULL) { + if (!PyTuple_Check(result)) { + PyErr_Format(PyExc_TypeError, + "divmod() returned non-tuple (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } + if (PyTuple_GET_SIZE(result) != 2) { + PyErr_Format(PyExc_TypeError, + "divmod() returned a tuple of size %zd", + PyTuple_GET_SIZE(result)); + Py_DECREF(result); + return NULL; + } + } + return result; +} + /* Convert a number of us (as a Python int or long) to a timedelta. */ static PyObject * @@ -1554,70 +1577,49 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) int us; int s; int d; - long temp; PyObject *tuple = NULL; PyObject *num = NULL; PyObject *result = NULL; - assert(_PyAnyInt_CheckExact(pyus)); - tuple = PyNumber_Divmod(pyus, us_per_second); - if (tuple == NULL) + tuple = checked_divmod(pyus, us_per_second); + if (tuple == NULL) { goto Done; + } - num = PyTuple_GetItem(tuple, 1); /* us */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* us */ + us = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 1000000); - us = (int)temp; - if (us < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (us == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= us && us < 1000000)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover seconds */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover seconds */ Py_INCREF(num); Py_DECREF(tuple); - tuple = PyNumber_Divmod(num, seconds_per_day); + tuple = checked_divmod(num, seconds_per_day); if (tuple == NULL) goto Done; Py_DECREF(num); - num = PyTuple_GetItem(tuple, 1); /* seconds */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* seconds */ + s = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 24*3600); - s = (int)temp; - - if (s < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (s == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= s && s < 24*3600)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover days */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover days */ Py_INCREF(num); - temp = PyLong_AsLong(num); - if (temp == -1 && PyErr_Occurred()) - goto Done; - d = (int)temp; - if ((long)d != temp) { - PyErr_SetString(PyExc_OverflowError, "normalized days too " - "large to fit in a C int"); + d = _PyLong_AsInt(num); + if (d == -1 && PyErr_Occurred()) { goto Done; } result = new_delta_ex(d, s, us, 0, type); @@ -1626,6 +1628,11 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) Py_XDECREF(tuple); Py_XDECREF(num); return result; + +BadDivmod: + PyErr_SetString(PyExc_TypeError, + "divmod() returned a value out of range"); + goto Done; } #define microseconds_to_delta(pymicros) \ @@ -1642,7 +1649,7 @@ multiply_int_timedelta(PyObject *intobj, PyDateTime_Delta *delta) if (pyus_in == NULL) return NULL; - pyus_out = PyNumber_Multiply(pyus_in, intobj); + pyus_out = PyNumber_Multiply(intobj, pyus_in); Py_DECREF(pyus_in); if (pyus_out == NULL) return NULL; @@ -1853,13 +1860,11 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, assert(num != NULL); if (_PyAnyInt_Check(num)) { - prod = PyNumber_Multiply(factor, num); + prod = PyNumber_Multiply(num, factor); if (prod == NULL) return NULL; - assert(_PyAnyInt_CheckExact(prod)); sum = PyNumber_Add(sofar, prod); Py_DECREF(prod); - assert(sum == NULL || _PyAnyInt_CheckExact(sum)); return sum; } @@ -1920,7 +1925,6 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, Py_DECREF(sum); Py_DECREF(x); *leftover += fracpart; - assert(y == NULL || _PyAnyInt_CheckExact(y)); return y; }

1 0

bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039)
by Miss Islington (bot) 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/7a0d964afb41bde846771c81ba74623833… commit: 7a0d964afb41bde846771c81ba746238339cdd8c branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T11:02:49-08:00 summary: bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039) Fixes assertion failures in _datetimemodule.c introduced in the previous fix (see bpo-31752). Rather of trying to handle an int subclass as exact int, let it to use overridden special methods, but check the result of divmod(). (cherry picked from commit 3ec0f495163da3b7a15deb2805cec48aed432f58) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: M Lib/test/datetimetester.py M Modules/_datetimemodule.c diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 40a457c9c538..54035ab57f8d 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -872,19 +872,50 @@ def test_issue31752(self): class BadInt(int): def __mul__(self, other): return Prod() + def __rmul__(self, other): + return Prod() + def __floordiv__(self, other): + return Prod() + def __rfloordiv__(self, other): + return Prod() class Prod: + def __add__(self, other): + return Sum() def __radd__(self, other): return Sum() class Sum(int): def __divmod__(self, other): - # negative remainder - return (0, -1) - - timedelta(microseconds=BadInt(1)) - timedelta(hours=BadInt(1)) - timedelta(weeks=BadInt(1)) + return divmodresult + + for divmodresult in [None, (), (0, 1, 2), (0, -1)]: + with self.subTest(divmodresult=divmodresult): + # The following examples should not crash. + try: + timedelta(microseconds=BadInt(1)) + except TypeError: + pass + try: + timedelta(hours=BadInt(1)) + except TypeError: + pass + try: + timedelta(weeks=BadInt(1)) + except (TypeError, ValueError): + pass + try: + timedelta(1) * BadInt(1) + except (TypeError, ValueError): + pass + try: + BadInt(1) * timedelta(1) + except TypeError: + pass + try: + timedelta(1) // BadInt(1) + except TypeError: + pass ############################################################################# diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index b2e0176b3232..2edaf1b319c4 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1542,6 +1542,29 @@ delta_to_microseconds(PyDateTime_Delta *self) return result; } +static PyObject * +checked_divmod(PyObject *a, PyObject *b) +{ + PyObject *result = PyNumber_Divmod(a, b); + if (result != NULL) { + if (!PyTuple_Check(result)) { + PyErr_Format(PyExc_TypeError, + "divmod() returned non-tuple (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } + if (PyTuple_GET_SIZE(result) != 2) { + PyErr_Format(PyExc_TypeError, + "divmod() returned a tuple of size %zd", + PyTuple_GET_SIZE(result)); + Py_DECREF(result); + return NULL; + } + } + return result; +} + /* Convert a number of us (as a Python int) to a timedelta. */ static PyObject * @@ -1550,70 +1573,49 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) int us; int s; int d; - long temp; PyObject *tuple = NULL; PyObject *num = NULL; PyObject *result = NULL; - assert(PyLong_CheckExact(pyus)); - tuple = PyNumber_Divmod(pyus, us_per_second); - if (tuple == NULL) + tuple = checked_divmod(pyus, us_per_second); + if (tuple == NULL) { goto Done; + } - num = PyTuple_GetItem(tuple, 1); /* us */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* us */ + us = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 1000000); - us = (int)temp; - if (us < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (us == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= us && us < 1000000)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover seconds */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover seconds */ Py_INCREF(num); Py_DECREF(tuple); - tuple = PyNumber_Divmod(num, seconds_per_day); + tuple = checked_divmod(num, seconds_per_day); if (tuple == NULL) goto Done; Py_DECREF(num); - num = PyTuple_GetItem(tuple, 1); /* seconds */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* seconds */ + s = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 24*3600); - s = (int)temp; - - if (s < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (s == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= s && s < 24*3600)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover days */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover days */ Py_INCREF(num); - temp = PyLong_AsLong(num); - if (temp == -1 && PyErr_Occurred()) - goto Done; - d = (int)temp; - if ((long)d != temp) { - PyErr_SetString(PyExc_OverflowError, "normalized days too " - "large to fit in a C int"); + d = _PyLong_AsInt(num); + if (d == -1 && PyErr_Occurred()) { goto Done; } result = new_delta_ex(d, s, us, 0, type); @@ -1622,6 +1624,11 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) Py_XDECREF(tuple); Py_XDECREF(num); return result; + +BadDivmod: + PyErr_SetString(PyExc_TypeError, + "divmod() returned a value out of range"); + goto Done; } #define microseconds_to_delta(pymicros) \ @@ -1638,7 +1645,7 @@ multiply_int_timedelta(PyObject *intobj, PyDateTime_Delta *delta) if (pyus_in == NULL) return NULL; - pyus_out = PyNumber_Multiply(pyus_in, intobj); + pyus_out = PyNumber_Multiply(intobj, pyus_in); Py_DECREF(pyus_in); if (pyus_out == NULL) return NULL; @@ -2073,13 +2080,12 @@ delta_divmod(PyObject *left, PyObject *right) return NULL; } - divmod = PyNumber_Divmod(pyus_left, pyus_right); + divmod = checked_divmod(pyus_left, pyus_right); Py_DECREF(pyus_left); Py_DECREF(pyus_right); if (divmod == NULL) return NULL; - assert(PyTuple_Size(divmod) == 2); delta = microseconds_to_delta(PyTuple_GET_ITEM(divmod, 1)); if (delta == NULL) { Py_DECREF(divmod); @@ -2110,13 +2116,11 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, assert(num != NULL); if (PyLong_Check(num)) { - prod = PyNumber_Multiply(factor, num); + prod = PyNumber_Multiply(num, factor); if (prod == NULL) return NULL; - assert(PyLong_CheckExact(prod)); sum = PyNumber_Add(sofar, prod); Py_DECREF(prod); - assert(sum == NULL || PyLong_CheckExact(sum)); return sum; } @@ -2174,7 +2178,6 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, Py_DECREF(sum); Py_DECREF(x); *leftover += fracpart; - assert(y == NULL || PyLong_CheckExact(y)); return y; }

1 0

bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039)
by Miss Islington (bot) 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/d57ab8ac182d15558118523ad1b11b029e… commit: d57ab8ac182d15558118523ad1b11b029e105c46 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T10:59:12-08:00 summary: bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039) Fixes assertion failures in _datetimemodule.c introduced in the previous fix (see bpo-31752). Rather of trying to handle an int subclass as exact int, let it to use overridden special methods, but check the result of divmod(). (cherry picked from commit 3ec0f495163da3b7a15deb2805cec48aed432f58) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: M Lib/test/datetimetester.py M Modules/_datetimemodule.c diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 06fe647fb1a4..78b123f5b118 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -896,19 +896,50 @@ def test_issue31752(self): class BadInt(int): def __mul__(self, other): return Prod() + def __rmul__(self, other): + return Prod() + def __floordiv__(self, other): + return Prod() + def __rfloordiv__(self, other): + return Prod() class Prod: + def __add__(self, other): + return Sum() def __radd__(self, other): return Sum() class Sum(int): def __divmod__(self, other): - # negative remainder - return (0, -1) - - timedelta(microseconds=BadInt(1)) - timedelta(hours=BadInt(1)) - timedelta(weeks=BadInt(1)) + return divmodresult + + for divmodresult in [None, (), (0, 1, 2), (0, -1)]: + with self.subTest(divmodresult=divmodresult): + # The following examples should not crash. + try: + timedelta(microseconds=BadInt(1)) + except TypeError: + pass + try: + timedelta(hours=BadInt(1)) + except TypeError: + pass + try: + timedelta(weeks=BadInt(1)) + except (TypeError, ValueError): + pass + try: + timedelta(1) * BadInt(1) + except (TypeError, ValueError): + pass + try: + BadInt(1) * timedelta(1) + except TypeError: + pass + try: + timedelta(1) // BadInt(1) + except TypeError: + pass ############################################################################# diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 1f15367d3fa3..9477f776138c 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1783,6 +1783,29 @@ delta_to_microseconds(PyDateTime_Delta *self) return result; } +static PyObject * +checked_divmod(PyObject *a, PyObject *b) +{ + PyObject *result = PyNumber_Divmod(a, b); + if (result != NULL) { + if (!PyTuple_Check(result)) { + PyErr_Format(PyExc_TypeError, + "divmod() returned non-tuple (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } + if (PyTuple_GET_SIZE(result) != 2) { + PyErr_Format(PyExc_TypeError, + "divmod() returned a tuple of size %zd", + PyTuple_GET_SIZE(result)); + Py_DECREF(result); + return NULL; + } + } + return result; +} + /* Convert a number of us (as a Python int) to a timedelta. */ static PyObject * @@ -1791,70 +1814,49 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) int us; int s; int d; - long temp; PyObject *tuple = NULL; PyObject *num = NULL; PyObject *result = NULL; - assert(PyLong_CheckExact(pyus)); - tuple = PyNumber_Divmod(pyus, us_per_second); - if (tuple == NULL) + tuple = checked_divmod(pyus, us_per_second); + if (tuple == NULL) { goto Done; + } - num = PyTuple_GetItem(tuple, 1); /* us */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* us */ + us = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 1000000); - us = (int)temp; - if (us < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (us == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= us && us < 1000000)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover seconds */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover seconds */ Py_INCREF(num); Py_DECREF(tuple); - tuple = PyNumber_Divmod(num, seconds_per_day); + tuple = checked_divmod(num, seconds_per_day); if (tuple == NULL) goto Done; Py_DECREF(num); - num = PyTuple_GetItem(tuple, 1); /* seconds */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* seconds */ + s = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 24*3600); - s = (int)temp; - - if (s < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (s == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= s && s < 24*3600)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover days */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover days */ Py_INCREF(num); - temp = PyLong_AsLong(num); - if (temp == -1 && PyErr_Occurred()) - goto Done; - d = (int)temp; - if ((long)d != temp) { - PyErr_SetString(PyExc_OverflowError, "normalized days too " - "large to fit in a C int"); + d = _PyLong_AsInt(num); + if (d == -1 && PyErr_Occurred()) { goto Done; } result = new_delta_ex(d, s, us, 0, type); @@ -1863,6 +1865,11 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) Py_XDECREF(tuple); Py_XDECREF(num); return result; + +BadDivmod: + PyErr_SetString(PyExc_TypeError, + "divmod() returned a value out of range"); + goto Done; } #define microseconds_to_delta(pymicros) \ @@ -1879,7 +1886,7 @@ multiply_int_timedelta(PyObject *intobj, PyDateTime_Delta *delta) if (pyus_in == NULL) return NULL; - pyus_out = PyNumber_Multiply(pyus_in, intobj); + pyus_out = PyNumber_Multiply(intobj, pyus_in); Py_DECREF(pyus_in); if (pyus_out == NULL) return NULL; @@ -2283,13 +2290,12 @@ delta_divmod(PyObject *left, PyObject *right) return NULL; } - divmod = PyNumber_Divmod(pyus_left, pyus_right); + divmod = checked_divmod(pyus_left, pyus_right); Py_DECREF(pyus_left); Py_DECREF(pyus_right); if (divmod == NULL) return NULL; - assert(PyTuple_Size(divmod) == 2); delta = microseconds_to_delta(PyTuple_GET_ITEM(divmod, 1)); if (delta == NULL) { Py_DECREF(divmod); @@ -2320,13 +2326,11 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, assert(num != NULL); if (PyLong_Check(num)) { - prod = PyNumber_Multiply(factor, num); + prod = PyNumber_Multiply(num, factor); if (prod == NULL) return NULL; - assert(PyLong_CheckExact(prod)); sum = PyNumber_Add(sofar, prod); Py_DECREF(prod); - assert(sum == NULL || PyLong_CheckExact(sum)); return sum; } @@ -2384,7 +2388,6 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, Py_DECREF(sum); Py_DECREF(x); *leftover += fracpart; - assert(y == NULL || PyLong_CheckExact(y)); return y; }

1 0

bpo-25750: Fix a compiler warning introduced in GH-9084. (GH-10234)
by Serhiy Storchaka 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/b1dede3ee3498100b95265f7fdb0ea2bef… commit: b1dede3ee3498100b95265f7fdb0ea2bef9a2ba2 branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T20:45:40+02:00 summary: bpo-25750: Fix a compiler warning introduced in GH-9084. (GH-10234) files: M Modules/_testcapimodule.c diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 56a08a418b36..0dc4d7a36b76 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -4586,18 +4586,18 @@ new_hamt(PyObject *self, PyObject *args) static PyObject* bad_get(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { - if (nargs != 3) { - PyErr_SetString(PyExc_TypeError, "bad_get requires exactly 3 arguments"); + PyObject *self, *obj, *cls; + if (!_PyArg_UnpackStack(args, nargs, "bad_get", 3, 3, &self, &obj, &cls)) { return NULL; } - PyObject *res = PyObject_CallObject(args[2], NULL); + PyObject *res = PyObject_CallObject(cls, NULL); if (res == NULL) { return NULL; } Py_DECREF(res); - return PyObject_Repr(args[0]); + return PyObject_Repr(self); } @@ -4960,7 +4960,7 @@ static PyMethodDef TestMethods[] = { {"get_mapping_items", get_mapping_items, METH_O}, {"test_pythread_tss_key_state", test_pythread_tss_key_state, METH_VARARGS}, {"hamt", new_hamt, METH_NOARGS}, - {"bad_get", bad_get, METH_FASTCALL}, + {"bad_get", (PyCFunction)bad_get, METH_FASTCALL}, {"EncodeLocaleEx", encode_locale_ex, METH_VARARGS}, {"DecodeLocaleEx", decode_locale_ex, METH_VARARGS}, {"get_global_config", get_global_config, METH_NOARGS},

1 0

bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039)
by Serhiy Storchaka 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/3ec0f495163da3b7a15deb2805cec48aed… commit: 3ec0f495163da3b7a15deb2805cec48aed432f58 branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T20:41:09+02:00 summary: bpo-35021: Fix assertion failures in _datetimemodule.c. (GH-10039) Fixes assertion failures in _datetimemodule.c introduced in the previous fix (see bpo-31752). Rather of trying to handle an int subclass as exact int, let it to use overridden special methods, but check the result of divmod(). files: M Lib/test/datetimetester.py M Modules/_datetimemodule.c diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 06fe647fb1a4..78b123f5b118 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -896,19 +896,50 @@ def test_issue31752(self): class BadInt(int): def __mul__(self, other): return Prod() + def __rmul__(self, other): + return Prod() + def __floordiv__(self, other): + return Prod() + def __rfloordiv__(self, other): + return Prod() class Prod: + def __add__(self, other): + return Sum() def __radd__(self, other): return Sum() class Sum(int): def __divmod__(self, other): - # negative remainder - return (0, -1) - - timedelta(microseconds=BadInt(1)) - timedelta(hours=BadInt(1)) - timedelta(weeks=BadInt(1)) + return divmodresult + + for divmodresult in [None, (), (0, 1, 2), (0, -1)]: + with self.subTest(divmodresult=divmodresult): + # The following examples should not crash. + try: + timedelta(microseconds=BadInt(1)) + except TypeError: + pass + try: + timedelta(hours=BadInt(1)) + except TypeError: + pass + try: + timedelta(weeks=BadInt(1)) + except (TypeError, ValueError): + pass + try: + timedelta(1) * BadInt(1) + except (TypeError, ValueError): + pass + try: + BadInt(1) * timedelta(1) + except TypeError: + pass + try: + timedelta(1) // BadInt(1) + except TypeError: + pass ############################################################################# diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 02c510f055bd..371bfaec3a05 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1797,6 +1797,29 @@ delta_to_microseconds(PyDateTime_Delta *self) return result; } +static PyObject * +checked_divmod(PyObject *a, PyObject *b) +{ + PyObject *result = PyNumber_Divmod(a, b); + if (result != NULL) { + if (!PyTuple_Check(result)) { + PyErr_Format(PyExc_TypeError, + "divmod() returned non-tuple (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } + if (PyTuple_GET_SIZE(result) != 2) { + PyErr_Format(PyExc_TypeError, + "divmod() returned a tuple of size %zd", + PyTuple_GET_SIZE(result)); + Py_DECREF(result); + return NULL; + } + } + return result; +} + /* Convert a number of us (as a Python int) to a timedelta. */ static PyObject * @@ -1805,70 +1828,49 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) int us; int s; int d; - long temp; PyObject *tuple = NULL; PyObject *num = NULL; PyObject *result = NULL; - assert(PyLong_CheckExact(pyus)); - tuple = PyNumber_Divmod(pyus, us_per_second); - if (tuple == NULL) + tuple = checked_divmod(pyus, us_per_second); + if (tuple == NULL) { goto Done; + } - num = PyTuple_GetItem(tuple, 1); /* us */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* us */ + us = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 1000000); - us = (int)temp; - if (us < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (us == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= us && us < 1000000)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover seconds */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover seconds */ Py_INCREF(num); Py_DECREF(tuple); - tuple = PyNumber_Divmod(num, seconds_per_day); + tuple = checked_divmod(num, seconds_per_day); if (tuple == NULL) goto Done; Py_DECREF(num); - num = PyTuple_GetItem(tuple, 1); /* seconds */ - if (num == NULL) - goto Done; - temp = PyLong_AsLong(num); + num = PyTuple_GET_ITEM(tuple, 1); /* seconds */ + s = _PyLong_AsInt(num); num = NULL; - if (temp == -1 && PyErr_Occurred()) - goto Done; - assert(0 <= temp && temp < 24*3600); - s = (int)temp; - - if (s < 0) { - /* The divisor was positive, so this must be an error. */ - assert(PyErr_Occurred()); + if (s == -1 && PyErr_Occurred()) { goto Done; } + if (!(0 <= s && s < 24*3600)) { + goto BadDivmod; + } - num = PyTuple_GetItem(tuple, 0); /* leftover days */ - if (num == NULL) - goto Done; + num = PyTuple_GET_ITEM(tuple, 0); /* leftover days */ Py_INCREF(num); - temp = PyLong_AsLong(num); - if (temp == -1 && PyErr_Occurred()) - goto Done; - d = (int)temp; - if ((long)d != temp) { - PyErr_SetString(PyExc_OverflowError, "normalized days too " - "large to fit in a C int"); + d = _PyLong_AsInt(num); + if (d == -1 && PyErr_Occurred()) { goto Done; } result = new_delta_ex(d, s, us, 0, type); @@ -1877,6 +1879,11 @@ microseconds_to_delta_ex(PyObject *pyus, PyTypeObject *type) Py_XDECREF(tuple); Py_XDECREF(num); return result; + +BadDivmod: + PyErr_SetString(PyExc_TypeError, + "divmod() returned a value out of range"); + goto Done; } #define microseconds_to_delta(pymicros) \ @@ -1893,7 +1900,7 @@ multiply_int_timedelta(PyObject *intobj, PyDateTime_Delta *delta) if (pyus_in == NULL) return NULL; - pyus_out = PyNumber_Multiply(pyus_in, intobj); + pyus_out = PyNumber_Multiply(intobj, pyus_in); Py_DECREF(pyus_in); if (pyus_out == NULL) return NULL; @@ -2297,13 +2304,12 @@ delta_divmod(PyObject *left, PyObject *right) return NULL; } - divmod = PyNumber_Divmod(pyus_left, pyus_right); + divmod = checked_divmod(pyus_left, pyus_right); Py_DECREF(pyus_left); Py_DECREF(pyus_right); if (divmod == NULL) return NULL; - assert(PyTuple_Size(divmod) == 2); delta = microseconds_to_delta(PyTuple_GET_ITEM(divmod, 1)); if (delta == NULL) { Py_DECREF(divmod); @@ -2334,13 +2340,11 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, assert(num != NULL); if (PyLong_Check(num)) { - prod = PyNumber_Multiply(factor, num); + prod = PyNumber_Multiply(num, factor); if (prod == NULL) return NULL; - assert(PyLong_CheckExact(prod)); sum = PyNumber_Add(sofar, prod); Py_DECREF(prod); - assert(sum == NULL || PyLong_CheckExact(sum)); return sum; } @@ -2398,7 +2402,6 @@ accum(const char* tag, PyObject *sofar, PyObject *num, PyObject *factor, Py_DECREF(sum); Py_DECREF(x); *leftover += fracpart; - assert(y == NULL || PyLong_CheckExact(y)); return y; }

1 0

bpo-9842: Add references for using "..." as a placeholder to the index. (GH-10330)
by Miss Islington (bot) 20 Nov '18

20 Nov '18

https://github.com/python/cpython/commit/dac5124ba498b51a2c46e2bda751150ae2… commit: dac5124ba498b51a2c46e2bda751150ae244ed47 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2018-11-20T10:08:07-08:00 summary: bpo-9842: Add references for using "..." as a placeholder to the index. (GH-10330) (cherry picked from commit 6c48bf2d9e1e18dfbfa35f7582ddd32f11f75129) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: M Doc/library/pprint.rst M Doc/library/reprlib.rst M Doc/library/textwrap.rst diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst index 3922a768311c..deadf1820851 100644 --- a/Doc/library/pprint.rst +++ b/Doc/library/pprint.rst @@ -30,6 +30,8 @@ The :mod:`pprint` module defines one class: .. First the implementation class: +.. index:: single: ...; placeholder + .. class:: PrettyPrinter(indent=1, width=80, depth=None, stream=None, *, \ compact=False) diff --git a/Doc/library/reprlib.rst b/Doc/library/reprlib.rst index 0905b982cd83..dbe2a7182aec 100644 --- a/Doc/library/reprlib.rst +++ b/Doc/library/reprlib.rst @@ -42,6 +42,9 @@ In addition to size-limiting tools, the module also provides a decorator for detecting recursive calls to :meth:`__repr__` and substituting a placeholder string instead. + +.. index:: single: ...; placeholder + .. decorator:: recursive_repr(fillvalue="...") Decorator for :meth:`__repr__` methods to detect recursive calls within the diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst index 438007d0028d..d254466c9a32 100644 --- a/Doc/library/textwrap.rst +++ b/Doc/library/textwrap.rst @@ -268,6 +268,8 @@ hyphenated words; only then will long words be broken if necessary, unless .. versionadded:: 3.4 + .. index:: single: ...; placeholder + .. attribute:: placeholder (default: ``' [...]'``) String that will appear at the end of the output

1 0