Mailman 3 March 2019 - Python-checkins

Fixed capital letters missing and missing . (GH-12584)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/3d78c4a6e5ae91eaf337b6f5cc6e8bb01a… commit: 3d78c4a6e5ae91eaf337b6f5cc6e8bb01af7c7b1 branch: master author: Jules Lasne (jlasne) <jlasne(a)student.42.fr> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-03-28T14:06:27-07:00 summary: Fixed capital letters missing and missing . (GH-12584) No `bpo` for minor doc fix files: M Doc/library/unittest.rst diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index d1b4f3ba0bbb..8ad2abd3d89a 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1000,7 +1000,7 @@ Test cases int('XYZ') .. versionadded:: 3.1 - under the name ``assertRaisesRegexp``. + Added under the name ``assertRaisesRegexp``. .. versionchanged:: 3.2 Renamed to :meth:`assertRaisesRegex`. @@ -1145,7 +1145,7 @@ Test cases +---------------------------------------+--------------------------------+--------------+ | :meth:`assertCountEqual(a, b) | *a* and *b* have the same | 3.2 | | <TestCase.assertCountEqual>` | elements in the same number, | | - | | regardless of their order | | + | | regardless of their order. | | +---------------------------------------+--------------------------------+--------------+ @@ -1193,7 +1193,7 @@ Test cases expression suitable for use by :func:`re.search`. .. versionadded:: 3.1 - under the name ``assertRegexpMatches``. + Added under the name ``assertRegexpMatches``. .. versionchanged:: 3.2 The method ``assertRegexpMatches()`` has been renamed to :meth:`.assertRegex`. @@ -1516,14 +1516,14 @@ along with their deprecated aliases: ============================== ====================== ======================= .. deprecated:: 3.1 - the fail* aliases listed in the second column. + The fail* aliases listed in the second column have been deprecated. .. deprecated:: 3.2 - the assert* aliases listed in the third column. + The assert* aliases listed in the third column have been deprecated. .. deprecated:: 3.2 ``assertRegexpMatches`` and ``assertRaisesRegexp`` have been renamed to :meth:`.assertRegex` and :meth:`.assertRaisesRegex`. .. deprecated:: 3.5 - the ``assertNotRegexpMatches`` name in favor of :meth:`.assertNotRegex`. + The ``assertNotRegexpMatches`` name is deprecated in favor of :meth:`.assertNotRegex`. .. _testsuite-objects:

1 0

Fix typo in email.encoders doc (GH-9700)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/e63fc11b1d64d9586003ce4aebf4eb2fdc… commit: e63fc11b1d64d9586003ce4aebf4eb2fdcaf8125 branch: master author: ksamuel <ksamuel(a)users.noreply.github.com> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-03-28T13:38:30-07:00 summary: Fix typo in email.encoders doc (GH-9700) Make the encoding/encoders mention congruent. files: M Doc/library/email.encoders.rst diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index e24ac7b90d29..70bf61323c39 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -19,7 +19,7 @@ need to encode the payloads for transport through compliant mail servers. This is especially true for :mimetype:`image/\*` and :mimetype:`text/\*` type messages containing binary data. -The :mod:`email` package provides some convenient encodings in its +The :mod:`email` package provides some convenient encoders in its :mod:`encoders` module. These encoders are actually used by the :class:`~email.mime.audio.MIMEAudio` and :class:`~email.mime.image.MIMEImage` class constructors to provide default encodings. All encoder functions take

1 0

bpo-35941: Fix ssl certificate enumeration for windows (GH-12486)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/e9868c5416731f5ca5378a1d36e4b020c3… commit: e9868c5416731f5ca5378a1d36e4b020c349291c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T11:56:50-07:00 summary: bpo-35941: Fix ssl certificate enumeration for windows (GH-12486) Add a function to collect certificates from several certificate stores into one certificate collection store that is then enumerated. This ensures we load as many certificates as we can access. (cherry picked from commit d93fbbf88e4abdd24a0a55e3ddf85b8420c62052) Co-authored-by: kctherookie <48805853+kctherookie(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst b/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst new file mode 100644 index 000000000000..cda654bfa5b9 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst @@ -0,0 +1,3 @@ +enum_certificates function of the ssl module now returns certificates from all available certificate stores inside windows in a query instead of returning only certificates from the system wide certificate store. +This includes certificates from these certificate stores: local machine, local machine enterprise, local machine group policy, current user, current user group policy, services, users. +ssl.enum_crls() function is changed in the same way to return all certificate revocation lists inside the windows certificate revocation list stores. \ No newline at end of file diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 7076fddf8dd4..fff1a28843d0 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -5401,6 +5401,68 @@ parseKeyUsage(PCCERT_CONTEXT pCertCtx, DWORD flags) return retval; } +static HCERTSTORE +ssl_collect_certificates(const char *store_name) +{ +/* this function collects the system certificate stores listed in + * system_stores into a collection certificate store for being + * enumerated. The store must be readable to be added to the + * store collection. + */ + + HCERTSTORE hCollectionStore = NULL, hSystemStore = NULL; + static DWORD system_stores[] = { + CERT_SYSTEM_STORE_LOCAL_MACHINE, + CERT_SYSTEM_STORE_LOCAL_MACHINE_ENTERPRISE, + CERT_SYSTEM_STORE_LOCAL_MACHINE_GROUP_POLICY, + CERT_SYSTEM_STORE_CURRENT_USER, + CERT_SYSTEM_STORE_CURRENT_USER_GROUP_POLICY, + CERT_SYSTEM_STORE_SERVICES, + CERT_SYSTEM_STORE_USERS}; + size_t i, storesAdded; + BOOL result; + + hCollectionStore = CertOpenStore(CERT_STORE_PROV_COLLECTION, 0, + (HCRYPTPROV)NULL, 0, NULL); + if (!hCollectionStore) { + return NULL; + } + storesAdded = 0; + for (i = 0; i < sizeof(system_stores) / sizeof(DWORD); i++) { + hSystemStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, + (HCRYPTPROV)NULL, + CERT_STORE_READONLY_FLAG | + system_stores[i], store_name); + if (hSystemStore) { + result = CertAddStoreToCollection(hCollectionStore, hSystemStore, + CERT_PHYSICAL_STORE_ADD_ENABLE_FLAG, 0); + if (result) { + ++storesAdded; + } + } + } + if (storesAdded == 0) { + CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG); + return NULL; + } + + return hCollectionStore; +} + +/* code from Objects/listobject.c */ + +static int +list_contains(PyListObject *a, PyObject *el) +{ + Py_ssize_t i; + int cmp; + + for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) + cmp = PyObject_RichCompareBool(el, PyList_GET_ITEM(a, i), + Py_EQ); + return cmp; +} + /*[clinic input] _ssl.enum_certificates store_name: str @@ -5418,7 +5480,7 @@ static PyObject * _ssl_enum_certificates_impl(PyObject *module, const char *store_name) /*[clinic end generated code: output=5134dc8bb3a3c893 input=915f60d70461ea4e]*/ { - HCERTSTORE hStore = NULL; + HCERTSTORE hCollectionStore = NULL; PCCERT_CONTEXT pCertCtx = NULL; PyObject *keyusage = NULL, *cert = NULL, *enc = NULL, *tup = NULL; PyObject *result = NULL; @@ -5427,15 +5489,13 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) if (result == NULL) { return NULL; } - hStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, (HCRYPTPROV)NULL, - CERT_STORE_READONLY_FLAG | CERT_SYSTEM_STORE_LOCAL_MACHINE, - store_name); - if (hStore == NULL) { + hCollectionStore = ssl_collect_certificates(store_name); + if (hCollectionStore == NULL) { Py_DECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } - while (pCertCtx = CertEnumCertificatesInStore(hStore, pCertCtx)) { + while (pCertCtx = CertEnumCertificatesInStore(hCollectionStore, pCertCtx)) { cert = PyBytes_FromStringAndSize((const char*)pCertCtx->pbCertEncoded, pCertCtx->cbCertEncoded); if (!cert) { @@ -5465,9 +5525,11 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) enc = NULL; PyTuple_SET_ITEM(tup, 2, keyusage); keyusage = NULL; - if (PyList_Append(result, tup) < 0) { - Py_CLEAR(result); - break; + if (!list_contains((PyListObject*)result, tup)) { + if (PyList_Append(result, tup) < 0) { + Py_CLEAR(result); + break; + } } Py_CLEAR(tup); } @@ -5482,11 +5544,15 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) Py_XDECREF(keyusage); Py_XDECREF(tup); - if (!CertCloseStore(hStore, 0)) { + /* CERT_CLOSE_STORE_FORCE_FLAG forces freeing of memory for all contexts + associated with the store, in this case our collection store and the + associated system stores. */ + if (!CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG)) { /* This error case might shadow another exception.*/ Py_XDECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } + return result; } @@ -5506,7 +5572,7 @@ static PyObject * _ssl_enum_crls_impl(PyObject *module, const char *store_name) /*[clinic end generated code: output=bce467f60ccd03b6 input=a1f1d7629f1c5d3d]*/ { - HCERTSTORE hStore = NULL; + HCERTSTORE hCollectionStore = NULL; PCCRL_CONTEXT pCrlCtx = NULL; PyObject *crl = NULL, *enc = NULL, *tup = NULL; PyObject *result = NULL; @@ -5515,15 +5581,13 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) if (result == NULL) { return NULL; } - hStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, (HCRYPTPROV)NULL, - CERT_STORE_READONLY_FLAG | CERT_SYSTEM_STORE_LOCAL_MACHINE, - store_name); - if (hStore == NULL) { + hCollectionStore = ssl_collect_certificates(store_name); + if (hCollectionStore == NULL) { Py_DECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } - while (pCrlCtx = CertEnumCRLsInStore(hStore, pCrlCtx)) { + while (pCrlCtx = CertEnumCRLsInStore(hCollectionStore, pCrlCtx)) { crl = PyBytes_FromStringAndSize((const char*)pCrlCtx->pbCrlEncoded, pCrlCtx->cbCrlEncoded); if (!crl) { @@ -5543,9 +5607,11 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) PyTuple_SET_ITEM(tup, 1, enc); enc = NULL; - if (PyList_Append(result, tup) < 0) { - Py_CLEAR(result); - break; + if (!list_contains((PyListObject*)result, tup)) { + if (PyList_Append(result, tup) < 0) { + Py_CLEAR(result); + break; + } } Py_CLEAR(tup); } @@ -5559,7 +5625,10 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) Py_XDECREF(enc); Py_XDECREF(tup); - if (!CertCloseStore(hStore, 0)) { + /* CERT_CLOSE_STORE_FORCE_FLAG forces freeing of memory for all contexts + associated with the store, in this case our collection store and the + associated system stores. */ + if (!CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG)) { /* This error case might shadow another exception.*/ Py_XDECREF(result); return PyErr_SetFromWindowsErr(GetLastError());

1 0

bpo-36425: Add Simplified Chinese to the language switcher (GH-12537)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/1d9f1a0c9690f4e53003dc5e625a286771… commit: 1d9f1a0c9690f4e53003dc5e625a2867715c828a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T11:12:39-07:00 summary: bpo-36425: Add Simplified Chinese to the language switcher (GH-12537) (cherry picked from commit 45a5fdb91cee665161a8b1980bb4e6ccb999f58f) Co-authored-by: zhsj <zsj950618(a)gmail.com> files: A Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst M Doc/tools/static/switchers.js diff --git a/Doc/tools/static/switchers.js b/Doc/tools/static/switchers.js index 20dad93d6a5e..346b31494e60 100644 --- a/Doc/tools/static/switchers.js +++ b/Doc/tools/static/switchers.js @@ -22,6 +22,7 @@ 'fr': 'French', 'ja': 'Japanese', 'ko': 'Korean', + 'zh-cn': 'Simplified Chinese', }; function build_version_select(current_version, current_release) { diff --git a/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst b/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst new file mode 100644 index 000000000000..12bd833a88b5 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst @@ -0,0 +1,2 @@ +New documentation translation: `Simplified Chinese +<https://docs.python.org/zh-cn/>`_.

1 0

bpo-35941: Fix ssl certificate enumeration for windows (GH-12486)
by Steve Dower 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/d93fbbf88e4abdd24a0a55e3ddf85b8420… commit: d93fbbf88e4abdd24a0a55e3ddf85b8420c62052 branch: master author: kctherookie <48805853+kctherookie(a)users.noreply.github.com> committer: Steve Dower <steve.dower(a)microsoft.com> date: 2019-03-28T10:59:06-07:00 summary: bpo-35941: Fix ssl certificate enumeration for windows (GH-12486) Add a function to collect certificates from several certificate stores into one certificate collection store that is then enumerated. This ensures we load as many certificates as we can access. files: A Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst b/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst new file mode 100644 index 000000000000..cda654bfa5b9 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-03-28-03-51-16.bpo-35941.UnlAEE.rst @@ -0,0 +1,3 @@ +enum_certificates function of the ssl module now returns certificates from all available certificate stores inside windows in a query instead of returning only certificates from the system wide certificate store. +This includes certificates from these certificate stores: local machine, local machine enterprise, local machine group policy, current user, current user group policy, services, users. +ssl.enum_crls() function is changed in the same way to return all certificate revocation lists inside the windows certificate revocation list stores. \ No newline at end of file diff --git a/Modules/_ssl.c b/Modules/_ssl.c index f2ce7fa627ee..f8ae916735f4 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -5400,6 +5400,68 @@ parseKeyUsage(PCCERT_CONTEXT pCertCtx, DWORD flags) return retval; } +static HCERTSTORE +ssl_collect_certificates(const char *store_name) +{ +/* this function collects the system certificate stores listed in + * system_stores into a collection certificate store for being + * enumerated. The store must be readable to be added to the + * store collection. + */ + + HCERTSTORE hCollectionStore = NULL, hSystemStore = NULL; + static DWORD system_stores[] = { + CERT_SYSTEM_STORE_LOCAL_MACHINE, + CERT_SYSTEM_STORE_LOCAL_MACHINE_ENTERPRISE, + CERT_SYSTEM_STORE_LOCAL_MACHINE_GROUP_POLICY, + CERT_SYSTEM_STORE_CURRENT_USER, + CERT_SYSTEM_STORE_CURRENT_USER_GROUP_POLICY, + CERT_SYSTEM_STORE_SERVICES, + CERT_SYSTEM_STORE_USERS}; + size_t i, storesAdded; + BOOL result; + + hCollectionStore = CertOpenStore(CERT_STORE_PROV_COLLECTION, 0, + (HCRYPTPROV)NULL, 0, NULL); + if (!hCollectionStore) { + return NULL; + } + storesAdded = 0; + for (i = 0; i < sizeof(system_stores) / sizeof(DWORD); i++) { + hSystemStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, + (HCRYPTPROV)NULL, + CERT_STORE_READONLY_FLAG | + system_stores[i], store_name); + if (hSystemStore) { + result = CertAddStoreToCollection(hCollectionStore, hSystemStore, + CERT_PHYSICAL_STORE_ADD_ENABLE_FLAG, 0); + if (result) { + ++storesAdded; + } + } + } + if (storesAdded == 0) { + CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG); + return NULL; + } + + return hCollectionStore; +} + +/* code from Objects/listobject.c */ + +static int +list_contains(PyListObject *a, PyObject *el) +{ + Py_ssize_t i; + int cmp; + + for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) + cmp = PyObject_RichCompareBool(el, PyList_GET_ITEM(a, i), + Py_EQ); + return cmp; +} + /*[clinic input] _ssl.enum_certificates store_name: str @@ -5417,7 +5479,7 @@ static PyObject * _ssl_enum_certificates_impl(PyObject *module, const char *store_name) /*[clinic end generated code: output=5134dc8bb3a3c893 input=915f60d70461ea4e]*/ { - HCERTSTORE hStore = NULL; + HCERTSTORE hCollectionStore = NULL; PCCERT_CONTEXT pCertCtx = NULL; PyObject *keyusage = NULL, *cert = NULL, *enc = NULL, *tup = NULL; PyObject *result = NULL; @@ -5426,15 +5488,13 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) if (result == NULL) { return NULL; } - hStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, (HCRYPTPROV)NULL, - CERT_STORE_READONLY_FLAG | CERT_SYSTEM_STORE_LOCAL_MACHINE, - store_name); - if (hStore == NULL) { + hCollectionStore = ssl_collect_certificates(store_name); + if (hCollectionStore == NULL) { Py_DECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } - while (pCertCtx = CertEnumCertificatesInStore(hStore, pCertCtx)) { + while (pCertCtx = CertEnumCertificatesInStore(hCollectionStore, pCertCtx)) { cert = PyBytes_FromStringAndSize((const char*)pCertCtx->pbCertEncoded, pCertCtx->cbCertEncoded); if (!cert) { @@ -5464,9 +5524,11 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) enc = NULL; PyTuple_SET_ITEM(tup, 2, keyusage); keyusage = NULL; - if (PyList_Append(result, tup) < 0) { - Py_CLEAR(result); - break; + if (!list_contains((PyListObject*)result, tup)) { + if (PyList_Append(result, tup) < 0) { + Py_CLEAR(result); + break; + } } Py_CLEAR(tup); } @@ -5481,11 +5543,15 @@ _ssl_enum_certificates_impl(PyObject *module, const char *store_name) Py_XDECREF(keyusage); Py_XDECREF(tup); - if (!CertCloseStore(hStore, 0)) { + /* CERT_CLOSE_STORE_FORCE_FLAG forces freeing of memory for all contexts + associated with the store, in this case our collection store and the + associated system stores. */ + if (!CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG)) { /* This error case might shadow another exception.*/ Py_XDECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } + return result; } @@ -5505,7 +5571,7 @@ static PyObject * _ssl_enum_crls_impl(PyObject *module, const char *store_name) /*[clinic end generated code: output=bce467f60ccd03b6 input=a1f1d7629f1c5d3d]*/ { - HCERTSTORE hStore = NULL; + HCERTSTORE hCollectionStore = NULL; PCCRL_CONTEXT pCrlCtx = NULL; PyObject *crl = NULL, *enc = NULL, *tup = NULL; PyObject *result = NULL; @@ -5514,15 +5580,13 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) if (result == NULL) { return NULL; } - hStore = CertOpenStore(CERT_STORE_PROV_SYSTEM_A, 0, (HCRYPTPROV)NULL, - CERT_STORE_READONLY_FLAG | CERT_SYSTEM_STORE_LOCAL_MACHINE, - store_name); - if (hStore == NULL) { + hCollectionStore = ssl_collect_certificates(store_name); + if (hCollectionStore == NULL) { Py_DECREF(result); return PyErr_SetFromWindowsErr(GetLastError()); } - while (pCrlCtx = CertEnumCRLsInStore(hStore, pCrlCtx)) { + while (pCrlCtx = CertEnumCRLsInStore(hCollectionStore, pCrlCtx)) { crl = PyBytes_FromStringAndSize((const char*)pCrlCtx->pbCrlEncoded, pCrlCtx->cbCrlEncoded); if (!crl) { @@ -5542,9 +5606,11 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) PyTuple_SET_ITEM(tup, 1, enc); enc = NULL; - if (PyList_Append(result, tup) < 0) { - Py_CLEAR(result); - break; + if (!list_contains((PyListObject*)result, tup)) { + if (PyList_Append(result, tup) < 0) { + Py_CLEAR(result); + break; + } } Py_CLEAR(tup); } @@ -5558,7 +5624,10 @@ _ssl_enum_crls_impl(PyObject *module, const char *store_name) Py_XDECREF(enc); Py_XDECREF(tup); - if (!CertCloseStore(hStore, 0)) { + /* CERT_CLOSE_STORE_FORCE_FLAG forces freeing of memory for all contexts + associated with the store, in this case our collection store and the + associated system stores. */ + if (!CertCloseStore(hCollectionStore, CERT_CLOSE_STORE_FORCE_FLAG)) { /* This error case might shadow another exception.*/ Py_XDECREF(result); return PyErr_SetFromWindowsErr(GetLastError());

1 0

bpo-36425: Add Simplified Chinese to the language switcher (GH-12537)
by Julien Palard 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/45a5fdb91cee665161a8b1980bb4e6ccb9… commit: 45a5fdb91cee665161a8b1980bb4e6ccb999f58f branch: master author: zhsj <zsj950618(a)gmail.com> committer: Julien Palard <julien(a)palard.fr> date: 2019-03-28T16:32:25+01:00 summary: bpo-36425: Add Simplified Chinese to the language switcher (GH-12537) files: A Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst M Doc/tools/static/switchers.js diff --git a/Doc/tools/static/switchers.js b/Doc/tools/static/switchers.js index 20dad93d6a5e..346b31494e60 100644 --- a/Doc/tools/static/switchers.js +++ b/Doc/tools/static/switchers.js @@ -22,6 +22,7 @@ 'fr': 'French', 'ja': 'Japanese', 'ko': 'Korean', + 'zh-cn': 'Simplified Chinese', }; function build_version_select(current_version, current_release) { diff --git a/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst b/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst new file mode 100644 index 000000000000..12bd833a88b5 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-03-27-22-46-00.bpo-36425.kG9gx1.rst @@ -0,0 +1,2 @@ +New documentation translation: `Simplified Chinese +<https://docs.python.org/zh-cn/>`_.

1 0

bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's tok_nextc() (12601)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/6fd3c852b15820480ad2ea83e7857615c4… commit: 6fd3c852b15820480ad2ea83e7857615c4976304 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T08:08:35-07:00 summary: bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's tok_nextc() (12601) Remove the PyMem_FREE() call added in cb90c89. The buffer will be freed when PyTokenizer_Free() is called on the tokenizer state. (cherry picked from commit cda139d1ded6708665b53e4ed32ccc1d2627e1da) Co-authored-by: Zackery Spytz <zspytz(a)gmail.com> files: A Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst M Parser/tokenizer.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst b/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst new file mode 100644 index 000000000000..6c234a6a76d9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst @@ -0,0 +1 @@ +Fix a possible double ``PyMem_FREE()`` due to tokenizer.c's ``tok_nextc()``. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 34722f85b94a..e374c5a4aee6 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1055,7 +1055,6 @@ tok_nextc(struct tok_state *tok) newbuf = (char *)PyMem_REALLOC(newbuf, newsize); if (newbuf == NULL) { - PyMem_FREE(tok->buf); tok->done = E_NOMEM; tok->cur = tok->inp; return EOF;

1 0

bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's tok_nextc() (12601)
by Miss Islington (bot) 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/dffe90ee0eaf77785ad3d4ad7fb3249430… commit: dffe90ee0eaf77785ad3d4ad7fb3249430ed1cb9 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T07:44:21-07:00 summary: bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's tok_nextc() (12601) Remove the PyMem_FREE() call added in cb90c89. The buffer will be freed when PyTokenizer_Free() is called on the tokenizer state. (cherry picked from commit cda139d1ded6708665b53e4ed32ccc1d2627e1da) Co-authored-by: Zackery Spytz <zspytz(a)gmail.com> files: A Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst M Parser/tokenizer.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst b/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst new file mode 100644 index 000000000000..6c234a6a76d9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-03-27-22-35-16.bpo-36459.UAvkKp.rst @@ -0,0 +1 @@ +Fix a possible double ``PyMem_FREE()`` due to tokenizer.c's ``tok_nextc()``. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 6d7869c88e22..89666611940c 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -979,7 +979,6 @@ tok_nextc(register struct tok_state *tok) newbuf = (char *)PyMem_REALLOC(newbuf, newsize); if (newbuf == NULL) { - PyMem_FREE(tok->buf); tok->done = E_NOMEM; tok->cur = tok->inp; return EOF;

1 0

bpo-29515: add missing socket.IPPROTO_* constants on Windows (GH-12183)
by Giampaolo Rodola 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/3eca28c61363a03b81b9fb12775490d6e4… commit: 3eca28c61363a03b81b9fb12775490d6e42d8ecf branch: master author: Giampaolo Rodola <g.rodola(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T15:20:30+01:00 summary: bpo-29515: add missing socket.IPPROTO_* constants on Windows (GH-12183) files: A Misc/NEWS.d/next/Windows/2019-03-05-18-09-43.bpo-29515.vwUTv0.rst M Lib/test/test_socket.py M Modules/socketmodule.c diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 08c7cdef73dd..8a990ea31410 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -867,6 +867,8 @@ def testSendtoErrors(self): def testCrucialConstants(self): # Testing for mission critical constants socket.AF_INET + if socket.has_ipv6: + socket.AF_INET6 socket.SOCK_STREAM socket.SOCK_DGRAM socket.SOCK_RAW @@ -875,6 +877,23 @@ def testCrucialConstants(self): socket.SOL_SOCKET socket.SO_REUSEADDR + def testCrucialIpProtoConstants(self): + socket.IPPROTO_TCP + socket.IPPROTO_UDP + if socket.has_ipv6: + socket.IPPROTO_IPV6 + + @unittest.skipUnless(os.name == "nt", "Windows specific") + def testWindowsSpecificConstants(self): + socket.IPPROTO_ICLFXBM + socket.IPPROTO_ST + socket.IPPROTO_CBT + socket.IPPROTO_IGP + socket.IPPROTO_RDP + socket.IPPROTO_PGM + socket.IPPROTO_L2TP + socket.IPPROTO_SCTP + def testHostnameRes(self): # Testing hostname resolution mechanisms hostname = socket.gethostname() diff --git a/Misc/NEWS.d/next/Windows/2019-03-05-18-09-43.bpo-29515.vwUTv0.rst b/Misc/NEWS.d/next/Windows/2019-03-05-18-09-43.bpo-29515.vwUTv0.rst new file mode 100644 index 000000000000..2f3f0991d9a2 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-03-05-18-09-43.bpo-29515.vwUTv0.rst @@ -0,0 +1,27 @@ +Add the following socket module constants on Windows: +IPPROTO_AH +IPPROTO_CBT +IPPROTO_DSTOPTS +IPPROTO_EGP +IPPROTO_ESP +IPPROTO_FRAGMENT +IPPROTO_GGP +IPPROTO_HOPOPTS +IPPROTO_ICLFXBM +IPPROTO_ICMPV6 +IPPROTO_IDP +IPPROTO_IGMP +IPPROTO_IGP +IPPROTO_IPV4 +IPPROTO_IPV6 +IPPROTO_L2TP +IPPROTO_MAX +IPPROTO_ND +IPPROTO_NONE +IPPROTO_PGM +IPPROTO_PIM +IPPROTO_PUP +IPPROTO_RDP +IPPROTO_ROUTING +IPPROTO_SCTP +IPPROTO_ST diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index b48f8a9c3093..c024542fe709 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -309,6 +309,40 @@ if_indextoname(index) -- return the corresponding interface name\n\ # include <fcntl.h> # endif +/* Macros based on the IPPROTO enum, see: https://bugs.python.org/issue29515 */ +#ifdef MS_WINDOWS +#define IPPROTO_ICMP IPPROTO_ICMP +#define IPPROTO_IGMP IPPROTO_IGMP +#define IPPROTO_GGP IPPROTO_GGP +#define IPPROTO_TCP IPPROTO_TCP +#define IPPROTO_PUP IPPROTO_PUP +#define IPPROTO_UDP IPPROTO_UDP +#define IPPROTO_IDP IPPROTO_IDP +#define IPPROTO_ND IPPROTO_ND +#define IPPROTO_RAW IPPROTO_RAW +#define IPPROTO_MAX IPPROTO_MAX +#define IPPROTO_HOPOPTS IPPROTO_HOPOPTS +#define IPPROTO_IPV4 IPPROTO_IPV4 +#define IPPROTO_IPV6 IPPROTO_IPV6 +#define IPPROTO_ROUTING IPPROTO_ROUTING +#define IPPROTO_FRAGMENT IPPROTO_FRAGMENT +#define IPPROTO_ESP IPPROTO_ESP +#define IPPROTO_AH IPPROTO_AH +#define IPPROTO_ICMPV6 IPPROTO_ICMPV6 +#define IPPROTO_NONE IPPROTO_NONE +#define IPPROTO_DSTOPTS IPPROTO_DSTOPTS +#define IPPROTO_EGP IPPROTO_EGP +#define IPPROTO_PIM IPPROTO_PIM +#define IPPROTO_ICLFXBM IPPROTO_ICLFXBM // WinSock2 only +#define IPPROTO_ST IPPROTO_ST // WinSock2 only +#define IPPROTO_CBT IPPROTO_CBT // WinSock2 only +#define IPPROTO_IGP IPPROTO_IGP // WinSock2 only +#define IPPROTO_RDP IPPROTO_RDP // WinSock2 only +#define IPPROTO_PGM IPPROTO_PGM // WinSock2 only +#define IPPROTO_L2TP IPPROTO_L2TP // WinSock2 only +#define IPPROTO_SCTP IPPROTO_SCTP // WinSock2 only +#endif /* MS_WINDOWS */ + /* Provides the IsWindows7SP1OrGreater() function */ #include <versionhelpers.h> @@ -356,7 +390,7 @@ remove_unusable_flags(PyObject *m) for (int i=0; i<sizeof(win_runtime_flags)/sizeof(FlagRuntimeInfo); i++) { info.dwBuildNumber = win_runtime_flags[i].build_number; - /* greater than or equal to the specified version? + /* greater than or equal to the specified version? Compatibility Mode will not cheat VerifyVersionInfo(...) */ if (VerifyVersionInfo( &info, @@ -7659,6 +7693,17 @@ PyInit__socket(void) PyModule_AddIntMacro(m, IPPROTO_MAX); #endif +#ifdef MS_WINDOWS + PyModule_AddIntMacro(m, IPPROTO_ICLFXBM); + PyModule_AddIntMacro(m, IPPROTO_ST); + PyModule_AddIntMacro(m, IPPROTO_CBT); + PyModule_AddIntMacro(m, IPPROTO_IGP); + PyModule_AddIntMacro(m, IPPROTO_RDP); + PyModule_AddIntMacro(m, IPPROTO_PGM); + PyModule_AddIntMacro(m, IPPROTO_L2TP); + PyModule_AddIntMacro(m, IPPROTO_SCTP); +#endif + #ifdef SYSPROTO_CONTROL PyModule_AddIntMacro(m, SYSPROTO_CONTROL); #endif

1 0

bpo-36387: Refactor getenvironment() in _winapi.c. (GH-12482)
by Serhiy Storchaka 28 Mar '19

28 Mar '19

https://github.com/python/cpython/commit/8abd7c7e37714ce0c42f871f81e52f14c1… commit: 8abd7c7e37714ce0c42f871f81e52f14c155d1bd branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T16:01:34+02:00 summary: bpo-36387: Refactor getenvironment() in _winapi.c. (GH-12482) Make it doing less memory allocations and using the modern C API. files: M Modules/_winapi.c diff --git a/Modules/_winapi.c b/Modules/_winapi.c index e7b221d888ef..2eb708e9073e 100644 --- a/Modules/_winapi.c +++ b/Modules/_winapi.c @@ -752,12 +752,12 @@ gethandle(PyObject* obj, const char* name) return ret; } -static PyObject* +static wchar_t * getenvironment(PyObject* environment) { Py_ssize_t i, envsize, totalsize; - Py_UCS4 *buffer = NULL, *p, *end; - PyObject *keys, *values, *res; + wchar_t *buffer = NULL, *p, *end; + PyObject *keys, *values; /* convert environment dictionary to windows environment string */ if (! PyMapping_Check(environment)) { @@ -775,8 +775,8 @@ getenvironment(PyObject* environment) goto error; } - envsize = PySequence_Fast_GET_SIZE(keys); - if (PySequence_Fast_GET_SIZE(values) != envsize) { + envsize = PyList_GET_SIZE(keys); + if (PyList_GET_SIZE(values) != envsize) { PyErr_SetString(PyExc_RuntimeError, "environment changed size during iteration"); goto error; @@ -784,8 +784,9 @@ getenvironment(PyObject* environment) totalsize = 1; /* trailing null character */ for (i = 0; i < envsize; i++) { - PyObject* key = PySequence_Fast_GET_ITEM(keys, i); - PyObject* value = PySequence_Fast_GET_ITEM(values, i); + PyObject* key = PyList_GET_ITEM(keys, i); + PyObject* value = PyList_GET_ITEM(values, i); + Py_ssize_t size; if (! PyUnicode_Check(key) || ! PyUnicode_Check(value)) { PyErr_SetString(PyExc_TypeError, @@ -806,19 +807,25 @@ getenvironment(PyObject* environment) PyErr_SetString(PyExc_ValueError, "illegal environment variable name"); goto error; } - if (totalsize > PY_SSIZE_T_MAX - PyUnicode_GET_LENGTH(key) - 1) { + + size = PyUnicode_AsWideChar(key, NULL, 0); + assert(size > 1); + if (totalsize > PY_SSIZE_T_MAX - size) { PyErr_SetString(PyExc_OverflowError, "environment too long"); goto error; } - totalsize += PyUnicode_GET_LENGTH(key) + 1; /* +1 for '=' */ - if (totalsize > PY_SSIZE_T_MAX - PyUnicode_GET_LENGTH(value) - 1) { + totalsize += size; /* including '=' */ + + size = PyUnicode_AsWideChar(value, NULL, 0); + assert(size > 0); + if (totalsize > PY_SSIZE_T_MAX - size) { PyErr_SetString(PyExc_OverflowError, "environment too long"); goto error; } - totalsize += PyUnicode_GET_LENGTH(value) + 1; /* +1 for '\0' */ + totalsize += size; /* including trailing '\0' */ } - buffer = PyMem_NEW(Py_UCS4, totalsize); + buffer = PyMem_NEW(wchar_t, totalsize); if (! buffer) { PyErr_NoMemory(); goto error; @@ -827,34 +834,25 @@ getenvironment(PyObject* environment) end = buffer + totalsize; for (i = 0; i < envsize; i++) { - PyObject* key = PySequence_Fast_GET_ITEM(keys, i); - PyObject* value = PySequence_Fast_GET_ITEM(values, i); - if (!PyUnicode_AsUCS4(key, p, end - p, 0)) - goto error; - p += PyUnicode_GET_LENGTH(key); - *p++ = '='; - if (!PyUnicode_AsUCS4(value, p, end - p, 0)) - goto error; - p += PyUnicode_GET_LENGTH(value); - *p++ = '\0'; + PyObject* key = PyList_GET_ITEM(keys, i); + PyObject* value = PyList_GET_ITEM(values, i); + Py_ssize_t size = PyUnicode_AsWideChar(key, p, end - p); + assert(1 <= size && size < end - p); + p += size; + *p++ = L'='; + size = PyUnicode_AsWideChar(value, p, end - p); + assert(0 <= size && size < end - p); + p += size + 1; } - /* add trailing null byte */ - *p++ = '\0'; + /* add trailing null character */ + *p++ = L'\0'; assert(p == end); - Py_XDECREF(keys); - Py_XDECREF(values); - - res = PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, buffer, p - buffer); - PyMem_Free(buffer); - return res; - error: - PyMem_Free(buffer); Py_XDECREF(keys); Py_XDECREF(values); - return NULL; + return buffer; } static LPHANDLE @@ -1053,8 +1051,7 @@ _winapi_CreateProcess_impl(PyObject *module, BOOL result; PROCESS_INFORMATION pi; STARTUPINFOEXW si; - PyObject *environment = NULL; - wchar_t *wenvironment; + wchar_t *wenvironment = NULL; wchar_t *command_line_copy = NULL; AttributeList attribute_list = {0}; @@ -1071,20 +1068,11 @@ _winapi_CreateProcess_impl(PyObject *module, goto cleanup; if (env_mapping != Py_None) { - environment = getenvironment(env_mapping); - if (environment == NULL) { - goto cleanup; - } - /* contains embedded null characters */ - wenvironment = PyUnicode_AsUnicode(environment); + wenvironment = getenvironment(env_mapping); if (wenvironment == NULL) { goto cleanup; } } - else { - environment = NULL; - wenvironment = NULL; - } if (getattributelist(startup_info, "lpAttributeList", &attribute_list) < 0) goto cleanup; @@ -1131,7 +1119,7 @@ _winapi_CreateProcess_impl(PyObject *module, cleanup: PyMem_Free(command_line_copy); - Py_XDECREF(environment); + PyMem_Free(wenvironment); freeattributelist(&attribute_list); return ret;

1 0