Mailman 3 October 2021 - Python-checkins

bpo-45655: Add ref to union type expressions at top of typing docs (GH-29309)
by miss-islington Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/6742b0dfb61ebdb92a1ff633ec071734b5… commit: 6742b0dfb61ebdb92a1ff633ec071734b5d39981 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-10-29T07:16:19-07:00 summary: bpo-45655: Add ref to union type expressions at top of typing docs (GH-29309) (cherry picked from commit d9575218d7ab3d85b15ce3d4779660b9b724d343) Co-authored-by: Alex Waygood <Alex.Waygood(a)Gmail.com> files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e5b0468399e50..7652c87827220 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -61,7 +61,8 @@ annotations. These include: *Introducing* :data:`Annotated` * :pep:`604`: Allow writing union types as ``X | Y`` *Introducing* :data:`types.UnionType` and the ability to use - the binary-or operator ``|`` as syntactic sugar for a union of types + the binary-or operator ``|`` to signify a + :ref:`union of types<types-union>` * :pep:`612`: Parameter Specification Variables *Introducing* :class:`ParamSpec` and :data:`Concatenate` * :pep:`613`: Explicit Type Aliases

1 0

bpo-45655: Add ref to union type expressions at top of typing docs (GH-29309)
by Fidget-Spinner Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/d9575218d7ab3d85b15ce3d4779660b9b7… commit: d9575218d7ab3d85b15ce3d4779660b9b724d343 branch: main author: Alex Waygood <Alex.Waygood(a)Gmail.com> committer: Fidget-Spinner <28750310+Fidget-Spinner(a)users.noreply.github.com> date: 2021-10-29T21:56:21+08:00 summary: bpo-45655: Add ref to union type expressions at top of typing docs (GH-29309) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 6f501ec136ee6..eb95af378d45f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -61,7 +61,8 @@ annotations. These include: *Introducing* :data:`Annotated` * :pep:`604`: Allow writing union types as ``X | Y`` *Introducing* :data:`types.UnionType` and the ability to use - the binary-or operator ``|`` as syntactic sugar for a union of types + the binary-or operator ``|`` to signify a + :ref:`union of types<types-union>` * :pep:`612`: Parameter Specification Variables *Introducing* :class:`ParamSpec` and :data:`Concatenate` * :pep:`613`: Explicit Type Aliases

1 0

bpo-45628: Check all parts of the suffix for an extension match. (GH-29310)
by vsajip Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/8a77f59de51f1fd6062f0fefe73ee3059d… commit: 8a77f59de51f1fd6062f0fefe73ee3059d714144 branch: main author: Vinay Sajip <vinay_sajip(a)yahoo.co.uk> committer: vsajip <vinay_sajip(a)yahoo.co.uk> date: 2021-10-29T14:40:37+01:00 summary: bpo-45628: Check all parts of the suffix for an extension match. (GH-29310) files: M Lib/logging/handlers.py diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index b613bec1c4270..d42c48de5f062 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -371,8 +371,13 @@ def getFilesToDelete(self): for fileName in fileNames: if fileName[:plen] == prefix: suffix = fileName[plen:] - if self.extMatch.match(suffix): - result.append(os.path.join(dirName, fileName)) + # See bpo-45628: The date/time suffix could be anywhere in the + # filename + parts = suffix.split('.') + for part in parts: + if self.extMatch.match(part): + result.append(os.path.join(dirName, fileName)) + break if len(result) < self.backupCount: result = [] else:

1 0

[3.9] [3.10] bpo-45502: Fix test_shelve (GH-29003) (GH-29305) (GH-29306)
by serhiy-storchaka Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/a043706f907e82ee6a562005991ff0b896… commit: a043706f907e82ee6a562005991ff0b896a4e64d branch: 3.9 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-10-29T13:18:34+03:00 summary: [3.9] [3.10] bpo-45502: Fix test_shelve (GH-29003) (GH-29305) (GH-29306) Run test_shelve with all underlying dbm implementations and pickle protocols. Also make test_shelve discoverable. (cherry picked from commit b781cc3bfce7c052728b06aad9f1a467cced289d). (cherry picked from commit 6b867022d926be9fcc6f8038fb1093ba8c348ca5) files: M Lib/test/test_shelve.py diff --git a/Lib/test/test_shelve.py b/Lib/test/test_shelve.py index 9ffe2cbeae4d8..61dae75cb325e 100644 --- a/Lib/test/test_shelve.py +++ b/Lib/test/test_shelve.py @@ -1,6 +1,9 @@ import unittest +import dbm +import os import shelve import glob +import pickle from test import support from collections.abc import MutableMapping from test.test_dbm import dbm_iterator @@ -40,12 +43,8 @@ def copy(self): class TestCase(unittest.TestCase): - - fn = "shelftemp.db" - - def tearDown(self): - for f in glob.glob(self.fn+"*"): - support.unlink(f) + dirname = support.TESTFN + fn = os.path.join(support.TESTFN, "shelftemp.db") def test_close(self): d1 = {} @@ -62,29 +61,24 @@ def test_close(self): else: self.fail('Closed shelf should not find a key') - def test_ascii_file_shelf(self): - s = shelve.open(self.fn, protocol=0) + def test_open_template(self, protocol=None): + os.mkdir(self.dirname) + self.addCleanup(support.rmtree, self.dirname) + s = shelve.open(self.fn, protocol=protocol) try: s['key1'] = (1,2,3,4) self.assertEqual(s['key1'], (1,2,3,4)) finally: s.close() + def test_ascii_file_shelf(self): + self.test_open_template(protocol=0) + def test_binary_file_shelf(self): - s = shelve.open(self.fn, protocol=1) - try: - s['key1'] = (1,2,3,4) - self.assertEqual(s['key1'], (1,2,3,4)) - finally: - s.close() + self.test_open_template(protocol=1) def test_proto2_file_shelf(self): - s = shelve.open(self.fn, protocol=2) - try: - s['key1'] = (1,2,3,4) - self.assertEqual(s['key1'], (1,2,3,4)) - finally: - s.close() + self.test_open_template(protocol=2) def test_in_memory_shelf(self): d1 = byteskeydict() @@ -161,63 +155,52 @@ def test_default_protocol(self): with shelve.Shelf({}) as s: self.assertEqual(s._protocol, 3) -from test import mapping_tests -class TestShelveBase(mapping_tests.BasicTestMappingProtocol): - fn = "shelftemp.db" - counter = 0 - def __init__(self, *args, **kw): - self._db = [] - mapping_tests.BasicTestMappingProtocol.__init__(self, *args, **kw) +class TestShelveBase: type2test = shelve.Shelf + def _reference(self): return {"key1":"value1", "key2":2, "key3":(1,2,3)} + + +class TestShelveInMemBase(TestShelveBase): def _empty_mapping(self): - if self._in_mem: - x= shelve.Shelf(byteskeydict(), **self._args) - else: - self.counter+=1 - x= shelve.open(self.fn+str(self.counter), **self._args) - self._db.append(x) + return shelve.Shelf(byteskeydict(), **self._args) + + +class TestShelveFileBase(TestShelveBase): + counter = 0 + + def _empty_mapping(self): + self.counter += 1 + x = shelve.open(self.base_path + str(self.counter), **self._args) + self.addCleanup(x.close) return x - def tearDown(self): - for db in self._db: - db.close() - self._db = [] - if not self._in_mem: - for f in glob.glob(self.fn+"*"): - support.unlink(f) - -class TestAsciiFileShelve(TestShelveBase): - _args={'protocol':0} - _in_mem = False -class TestBinaryFileShelve(TestShelveBase): - _args={'protocol':1} - _in_mem = False -class TestProto2FileShelve(TestShelveBase): - _args={'protocol':2} - _in_mem = False -class TestAsciiMemShelve(TestShelveBase): - _args={'protocol':0} - _in_mem = True -class TestBinaryMemShelve(TestShelveBase): - _args={'protocol':1} - _in_mem = True -class TestProto2MemShelve(TestShelveBase): - _args={'protocol':2} - _in_mem = True - -def test_main(): - for module in dbm_iterator(): - support.run_unittest( - TestAsciiFileShelve, - TestBinaryFileShelve, - TestProto2FileShelve, - TestAsciiMemShelve, - TestBinaryMemShelve, - TestProto2MemShelve, - TestCase - ) + + def setUp(self): + dirname = support.TESTFN + os.mkdir(dirname) + self.addCleanup(support.rmtree, dirname) + self.base_path = os.path.join(dirname, "shelftemp.db") + self.addCleanup(setattr, dbm, '_defaultmod', dbm._defaultmod) + dbm._defaultmod = self.dbm_mod + + +from test import mapping_tests + +for proto in range(pickle.HIGHEST_PROTOCOL + 1): + bases = (TestShelveInMemBase, mapping_tests.BasicTestMappingProtocol) + name = f'TestProto{proto}MemShelve' + globals()[name] = type(name, bases, + {'_args': {'protocol': proto}}) + bases = (TestShelveFileBase, mapping_tests.BasicTestMappingProtocol) + for dbm_mod in dbm_iterator(): + assert dbm_mod.__name__.startswith('dbm.') + suffix = dbm_mod.__name__[4:] + name = f'TestProto{proto}File_{suffix}Shelve' + globals()[name] = type(name, bases, + {'dbm_mod': dbm_mod, '_args': {'protocol': proto}}) + if __name__ == "__main__": - test_main() + unittest.main()

1 0

[3.10] bo-45655: Add "relevant PEPs" section to typing documentation (GH-29302)
by Fidget-Spinner Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/2c8a0027e5555e371c1293f26b3262000b… commit: 2c8a0027e5555e371c1293f26b3262000b8cfe8a branch: 3.10 author: Alex Waygood <Alex.Waygood(a)Gmail.com> committer: Fidget-Spinner <28750310+Fidget-Spinner(a)users.noreply.github.com> date: 2021-10-29T18:03:48+08:00 summary: [3.10] bo-45655: Add "relevant PEPs" section to typing documentation (GH-29302) The list of PEPs at the top of the documentation for the ``typing`` module has become too long to be readable. This PR proposes presenting this information in a more structured and readable way by adding a new "relevant PEPs" section to the ``typing`` docs. (cherry picked from commit 03db1bbfd2d3f5a343c293b2f0e09a1e962df7ea) Co-authored-by: Alex Waygood <Alex.Waygood(a)Gmail.com> Co-authored-by: Łukasz Langa <lukasz(a)langa.pl> files: A Misc/NEWS.d/next/Documentation/2021-10-28-19-22-55.bpo-45655.aPYGaS.rst M Doc/library/typing.rst M Doc/tools/susp-ignored.csv diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c402c73b487ea..e5b0468399e50 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -17,13 +17,11 @@ -------------- -This module provides runtime support for type hints as specified by -:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, :pep:`591`, -:pep:`593`, :pep:`612`, :pep:`613` and :pep:`647`. -The most fundamental support consists of the types :data:`Any`, :data:`Union`, -:data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and -:class:`Generic`. For full specification please see :pep:`484`. For -a simplified introduction to type hints see :pep:`483`. +This module provides runtime support for type hints. The most fundamental +support consists of the types :data:`Any`, :data:`Union`, :data:`Tuple`, +:data:`Callable`, :class:`TypeVar`, and :class:`Generic`. For a full +specification, please see :pep:`484`. For a simplified introduction to type +hints, see :pep:`483`. The function below takes and returns a string and is annotated as follows:: @@ -35,6 +33,42 @@ In the function ``greeting``, the argument ``name`` is expected to be of type :class:`str` and the return type :class:`str`. Subtypes are accepted as arguments. +.. _relevant-peps: + +Relevant PEPs +============= + +Since the initial introduction of type hints in :pep:`484` and :pep:`483`, a +number of PEPs have modified and enhanced Python's framework for type +annotations. These include: + +* :pep:`526`: Syntax for Variable Annotations + *Introducing* syntax for annotating variables outside of function + definitions, and :data:`ClassVar` +* :pep:`544`: Protocols: Structural subtyping (static duck typing) + *Introducing* :class:`Protocol` and the + :func:`@runtime_checkable<runtime_checkable>` decorator +* :pep:`585`: Type Hinting Generics In Standard Collections + *Introducing* the ability to use builtin collections and ABCs as + :term:`generic types<generic type>` +* :pep:`586`: Literal Types + *Introducing* :data:`Literal` +* :pep:`589`: TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys + *Introducing* :class:`TypedDict` +* :pep:`591`: Adding a final qualifier to typing + *Introducing* :data:`Final` and the :func:`@final<final>` decorator +* :pep:`593`: Flexible function and variable annotations + *Introducing* :data:`Annotated` +* :pep:`604`: Allow writing union types as ``X | Y`` + *Introducing* :data:`types.UnionType` and the ability to use + the binary-or operator ``|`` as syntactic sugar for a union of types +* :pep:`612`: Parameter Specification Variables + *Introducing* :class:`ParamSpec` and :data:`Concatenate` +* :pep:`613`: Explicit Type Aliases + *Introducing* :data:`TypeAlias` +* :pep:`647`: User-Defined Type Guards + *Introducing* :data:`TypeGuard` + .. _type-aliases: Type aliases diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index 54360c2f8ebaa..9cbd09ad20357 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -376,7 +376,7 @@ library/importlib.metadata,,:main,"EntryPoint(name='wheel', value='wheel.cli:mai library/importlib.metadata,,`,loading the metadata for packages for the indicated ``context``. library/re,,`,"`" using/configure,84,:db2,=db1:db2:... -library/typing,1011,`,# Type of ``val`` is narrowed to ``str`` -library/typing,1011,`,"# Else, type of ``val`` is narrowed to ``float``." -library/typing,1011,`,# Type of ``val`` is narrowed to ``List[str]``. -library/typing,1011,`,# Type of ``val`` remains as ``List[object]``. +library/typing,,`,# Type of ``val`` is narrowed to ``str`` +library/typing,,`,"# Else, type of ``val`` is narrowed to ``float``." +library/typing,,`,# Type of ``val`` is narrowed to ``List[str]``. +library/typing,,`,# Type of ``val`` remains as ``List[object]``. diff --git a/Misc/NEWS.d/next/Documentation/2021-10-28-19-22-55.bpo-45655.aPYGaS.rst b/Misc/NEWS.d/next/Documentation/2021-10-28-19-22-55.bpo-45655.aPYGaS.rst new file mode 100644 index 0000000000000..fc5b3d0788817 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2021-10-28-19-22-55.bpo-45655.aPYGaS.rst @@ -0,0 +1,2 @@ +Add a new "relevant PEPs" section to the top of the documentation for the +``typing`` module. Patch by Alex Waygood.

1 0

[3.10] bpo-45502: Fix test_shelve (GH-29003) (GH-29305)
by serhiy-storchaka Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/6b867022d926be9fcc6f8038fb1093ba8c… commit: 6b867022d926be9fcc6f8038fb1093ba8c348ca5 branch: 3.10 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-10-29T12:40:22+03:00 summary: [3.10] bpo-45502: Fix test_shelve (GH-29003) (GH-29305) Run test_shelve with all underlying dbm implementations and pickle protocols. Also make test_shelve discoverable. (cherry picked from commit b781cc3bfce7c052728b06aad9f1a467cced289d) files: M Lib/test/test_shelve.py diff --git a/Lib/test/test_shelve.py b/Lib/test/test_shelve.py index cfdd67c26c5f5..dcb5aefc3bae1 100644 --- a/Lib/test/test_shelve.py +++ b/Lib/test/test_shelve.py @@ -1,4 +1,6 @@ import unittest +import dbm +import os import shelve import glob import pickle @@ -43,12 +45,8 @@ def copy(self): class TestCase(unittest.TestCase): - - fn = "shelftemp.db" - - def tearDown(self): - for f in glob.glob(self.fn+"*"): - os_helper.unlink(f) + dirname = os_helper.TESTFN + fn = os.path.join(os_helper.TESTFN, "shelftemp.db") def test_close(self): d1 = {} @@ -65,29 +63,24 @@ def test_close(self): else: self.fail('Closed shelf should not find a key') - def test_ascii_file_shelf(self): - s = shelve.open(self.fn, protocol=0) + def test_open_template(self, protocol=None): + os.mkdir(self.dirname) + self.addCleanup(os_helper.rmtree, self.dirname) + s = shelve.open(self.fn, protocol=protocol) try: s['key1'] = (1,2,3,4) self.assertEqual(s['key1'], (1,2,3,4)) finally: s.close() + def test_ascii_file_shelf(self): + self.test_open_template(protocol=0) + def test_binary_file_shelf(self): - s = shelve.open(self.fn, protocol=1) - try: - s['key1'] = (1,2,3,4) - self.assertEqual(s['key1'], (1,2,3,4)) - finally: - s.close() + self.test_open_template(protocol=1) def test_proto2_file_shelf(self): - s = shelve.open(self.fn, protocol=2) - try: - s['key1'] = (1,2,3,4) - self.assertEqual(s['key1'], (1,2,3,4)) - finally: - s.close() + self.test_open_template(protocol=2) def test_in_memory_shelf(self): d1 = byteskeydict() @@ -164,63 +157,52 @@ def test_default_protocol(self): with shelve.Shelf({}) as s: self.assertEqual(s._protocol, pickle.DEFAULT_PROTOCOL) -from test import mapping_tests -class TestShelveBase(mapping_tests.BasicTestMappingProtocol): - fn = "shelftemp.db" - counter = 0 - def __init__(self, *args, **kw): - self._db = [] - mapping_tests.BasicTestMappingProtocol.__init__(self, *args, **kw) +class TestShelveBase: type2test = shelve.Shelf + def _reference(self): return {"key1":"value1", "key2":2, "key3":(1,2,3)} + + +class TestShelveInMemBase(TestShelveBase): def _empty_mapping(self): - if self._in_mem: - x= shelve.Shelf(byteskeydict(), **self._args) - else: - self.counter+=1 - x= shelve.open(self.fn+str(self.counter), **self._args) - self._db.append(x) + return shelve.Shelf(byteskeydict(), **self._args) + + +class TestShelveFileBase(TestShelveBase): + counter = 0 + + def _empty_mapping(self): + self.counter += 1 + x = shelve.open(self.base_path + str(self.counter), **self._args) + self.addCleanup(x.close) return x - def tearDown(self): - for db in self._db: - db.close() - self._db = [] - if not self._in_mem: - for f in glob.glob(self.fn+"*"): - os_helper.unlink(f) - -class TestAsciiFileShelve(TestShelveBase): - _args={'protocol':0} - _in_mem = False -class TestBinaryFileShelve(TestShelveBase): - _args={'protocol':1} - _in_mem = False -class TestProto2FileShelve(TestShelveBase): - _args={'protocol':2} - _in_mem = False -class TestAsciiMemShelve(TestShelveBase): - _args={'protocol':0} - _in_mem = True -class TestBinaryMemShelve(TestShelveBase): - _args={'protocol':1} - _in_mem = True -class TestProto2MemShelve(TestShelveBase): - _args={'protocol':2} - _in_mem = True - -def test_main(): - for module in dbm_iterator(): - support.run_unittest( - TestAsciiFileShelve, - TestBinaryFileShelve, - TestProto2FileShelve, - TestAsciiMemShelve, - TestBinaryMemShelve, - TestProto2MemShelve, - TestCase - ) + + def setUp(self): + dirname = os_helper.TESTFN + os.mkdir(dirname) + self.addCleanup(os_helper.rmtree, dirname) + self.base_path = os.path.join(dirname, "shelftemp.db") + self.addCleanup(setattr, dbm, '_defaultmod', dbm._defaultmod) + dbm._defaultmod = self.dbm_mod + + +from test import mapping_tests + +for proto in range(pickle.HIGHEST_PROTOCOL + 1): + bases = (TestShelveInMemBase, mapping_tests.BasicTestMappingProtocol) + name = f'TestProto{proto}MemShelve' + globals()[name] = type(name, bases, + {'_args': {'protocol': proto}}) + bases = (TestShelveFileBase, mapping_tests.BasicTestMappingProtocol) + for dbm_mod in dbm_iterator(): + assert dbm_mod.__name__.startswith('dbm.') + suffix = dbm_mod.__name__[4:] + name = f'TestProto{proto}File_{suffix}Shelve' + globals()[name] = type(name, bases, + {'dbm_mod': dbm_mod, '_args': {'protocol': proto}}) + if __name__ == "__main__": - test_main() + unittest.main()

1 0

bpo-45621: Small changes to mmap (GH-29247)
by tjguk Oct. 29, 2021

Oct. 29, 2021

https://github.com/python/cpython/commit/7bddd96982072d04bd6314da1ee7f40b7f… commit: 7bddd96982072d04bd6314da1ee7f40b7f875f00 branch: main author: Tim Golden <mail(a)timgolden.me.uk> committer: tjguk <mail(a)timgolden.me.uk> date: 2021-10-29T09:20:21+01:00 summary: bpo-45621: Small changes to mmap (GH-29247) * Small tidy-ups / comments * Use randomized names when testing tagged mmaps to avoid any risk of parallel tests treading on each others' toes files: M Lib/test/test_mmap.py M Modules/mmapmodule.c diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 82e2d2adb7dcf..014171cbb4911 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -7,6 +7,7 @@ import itertools import random import socket +import string import sys import weakref @@ -15,6 +16,10 @@ PAGESIZE = mmap.PAGESIZE +tagname_prefix = f'python_{os.getpid()}_test_mmap' +def random_tagname(length=10): + suffix = ''.join(random.choices(string.ascii_uppercase, k=length)) + return f'{tagname_prefix}_{suffix}' class MmapTests(unittest.TestCase): @@ -610,11 +615,13 @@ def test_tagname(self): data1 = b"0123456789" data2 = b"abcdefghij" assert len(data1) == len(data2) + tagname1 = random_tagname() + tagname2 = random_tagname() # Test same tag - m1 = mmap.mmap(-1, len(data1), tagname="foo") + m1 = mmap.mmap(-1, len(data1), tagname=tagname1) m1[:] = data1 - m2 = mmap.mmap(-1, len(data2), tagname="foo") + m2 = mmap.mmap(-1, len(data2), tagname=tagname1) m2[:] = data2 self.assertEqual(m1[:], data2) self.assertEqual(m2[:], data2) @@ -622,9 +629,9 @@ def test_tagname(self): m1.close() # Test different tag - m1 = mmap.mmap(-1, len(data1), tagname="foo") + m1 = mmap.mmap(-1, len(data1), tagname=tagname1) m1[:] = data1 - m2 = mmap.mmap(-1, len(data2), tagname="boo") + m2 = mmap.mmap(-1, len(data2), tagname=tagname2) m2[:] = data2 self.assertEqual(m1[:], data1) self.assertEqual(m2[:], data2) @@ -635,7 +642,7 @@ def test_tagname(self): @unittest.skipUnless(os.name == 'nt', 'requires Windows') def test_sizeof(self): m1 = mmap.mmap(-1, 100) - tagname = "foo" + tagname = random_tagname() m2 = mmap.mmap(-1, 100, tagname=tagname) self.assertEqual(sys.getsizeof(m2), sys.getsizeof(m1) + len(tagname) + 1) @@ -643,9 +650,10 @@ def test_sizeof(self): @unittest.skipUnless(os.name == 'nt', 'requires Windows') def test_crasher_on_windows(self): # Should not crash (Issue 1733986) - m = mmap.mmap(-1, 1000, tagname="foo") + tagname = random_tagname() + m = mmap.mmap(-1, 1000, tagname=tagname) try: - mmap.mmap(-1, 5000, tagname="foo")[:] # same tagname, but larger size + mmap.mmap(-1, 5000, tagname=tagname)[:] # same tagname, but larger size except: pass m.close() @@ -857,7 +865,7 @@ def test_resize_succeeds_with_error_for_second_named_mapping(self): """ start_size = 2 * PAGESIZE reduced_size = PAGESIZE - tagname = "TEST" + tagname = random_tagname() data_length = 8 data = bytes(random.getrandbits(8) for _ in range(data_length)) diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index dfa10f633bbd5..21d53365776ab 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -32,7 +32,6 @@ #ifdef MS_WINDOWS #include <windows.h> -#include <winternl.h> static int my_getpagesize(void) { @@ -505,21 +504,6 @@ mmap_resize_method(mmap_object *self, } { - /* - To resize an mmap on Windows: - - - Close the existing mapping - - If the mapping is backed to a named file: - unmap the view, clear the data, and resize the file - If the file can't be resized (eg because it has other mapped references - to it) then let the mapping be recreated at the original size and set - an error code so an exception will be raised. - - Create a new mapping of the relevant size to the same file - - Map a new view of the resized file - - If the mapping is backed by the pagefile: - copy any previous data into the new mapped area - unmap the original view which will release the memory - */ #ifdef MS_WINDOWS DWORD error = 0, file_resize_error = 0; char* old_data = self->data; @@ -567,6 +551,11 @@ mmap_resize_method(mmap_object *self, self->tagname); error = GetLastError(); + /* ERROR_ALREADY_EXISTS implies that between our closing the handle above and + calling CreateFileMapping here, someone's created a different mapping with + the same name. There's nothing we can usefully do so we invalidate our + mapping and error out. + */ if (error == ERROR_ALREADY_EXISTS) { CloseHandle(self->map_handle); self->map_handle = NULL;

1 0

bpo-45577: test all pickle protocols in `test_zoneinfo` (GH-29167) (GH-29296)
by ambv Oct. 28, 2021

Oct. 28, 2021

https://github.com/python/cpython/commit/7203ecd332eca3a44a3f1c8bdadd76a08c… commit: 7203ecd332eca3a44a3f1c8bdadd76a08c5568a1 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T00:00:49+02:00 summary: bpo-45577: test all pickle protocols in `test_zoneinfo` (GH-29167) (GH-29296) (cherry picked from commit 66e6b3dcd3bbab06feeff2cbaf8aade7b6223d6c) Co-authored-by: Nikita Sobolev <mail(a)sobolevn.me> files: A Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst M Lib/test/test_zoneinfo/test_zoneinfo.py diff --git a/Lib/test/test_zoneinfo/test_zoneinfo.py b/Lib/test/test_zoneinfo/test_zoneinfo.py index a9375fd55857b..59b35ef63f987 100644 --- a/Lib/test/test_zoneinfo/test_zoneinfo.py +++ b/Lib/test/test_zoneinfo/test_zoneinfo.py @@ -1403,44 +1403,50 @@ def tzpath(self): return [self.zoneinfo_data.tzpath] def test_cache_hit(self): - zi_in = self.klass("Europe/Dublin") - pkl = pickle.dumps(zi_in) - zi_rt = pickle.loads(pkl) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_in = self.klass("Europe/Dublin") + pkl = pickle.dumps(zi_in, protocol=proto) + zi_rt = pickle.loads(pkl) - with self.subTest(test="Is non-pickled ZoneInfo"): - self.assertIs(zi_in, zi_rt) + with self.subTest(test="Is non-pickled ZoneInfo"): + self.assertIs(zi_in, zi_rt) - zi_rt2 = pickle.loads(pkl) - with self.subTest(test="Is unpickled ZoneInfo"): - self.assertIs(zi_rt, zi_rt2) + zi_rt2 = pickle.loads(pkl) + with self.subTest(test="Is unpickled ZoneInfo"): + self.assertIs(zi_rt, zi_rt2) def test_cache_miss(self): - zi_in = self.klass("Europe/Dublin") - pkl = pickle.dumps(zi_in) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_in = self.klass("Europe/Dublin") + pkl = pickle.dumps(zi_in, protocol=proto) - del zi_in - self.klass.clear_cache() # Induce a cache miss - zi_rt = pickle.loads(pkl) - zi_rt2 = pickle.loads(pkl) + del zi_in + self.klass.clear_cache() # Induce a cache miss + zi_rt = pickle.loads(pkl) + zi_rt2 = pickle.loads(pkl) - self.assertIs(zi_rt, zi_rt2) + self.assertIs(zi_rt, zi_rt2) def test_no_cache(self): - zi_no_cache = self.klass.no_cache("Europe/Dublin") + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_no_cache = self.klass.no_cache("Europe/Dublin") - pkl = pickle.dumps(zi_no_cache) - zi_rt = pickle.loads(pkl) + pkl = pickle.dumps(zi_no_cache, protocol=proto) + zi_rt = pickle.loads(pkl) - with self.subTest(test="Not the pickled object"): - self.assertIsNot(zi_rt, zi_no_cache) + with self.subTest(test="Not the pickled object"): + self.assertIsNot(zi_rt, zi_no_cache) - zi_rt2 = pickle.loads(pkl) - with self.subTest(test="Not a second unpickled object"): - self.assertIsNot(zi_rt, zi_rt2) + zi_rt2 = pickle.loads(pkl) + with self.subTest(test="Not a second unpickled object"): + self.assertIsNot(zi_rt, zi_rt2) - zi_cache = self.klass("Europe/Dublin") - with self.subTest(test="Not a cached object"): - self.assertIsNot(zi_rt, zi_cache) + zi_cache = self.klass("Europe/Dublin") + with self.subTest(test="Not a cached object"): + self.assertIsNot(zi_rt, zi_cache) def test_from_file(self): key = "Europe/Dublin" @@ -1456,35 +1462,38 @@ def test_from_file(self): ] for zi, test_name in test_cases: - with self.subTest(test_name=test_name): - with self.assertRaises(pickle.PicklingError): - pickle.dumps(zi) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(test_name=test_name, proto=proto): + with self.assertRaises(pickle.PicklingError): + pickle.dumps(zi, protocol=proto) def test_pickle_after_from_file(self): # This may be a bit of paranoia, but this test is to ensure that no # global state is maintained in order to handle the pickle cache and # from_file behavior, and that it is possible to interweave the # constructors of each of these and pickling/unpickling without issues. - key = "Europe/Dublin" - zi = self.klass(key) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + key = "Europe/Dublin" + zi = self.klass(key) - pkl_0 = pickle.dumps(zi) - zi_rt_0 = pickle.loads(pkl_0) - self.assertIs(zi, zi_rt_0) + pkl_0 = pickle.dumps(zi, protocol=proto) + zi_rt_0 = pickle.loads(pkl_0) + self.assertIs(zi, zi_rt_0) - with open(self.zoneinfo_data.path_from_key(key), "rb") as f: - zi_ff = self.klass.from_file(f, key=key) + with open(self.zoneinfo_data.path_from_key(key), "rb") as f: + zi_ff = self.klass.from_file(f, key=key) - pkl_1 = pickle.dumps(zi) - zi_rt_1 = pickle.loads(pkl_1) - self.assertIs(zi, zi_rt_1) + pkl_1 = pickle.dumps(zi, protocol=proto) + zi_rt_1 = pickle.loads(pkl_1) + self.assertIs(zi, zi_rt_1) - with self.assertRaises(pickle.PicklingError): - pickle.dumps(zi_ff) + with self.assertRaises(pickle.PicklingError): + pickle.dumps(zi_ff, protocol=proto) - pkl_2 = pickle.dumps(zi) - zi_rt_2 = pickle.loads(pkl_2) - self.assertIs(zi, zi_rt_2) + pkl_2 = pickle.dumps(zi, protocol=proto) + zi_rt_2 = pickle.loads(pkl_2) + self.assertIs(zi, zi_rt_2) class CZoneInfoPickleTest(ZoneInfoPickleTest): diff --git a/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst b/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst new file mode 100644 index 0000000000000..fc9783eb9ddaa --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst @@ -0,0 +1 @@ +Add subtests for all ``pickle`` protocols in ``test_zoneinfo``.

1 0

bpo-45577: test all pickle protocols in `test_zoneinfo` (GH-29167) (GH-29295)
by ambv Oct. 28, 2021

Oct. 28, 2021

https://github.com/python/cpython/commit/dd674ca96f2150fb3f7b4086ef7ec0022c… commit: dd674ca96f2150fb3f7b4086ef7ec0022c4e2058 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-29T00:00:34+02:00 summary: bpo-45577: test all pickle protocols in `test_zoneinfo` (GH-29167) (GH-29295) (cherry picked from commit 66e6b3dcd3bbab06feeff2cbaf8aade7b6223d6c) Co-authored-by: Nikita Sobolev <mail(a)sobolevn.me> files: A Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst M Lib/test/test_zoneinfo/test_zoneinfo.py diff --git a/Lib/test/test_zoneinfo/test_zoneinfo.py b/Lib/test/test_zoneinfo/test_zoneinfo.py index d4704b75d389d..08ebbb1dd9911 100644 --- a/Lib/test/test_zoneinfo/test_zoneinfo.py +++ b/Lib/test/test_zoneinfo/test_zoneinfo.py @@ -1403,44 +1403,50 @@ def tzpath(self): return [self.zoneinfo_data.tzpath] def test_cache_hit(self): - zi_in = self.klass("Europe/Dublin") - pkl = pickle.dumps(zi_in) - zi_rt = pickle.loads(pkl) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_in = self.klass("Europe/Dublin") + pkl = pickle.dumps(zi_in, protocol=proto) + zi_rt = pickle.loads(pkl) - with self.subTest(test="Is non-pickled ZoneInfo"): - self.assertIs(zi_in, zi_rt) + with self.subTest(test="Is non-pickled ZoneInfo"): + self.assertIs(zi_in, zi_rt) - zi_rt2 = pickle.loads(pkl) - with self.subTest(test="Is unpickled ZoneInfo"): - self.assertIs(zi_rt, zi_rt2) + zi_rt2 = pickle.loads(pkl) + with self.subTest(test="Is unpickled ZoneInfo"): + self.assertIs(zi_rt, zi_rt2) def test_cache_miss(self): - zi_in = self.klass("Europe/Dublin") - pkl = pickle.dumps(zi_in) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_in = self.klass("Europe/Dublin") + pkl = pickle.dumps(zi_in, protocol=proto) - del zi_in - self.klass.clear_cache() # Induce a cache miss - zi_rt = pickle.loads(pkl) - zi_rt2 = pickle.loads(pkl) + del zi_in + self.klass.clear_cache() # Induce a cache miss + zi_rt = pickle.loads(pkl) + zi_rt2 = pickle.loads(pkl) - self.assertIs(zi_rt, zi_rt2) + self.assertIs(zi_rt, zi_rt2) def test_no_cache(self): - zi_no_cache = self.klass.no_cache("Europe/Dublin") + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + zi_no_cache = self.klass.no_cache("Europe/Dublin") - pkl = pickle.dumps(zi_no_cache) - zi_rt = pickle.loads(pkl) + pkl = pickle.dumps(zi_no_cache, protocol=proto) + zi_rt = pickle.loads(pkl) - with self.subTest(test="Not the pickled object"): - self.assertIsNot(zi_rt, zi_no_cache) + with self.subTest(test="Not the pickled object"): + self.assertIsNot(zi_rt, zi_no_cache) - zi_rt2 = pickle.loads(pkl) - with self.subTest(test="Not a second unpickled object"): - self.assertIsNot(zi_rt, zi_rt2) + zi_rt2 = pickle.loads(pkl) + with self.subTest(test="Not a second unpickled object"): + self.assertIsNot(zi_rt, zi_rt2) - zi_cache = self.klass("Europe/Dublin") - with self.subTest(test="Not a cached object"): - self.assertIsNot(zi_rt, zi_cache) + zi_cache = self.klass("Europe/Dublin") + with self.subTest(test="Not a cached object"): + self.assertIsNot(zi_rt, zi_cache) def test_from_file(self): key = "Europe/Dublin" @@ -1456,35 +1462,38 @@ def test_from_file(self): ] for zi, test_name in test_cases: - with self.subTest(test_name=test_name): - with self.assertRaises(pickle.PicklingError): - pickle.dumps(zi) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(test_name=test_name, proto=proto): + with self.assertRaises(pickle.PicklingError): + pickle.dumps(zi, protocol=proto) def test_pickle_after_from_file(self): # This may be a bit of paranoia, but this test is to ensure that no # global state is maintained in order to handle the pickle cache and # from_file behavior, and that it is possible to interweave the # constructors of each of these and pickling/unpickling without issues. - key = "Europe/Dublin" - zi = self.klass(key) + for proto in range(pickle.HIGHEST_PROTOCOL + 1): + with self.subTest(proto=proto): + key = "Europe/Dublin" + zi = self.klass(key) - pkl_0 = pickle.dumps(zi) - zi_rt_0 = pickle.loads(pkl_0) - self.assertIs(zi, zi_rt_0) + pkl_0 = pickle.dumps(zi, protocol=proto) + zi_rt_0 = pickle.loads(pkl_0) + self.assertIs(zi, zi_rt_0) - with open(self.zoneinfo_data.path_from_key(key), "rb") as f: - zi_ff = self.klass.from_file(f, key=key) + with open(self.zoneinfo_data.path_from_key(key), "rb") as f: + zi_ff = self.klass.from_file(f, key=key) - pkl_1 = pickle.dumps(zi) - zi_rt_1 = pickle.loads(pkl_1) - self.assertIs(zi, zi_rt_1) + pkl_1 = pickle.dumps(zi, protocol=proto) + zi_rt_1 = pickle.loads(pkl_1) + self.assertIs(zi, zi_rt_1) - with self.assertRaises(pickle.PicklingError): - pickle.dumps(zi_ff) + with self.assertRaises(pickle.PicklingError): + pickle.dumps(zi_ff, protocol=proto) - pkl_2 = pickle.dumps(zi) - zi_rt_2 = pickle.loads(pkl_2) - self.assertIs(zi, zi_rt_2) + pkl_2 = pickle.dumps(zi, protocol=proto) + zi_rt_2 = pickle.loads(pkl_2) + self.assertIs(zi, zi_rt_2) class CZoneInfoPickleTest(ZoneInfoPickleTest): diff --git a/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst b/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst new file mode 100644 index 0000000000000..fc9783eb9ddaa --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2021-10-22-19-44-13.bpo-45577.dSaNvK.rst @@ -0,0 +1 @@ +Add subtests for all ``pickle`` protocols in ``test_zoneinfo``.

1 0

[3.9] bpo-45655: Add "relevant PEPs" section to typing documentation (GH-29297)
by ambv Oct. 28, 2021

Oct. 28, 2021

https://github.com/python/cpython/commit/8813a987b1df78b5eaddb085e514dfa5af… commit: 8813a987b1df78b5eaddb085e514dfa5af5c8634 branch: 3.9 author: Alex Waygood <Alex.Waygood(a)Gmail.com> committer: ambv <lukasz(a)langa.pl> date: 2021-10-28T23:17:28+02:00 summary: [3.9] bpo-45655: Add "relevant PEPs" section to typing documentation (GH-29297) Backport of https://github.com/python/cpython/pull/29280 to the 3.9 branch. files: A Misc/NEWS.d/next/Documentation/2021-10-28-21-28-07.bpo-45655.aPYGaS.rst M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 78ab760d86a54..295f2ae8905af 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -17,12 +17,11 @@ -------------- -This module provides runtime support for type hints as specified by -:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, and :pep:`591`. -The most fundamental support consists of the types :data:`Any`, :data:`Union`, -:data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and -:class:`Generic`. For full specification please see :pep:`484`. For -a simplified introduction to type hints see :pep:`483`. +This module provides runtime support for type hints. The most fundamental +support consists of the types :data:`Any`, :data:`Union`, :data:`Tuple`, +:data:`Callable`, :class:`TypeVar`, and :class:`Generic`. For a full +specification, please see :pep:`484`. For a simplified introduction to type +hints, see :pep:`483`. The function below takes and returns a string and is annotated as follows:: @@ -34,6 +33,33 @@ In the function ``greeting``, the argument ``name`` is expected to be of type :class:`str` and the return type :class:`str`. Subtypes are accepted as arguments. +.. _relevant-peps: + +Relevant PEPs +============= + +Since the initial introduction of type hints in :pep:`484` and :pep:`483`, a +number of PEPs have modified and enhanced Python's framework for type +annotations. These include: + +* :pep:`526`: Syntax for Variable Annotations + *Introducing* syntax for annotating variables outside of function + definitions, and :data:`ClassVar` +* :pep:`544`: Protocols: Structural subtyping (static duck typing) + *Introducing* :class:`Protocol` and the + :func:`@runtime_checkable<runtime_checkable>` decorator +* :pep:`585`: Type Hinting Generics In Standard Collections + *Introducing* the ability to use builtin collections and ABCs as + :term:`generic types<generic type>` +* :pep:`586`: Literal Types + *Introducing* :class:`Literal` +* :pep:`589`: TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys + *Introducing* :class:`TypedDict` +* :pep:`591`: Adding a final qualifier to typing + *Introducing* :data:`Final` and the :func:`@final<final>` decorator +* :pep:`593`: Flexible function and variable annotations + *Introducing* :class:`Annotated` + Type aliases ============ diff --git a/Misc/NEWS.d/next/Documentation/2021-10-28-21-28-07.bpo-45655.aPYGaS.rst b/Misc/NEWS.d/next/Documentation/2021-10-28-21-28-07.bpo-45655.aPYGaS.rst new file mode 100644 index 0000000000000..fc5b3d0788817 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2021-10-28-21-28-07.bpo-45655.aPYGaS.rst @@ -0,0 +1,2 @@ +Add a new "relevant PEPs" section to the top of the documentation for the +``typing`` module. Patch by Alex Waygood.

1 0