Mailman 3 August 2022 - Python-checkins

gh-95337: update TypeVarTuple example (GH-95338)
by miss-islington Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/d00a9e0176e9a1e2dd213a137757da36e0… commit: d00a9e0176e9a1e2dd213a137757da36e071bd81 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-30T08:06:19-07:00 summary: gh-95337: update TypeVarTuple example (GH-95338) Co-authored-by: Alex Waygood <Alex.Waygood(a)Gmail.com> (cherry picked from commit 07f12b5c1567581aa77d523e462b0e7f75c1f05c) Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb(a)users.noreply.github.com> files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 0951916f080..76394066431 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1305,20 +1305,25 @@ These are not used in annotations. They are building blocks for creating generic T = TypeVar('T') Ts = TypeVarTuple('Ts') - def remove_first_element(tup: tuple[T, *Ts]) -> tuple[*Ts]: - return tup[1:] + def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: + return (*tup[1:], tup[0]) # T is bound to int, Ts is bound to () - # Return value is (), which has type tuple[()] - remove_first_element(tup=(1,)) + # Return value is (1,), which has type tuple[int] + move_first_element_to_last(tup=(1,)) # T is bound to int, Ts is bound to (str,) - # Return value is ('spam',), which has type tuple[str] - remove_first_element(tup=(1, 'spam')) + # Return value is ('spam', 1), which has type tuple[str, int] + move_first_element_to_last(tup=(1, 'spam')) # T is bound to int, Ts is bound to (str, float) - # Return value is ('spam', 3.0), which has type tuple[str, float] - remove_first_element(tup=(1, 'spam', 3.0)) + # Return value is ('spam', 3.0, 1), which has type tuple[str, float, int] + move_first_element_to_last(tup=(1, 'spam', 3.0)) + + # This fails to type check (and fails at runtime) + # because tuple[()] is not compatible with tuple[T, *Ts] + # (at least one element is required) + move_first_element_to_last(tup=()) Note the use of the unpacking operator ``*`` in ``tuple[T, *Ts]``. Conceptually, you can think of ``Ts`` as a tuple of type variables

1 0

gh-95337: update TypeVarTuple example (#95338)
by JelleZijlstra Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/07f12b5c1567581aa77d523e462b0e7f75… commit: 07f12b5c1567581aa77d523e462b0e7f75c1f05c branch: main author: Adrian Garcia Badaracco <1755071+adriangb(a)users.noreply.github.com> committer: JelleZijlstra <jelle.zijlstra(a)gmail.com> date: 2022-08-30T07:57:03-07:00 summary: gh-95337: update TypeVarTuple example (#95338) Co-authored-by: Alex Waygood <Alex.Waygood(a)Gmail.com> files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c6dd6976f23..0939973cf24 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1305,20 +1305,25 @@ These are not used in annotations. They are building blocks for creating generic T = TypeVar('T') Ts = TypeVarTuple('Ts') - def remove_first_element(tup: tuple[T, *Ts]) -> tuple[*Ts]: - return tup[1:] + def move_first_element_to_last(tup: tuple[T, *Ts]) -> tuple[*Ts, T]: + return (*tup[1:], tup[0]) # T is bound to int, Ts is bound to () - # Return value is (), which has type tuple[()] - remove_first_element(tup=(1,)) + # Return value is (1,), which has type tuple[int] + move_first_element_to_last(tup=(1,)) # T is bound to int, Ts is bound to (str,) - # Return value is ('spam',), which has type tuple[str] - remove_first_element(tup=(1, 'spam')) + # Return value is ('spam', 1), which has type tuple[str, int] + move_first_element_to_last(tup=(1, 'spam')) # T is bound to int, Ts is bound to (str, float) - # Return value is ('spam', 3.0), which has type tuple[str, float] - remove_first_element(tup=(1, 'spam', 3.0)) + # Return value is ('spam', 3.0, 1), which has type tuple[str, float, int] + move_first_element_to_last(tup=(1, 'spam', 3.0)) + + # This fails to type check (and fails at runtime) + # because tuple[()] is not compatible with tuple[T, *Ts] + # (at least one element is required) + move_first_element_to_last(tup=()) Note the use of the unpacking operator ``*`` in ``tuple[T, *Ts]``. Conceptually, you can think of ``Ts`` as a tuple of type variables

1 0

gh-96349: fix minor performance regression initializing threading.Event (gh-96350)
by corona10 Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/22ed5233b76fe33f9767c6a5665608e7f3… commit: 22ed5233b76fe33f9767c6a5665608e7f398e7d7 branch: main author: Daniel Giger <danielg1111(a)verizon.net> committer: corona10 <donghee.na92(a)gmail.com> date: 2022-08-30T21:10:02+09:00 summary: gh-96349: fix minor performance regression initializing threading.Event (gh-96350) files: A Misc/NEWS.d/next/Library/2022-08-27-21-26-52.gh-issue-96349.XyYLlO.rst M Lib/threading.py diff --git a/Lib/threading.py b/Lib/threading.py index 7237a149ecc..d030e124362 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -262,18 +262,12 @@ def __init__(self, lock=None): # If the lock defines _release_save() and/or _acquire_restore(), # these override the default implementations (which just call # release() and acquire() on the lock). Ditto for _is_owned(). - try: + if hasattr(lock, '_release_save'): self._release_save = lock._release_save - except AttributeError: - pass - try: + if hasattr(lock, '_acquire_restore'): self._acquire_restore = lock._acquire_restore - except AttributeError: - pass - try: + if hasattr(lock, '_is_owned'): self._is_owned = lock._is_owned - except AttributeError: - pass self._waiters = _deque() def _at_fork_reinit(self): diff --git a/Misc/NEWS.d/next/Library/2022-08-27-21-26-52.gh-issue-96349.XyYLlO.rst b/Misc/NEWS.d/next/Library/2022-08-27-21-26-52.gh-issue-96349.XyYLlO.rst new file mode 100644 index 00000000000..59eb3517191 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-08-27-21-26-52.gh-issue-96349.XyYLlO.rst @@ -0,0 +1 @@ +Fixed a minor performance regression in :func:`threading.Event.__init__`

1 0

gh-95413: Remove references to deprecated CGI library (GH-95414)
by miss-islington Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/a1671a97d35dcec1df2152150fb0790b74… commit: a1671a97d35dcec1df2152150fb0790b74d84729 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-30T04:22:19-07:00 summary: gh-95413: Remove references to deprecated CGI library (GH-95414) (cherry picked from commit b17aae8bbd13bec28b7ecbb5a147503f2e9cf365) Co-authored-by: partev <petrosyan(a)gmail.com> files: M Doc/faq/general.rst M Doc/tutorial/whatnow.rst diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst index 510ebb5cf904..6c7e4fc67c0a 100644 --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -113,8 +113,8 @@ to many different classes of problems. The language comes with a large standard library that covers areas such as string processing (regular expressions, Unicode, calculating differences between -files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP, CGI -programming), software engineering (unit testing, logging, profiling, parsing +files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP), +software engineering (unit testing, logging, profiling, parsing Python code), and operating system interfaces (system calls, filesystems, TCP/IP sockets). Look at the table of contents for :ref:`library-index` to get an idea of what's available. A wide variety of third-party extensions are also diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 5f7010cb12e9..dbe2d7fc0992 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -17,7 +17,7 @@ the set are: reference material about types, functions, and the modules in the standard library. The standard Python distribution includes a *lot* of additional code. There are modules to read Unix mailboxes, retrieve documents via HTTP, generate - random numbers, parse command-line options, write CGI programs, compress data, + random numbers, parse command-line options, compress data, and many other tasks. Skimming through the Library Reference will give you an idea of what's available.

1 0

Docs: Improve clarity for bytes.hex() (GH-95257)
by miss-islington Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/d9d0d097a298132773e72a5a0a25eec196… commit: d9d0d097a298132773e72a5a0a25eec19645140e branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-30T04:18:30-07:00 summary: Docs: Improve clarity for bytes.hex() (GH-95257) (cherry picked from commit 860fa351452de1502da12ec6f027d3f72dfc309f) Co-authored-by: Tim Burke <tim.burke(a)gmail.com> files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2480e71dded..2c021866e29 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -2548,9 +2548,10 @@ data and are closely related to string objects in a variety of other ways. If you want to make the hex string easier to read, you can specify a single character separator *sep* parameter to include in the output. - By default between each byte. A second optional *bytes_per_sep* - parameter controls the spacing. Positive values calculate the - separator position from the right, negative values from the left. + By default, this separator will be included between each byte. + A second optional *bytes_per_sep* parameter controls the spacing. + Positive values calculate the separator position from the right, + negative values from the left. >>> value = b'\xf0\xf1\xf2' >>> value.hex('-')

1 0

Docs: Improve clarity for bytes.hex() (GH-95257)
by miss-islington Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/9389e2f08c9fcd998fc758a32d6e554a40… commit: 9389e2f08c9fcd998fc758a32d6e554a407bbc7c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-30T04:16:34-07:00 summary: Docs: Improve clarity for bytes.hex() (GH-95257) (cherry picked from commit 860fa351452de1502da12ec6f027d3f72dfc309f) Co-authored-by: Tim Burke <tim.burke(a)gmail.com> files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 5c0d259ba98..529dae871d3 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -2508,9 +2508,10 @@ data and are closely related to string objects in a variety of other ways. If you want to make the hex string easier to read, you can specify a single character separator *sep* parameter to include in the output. - By default between each byte. A second optional *bytes_per_sep* - parameter controls the spacing. Positive values calculate the - separator position from the right, negative values from the left. + By default, this separator will be included between each byte. + A second optional *bytes_per_sep* parameter controls the spacing. + Positive values calculate the separator position from the right, + negative values from the left. >>> value = b'\xf0\xf1\xf2' >>> value.hex('-')

1 0

gh-95413: Remove references to deprecated CGI library (#95414)
by erlend-aasland Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/b17aae8bbd13bec28b7ecbb5a147503f2e… commit: b17aae8bbd13bec28b7ecbb5a147503f2e9cf365 branch: main author: partev <petrosyan(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2022-08-30T13:14:08+02:00 summary: gh-95413: Remove references to deprecated CGI library (#95414) files: M Doc/faq/general.rst M Doc/tutorial/whatnow.rst diff --git a/Doc/faq/general.rst b/Doc/faq/general.rst index 510ebb5cf904..6c7e4fc67c0a 100644 --- a/Doc/faq/general.rst +++ b/Doc/faq/general.rst @@ -113,8 +113,8 @@ to many different classes of problems. The language comes with a large standard library that covers areas such as string processing (regular expressions, Unicode, calculating differences between -files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP, CGI -programming), software engineering (unit testing, logging, profiling, parsing +files), internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP), +software engineering (unit testing, logging, profiling, parsing Python code), and operating system interfaces (system calls, filesystems, TCP/IP sockets). Look at the table of contents for :ref:`library-index` to get an idea of what's available. A wide variety of third-party extensions are also diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 5f7010cb12e9..dbe2d7fc0992 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -17,7 +17,7 @@ the set are: reference material about types, functions, and the modules in the standard library. The standard Python distribution includes a *lot* of additional code. There are modules to read Unix mailboxes, retrieve documents via HTTP, generate - random numbers, parse command-line options, write CGI programs, compress data, + random numbers, parse command-line options, compress data, and many other tasks. Skimming through the Library Reference will give you an idea of what's available.

1 0

Docs: Improve clarity for bytes.hex() (#95257)
by erlend-aasland Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/860fa351452de1502da12ec6f027d3f72d… commit: 860fa351452de1502da12ec6f027d3f72dfc309f branch: main author: Tim Burke <tim.burke(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2022-08-30T13:09:56+02:00 summary: Docs: Improve clarity for bytes.hex() (#95257) files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2480e71dded..2c021866e29 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -2548,9 +2548,10 @@ data and are closely related to string objects in a variety of other ways. If you want to make the hex string easier to read, you can specify a single character separator *sep* parameter to include in the output. - By default between each byte. A second optional *bytes_per_sep* - parameter controls the spacing. Positive values calculate the - separator position from the right, negative values from the left. + By default, this separator will be included between each byte. + A second optional *bytes_per_sep* parameter controls the spacing. + Positive values calculate the separator position from the right, + negative values from the left. >>> value = b'\xf0\xf1\xf2' >>> value.hex('-')

1 0

GH-74116: Allow multiple drain waiters for asyncio.StreamWriter (GH-94705) (#96395)
by pablogsal Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/2e9f29e6a660a33f84d64a5bce5b422e72… commit: 2e9f29e6a660a33f84d64a5bce5b422e72c06e39 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2022-08-30T12:00:21+01:00 summary: GH-74116: Allow multiple drain waiters for asyncio.StreamWriter (GH-94705) (#96395) (cherry picked from commit e5b2453e61ba5376831093236d598ef5f9f1de61) Co-authored-by: Kumar Aditya <59607654+kumaraditya303(a)users.noreply.github.com> Co-authored-by: Kumar Aditya <59607654+kumaraditya303(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-07-09-08-55-04.gh-issue-74116.0XwYC1.rst M Lib/asyncio/streams.py M Lib/test/test_asyncio/test_streams.py diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index af78e6b45881..d21a31d1860d 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -2,6 +2,7 @@ 'StreamReader', 'StreamWriter', 'StreamReaderProtocol', 'open_connection', 'start_server') +import collections import socket import sys import warnings @@ -129,7 +130,7 @@ def __init__(self, loop=None): else: self._loop = loop self._paused = False - self._drain_waiter = None + self._drain_waiters = collections.deque() self._connection_lost = False def pause_writing(self): @@ -144,38 +145,34 @@ def resume_writing(self): if self._loop.get_debug(): logger.debug("%r resumes writing", self) - waiter = self._drain_waiter - if waiter is not None: - self._drain_waiter = None + for waiter in self._drain_waiters: if not waiter.done(): waiter.set_result(None) def connection_lost(self, exc): self._connection_lost = True - # Wake up the writer if currently paused. + # Wake up the writer(s) if currently paused. if not self._paused: return - waiter = self._drain_waiter - if waiter is None: - return - self._drain_waiter = None - if waiter.done(): - return - if exc is None: - waiter.set_result(None) - else: - waiter.set_exception(exc) + + for waiter in self._drain_waiters: + if not waiter.done(): + if exc is None: + waiter.set_result(None) + else: + waiter.set_exception(exc) async def _drain_helper(self): if self._connection_lost: raise ConnectionResetError('Connection lost') if not self._paused: return - waiter = self._drain_waiter - assert waiter is None or waiter.cancelled() waiter = self._loop.create_future() - self._drain_waiter = waiter - await waiter + self._drain_waiters.append(waiter) + try: + await waiter + finally: + self._drain_waiters.remove(waiter) def _get_close_waiter(self, stream): raise NotImplementedError diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py index 098a0da344d0..0c49099bc499 100644 --- a/Lib/test/test_asyncio/test_streams.py +++ b/Lib/test/test_asyncio/test_streams.py @@ -864,6 +864,25 @@ def test_streamreaderprotocol_constructor_use_global_loop(self): self.assertEqual(cm.filename, __file__) self.assertIs(protocol._loop, self.loop) + def test_multiple_drain(self): + # See https://github.com/python/cpython/issues/74116 + drained = 0 + + async def drainer(stream): + nonlocal drained + await stream._drain_helper() + drained += 1 + + async def main(): + loop = asyncio.get_running_loop() + stream = asyncio.streams.FlowControlMixin(loop) + stream.pause_writing() + loop.call_later(0.1, stream.resume_writing) + await asyncio.gather(*[drainer(stream) for _ in range(10)]) + self.assertEqual(drained, 10) + + self.loop.run_until_complete(main()) + def test_drain_raises(self): # See http://bugs.python.org/issue25441 diff --git a/Misc/NEWS.d/next/Library/2022-07-09-08-55-04.gh-issue-74116.0XwYC1.rst b/Misc/NEWS.d/next/Library/2022-07-09-08-55-04.gh-issue-74116.0XwYC1.rst new file mode 100644 index 000000000000..33782598745b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-07-09-08-55-04.gh-issue-74116.0XwYC1.rst @@ -0,0 +1 @@ +Allow :meth:`asyncio.StreamWriter.drain` to be awaited concurrently by multiple tasks. Patch by Kumar Aditya.

1 0

gh-90467: StreamReaderProtocol - add strong reference to created task (GH-96323) (#96344)
by pablogsal Aug. 30, 2022

Aug. 30, 2022

https://github.com/python/cpython/commit/126ec34558ef9e2fd688cc85b0951b559c… commit: 126ec34558ef9e2fd688cc85b0951b559ce6a889 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2022-08-30T11:59:32+01:00 summary: gh-90467: StreamReaderProtocol - add strong reference to created task (GH-96323) (#96344) (cherry picked from commit e860e521ec0d84e175269aeb15cf24bd6053ad17) Co-authored-by: Kirill <iam(a)python273.pw> Co-authored-by: Kirill <iam(a)python273.pw> files: A Misc/NEWS.d/next/Library/2022-08-27-14-38-49.gh-issue-90467.VOOB0p.rst M Lib/asyncio/streams.py M Misc/ACKS diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py index a568c4e4b295..af78e6b45881 100644 --- a/Lib/asyncio/streams.py +++ b/Lib/asyncio/streams.py @@ -206,6 +206,7 @@ def __init__(self, stream_reader, client_connected_cb=None, loop=None): self._strong_reader = stream_reader self._reject_connection = False self._stream_writer = None + self._task = None self._transport = None self._client_connected_cb = client_connected_cb self._over_ssl = False @@ -248,7 +249,7 @@ def connection_made(self, transport): res = self._client_connected_cb(reader, self._stream_writer) if coroutines.iscoroutine(res): - self._loop.create_task(res) + self._task = self._loop.create_task(res) self._strong_reader = None def connection_lost(self, exc): @@ -266,6 +267,7 @@ def connection_lost(self, exc): super().connection_lost(exc) self._stream_reader_wr = None self._stream_writer = None + self._task = None self._transport = None def data_received(self, data): diff --git a/Misc/ACKS b/Misc/ACKS index 6e53d5a7cc4b..58b29c19400e 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1438,6 +1438,7 @@ Ram Rachum Jeffrey Rackauckas Jérôme Radix Burton Radons +Kirill (python273) R. Abhilash Raj Shorya Raj Ajith Ramachandran @@ -1981,6 +1982,7 @@ Gordon Worley Darren Worrall Thomas Wouters Daniel Wozniak +Simon Wrede Marcin Niemira Wei Wu Heiko Wundram diff --git a/Misc/NEWS.d/next/Library/2022-08-27-14-38-49.gh-issue-90467.VOOB0p.rst b/Misc/NEWS.d/next/Library/2022-08-27-14-38-49.gh-issue-90467.VOOB0p.rst new file mode 100644 index 000000000000..282c0e76a8c8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-08-27-14-38-49.gh-issue-90467.VOOB0p.rst @@ -0,0 +1,2 @@ +Fix :class:`asyncio.streams.StreamReaderProtocol` to keep a strong reference +to the created task, so that it's not garbage collected

1 0