Mailman 3 November 2017 - Python-Dev

PEPs: ``.. code:: python`` or ``::`` (syntax highlighting)
by Wes Turner 03 Dec '17

03 Dec '17

In ReStructuredText, this gets syntax highlighted because of the code directive [1][2][3]: .. code:: python import this def func(*args, **kwargs): pass This also gets syntax highlighted as python[3]: .. code:: python import this def func(*args, **kwargs): pass This does not:: import this def func(*args, **kwargs): pass Syntax highlighting in Docutils 0.9+ is powered by Pygments. If Pygments is not installed, or there is a syntax error, syntax highlighting is absent. GitHub does show Pygments syntax highlighting in .. code:: blocks for .rst and .restructuredtext documents [4] 1. Does the python.org PEP view support .. code:: blocks? [5] 2. Syntax highlighting is an advantage for writers, editors, and readers. 3. Should PEPs use .. code:: blocks to provide this advantage? [1] http://docutils.sourceforge.net/docs/ref/rst/directives.html#code [2] http://www.sphinx-doc.org/en/stable/markup/code.html [3] http://www.sphinx-doc.org/en/stable/config.html#confval-highlight_language [4] https://github.com/python/peps/blob/master/pep-0557.rst [5] https://www.python.org/dev/peps/pep-0557/ https://www.python.org/dev/peps/pep-0458/

4 3

Third and hopefully final post: PEP 557, Data Classes
by Eric V. Smith 03 Dec '17

03 Dec '17

I've posted a new version of PEP 557, it should soon be available at https://www.python.org/dev/peps/pep-0557/. The only significant changes since the last version are: - changing the "compare" parameter to be "order", since that more accurately reflects what it does. - Having the combination of "eq=False" and "order=True" raise an exception instead of silently changing eq to True. There were no other issues raised with the previous version of the PEP. So with that, I think it's ready for a pronouncement. Eric.

10 21

Allow tuple unpacking in return and yield statements
by David Cuthbert 01 Dec '17

01 Dec '17

First time contributing back -- if I should be filing a PEP or something like that for this, please let me know. Coming from https://bugs.python.org/issue32117, unparenthesized tuple unpacking is allowed in assignments: rest = (4, 5, 6) a = 1, 2, 3, *rest but not in yield or return statements (these result in SyntaxErrors): return 1, 2, 3, *rest yield 1, 2, 3, *rest The unpacking in assignments was enabled by a pre-3.2 commit that I haven't yet been able to track back to a discussion, but I suspect this asymmetry is unintentional. Here's the original commit: https://github.com/python/cpython/commit/4905e80c3d2f6abb613d212f0313d1dfe0… I've submitted a patch (CLA is signed and submitted, not yet processed), and Serihy said that since it changes the grammar I should have it reviewed here and have signoff by the BDFL. While I haven't had a need for this myself, it was brought up by a user on StackOverflow (https://stackoverflow.com/questions/47272460/python-tuple-unpacking-in-retu…). Thanks! Dave

6 6

Py_DECREF(m) on PyInit_xxx() failure?
by Victor Stinner 30 Nov '17

30 Nov '17

Hi, CPython has many C extensions with non-trivial PyInit_xxx() functions which has to handle failures. A few modules use "Py_DECREF(m); retutrn NULL;", but most functions only do "return NULL;". Is it a reference leak or not? Example from Modules/posixmodule.c: v = convertenviron(); Py_XINCREF(v); if (v == NULL || PyModule_AddObject(m, "environ", v) != 0) return NULL; Py_DECREF(v); Does this code leak a reference on m? ... Oh, and maybe also leaks a reference on v? Victor

2 1

Removing files from the repository
by Serhiy Storchaka 29 Nov '17

29 Nov '17

After removing files from the repository they disappear from the source tree, and it is even hard to notice this if you don't use it regularly. It is hard to track the history of the removed file even if you know it exact path. If you know it only approximate this is harder. I think that any file removals from the repository should pass some PEP-like process. Declaring the intention with the rationale, taking a feedback, discussing, and finally documenting the removal. Perhaps it is worth to track all removals in a special file, so if later you will find that the removed file can be useful you could restore it instead of recreating its functionality from zero in the case if you even don't know that similar file existed.

6 8

Second post: PEP 557, Data Classes
by Eric V. Smith 29 Nov '17

29 Nov '17

The updated version should show up at https://www.python.org/dev/peps/pep-0557/ shortly. The major changes from the previous version are: - Add InitVar to specify initialize-only fields. - Renamed __dataclass_post_init__() to __post_init(). - Rename cmp to compare. - Added eq, separate from compare, so you can test unorderable items for equality. - Flushed out asdict() and astuple(). - Changed replace() to just call __init__(), and dropped the complex post-create logic. The only open issues I know of are: - Should object comparison require an exact match on the type? https://github.com/ericvsmith/dataclasses/issues/51 - Should the replace() function be renamed to something else? https://github.com/ericvsmith/dataclasses/issues/77 Most of the items that were previously discussed on python-dev were discussed in detail at https://github.com/ericvsmith/dataclasses. Before rehashing an old discussion, please check there first. Also at https://github.com/ericvsmith/dataclasses is an implementation, with tests, that should work with 3.6 and 3.7. The only action item for the code is to clean up the implementation of InitVar, but that's waiting for PEP 560. Oh, and if PEP 563 is accepted I'll also need to do some work. Feedback is welcomed! Eric.

7 29

Can Python guarantee the order of keyword-only parameters?
by Larry Hastings 29 Nov '17

29 Nov '17

First, a thirty-second refresher, so we're all using the same terminology: A *parameter* is a declared input variable to a function. An *argument* is a value passed into a function. (*Arguments* are stored in *parameters.*) So in the example "def foo(clonk): pass; foo(3)", clonk is a parameter, and 3 is an argument. ++ Keyword-only arguments were conceived of as being unordered. They're stored in a dictionary--by convention called **kwargs--and dictionaries didn't preserve order. But knowing the order of arguments is occasionally very useful. PEP 468 proposed that Python preserve the order of keyword-only arguments in kwargs. This became easy with the order-preserving dictionaries added to Python 3.6. I don't recall the order of events, but in the end PEP 468 was accepted, and as of 3.6 Python guarantees order in **kwargs. But that's arguments. What about parameters? Although this isn't as directly impactful, the order of keyword-only parameters *is* visible to the programmer. The best way to see a function's parameters is with inspect.signature, although there's also the deprecated inspect.getfullargspec; in CPython you can also directly examine fn.__code__.co_varnames. Two of these methods present their data in a way that preserves order for all parameters, including keyword-only parameters--and the third one is deprecated. Python must (and does) guarantee the order of positional and positional-or-keyword parameters, because it uses position to map arguments to parameters when the function is called. But conceptually this isn't necessary for keyword-only parameters because their position is irrelevant. I only see one place in the language & library that addresses the ordering of keyword-only parameters, by way of omission. The PEP for inspect.signature (PEP 362) says that when comparing two signatures for equality, their positional and positional-or-keyword parameters must be in the same order. It makes a point of *not* requiring that the two functions' keyword-only parameters be in the same order. For every currently supported version of Python 3, inspect.signature and fn.__code__.co_varnames preserve the order of keyword-only parameters. This isn't surprising; it's basically the same code path implementing those as the two types of positional-relevant parameters, so the most straightforward implementation would naturally preserve their order. It's just not guaranteed. I'd like inspect.signature to guarantee that the order of keyword-only parameters always matches the order they were declared in. Technically this isn't a language feature, it's a library feature. But making this guarantee would require that CPython internally cooperate, so it's kind of a language feature too. Does this sound reasonable? Would it need a PEP? I'm hoping for "yes" and "no", respectively. Three final notes: * Yes, I do have a use case. I'm using inspect.signature metadata to mechanically map arguments from an external domain (command-line arguments) to a Python function. Relying on the declaration order of keyword-only parameters would elegantly solve one small problem. * I asked Armin Rigo about PyPy's support for Python 3. He said it should already maintain the order of keyword-only parameters, and if I ever catch it not maintaining them in order I should file a bug. I assert that making this guarantee would be nearly zero effort for any Python implementation--I bet they all already behave this way, all they need is a test case and some documentation. * One can extend this concept to functools.partial and inspect.Signature.bind: should its transformations of keyword-only parameters also maintain order in a consistent way? I suspect the answer there is much the same--there's an obvious way it should behave, it almost certainly already behaves that way, but it doesn't guarantee it. I don't think I need this for my use case. //arry/ ++ Yes, that means "Argument Clinic" should really have been called "Parameter Clinic". But the "Parameter Clinic" sketch is nowhere near as funny.

8 10

Regular expressions: splitting on zero-width patterns
by Serhiy Storchaka 28 Nov '17

28 Nov '17

The two largest problems in the re module are splitting on zero-width patterns and complete and correct support of the Unicode standard. These problems are solved in regex. regex has many other features, but they are less important. I want to tell the problem of splitting on zero-width patterns. It already was discussed on Python-Dev 13 years ago [3] and maybe later. See also issues: [4], [5], [6], [7], [8]. In short it doesn't work. Splitting on the pattern r'\b' doesn't split the text at boundaries of words, and splitting on the pattern r'\s+|(?<=-)' will split the text on whitespaces, but will not split words with hypens as expected. In Python 3.4 and earlier: >>> re.split(r'\b', 'Self-Defence Class') ['Self-Defence Class'] >>> re.split(r'\s+|(?<=-)', 'Self-Defence Class') ['Self-Defence', 'Class'] >>> re.split(r'\s*', 'Self-Defence Class') ['Self-Defence', 'Class'] Note that splitting on r'\s*' (0 or more whitespaces) actually split on r'\s+' (1 or more whitespaces). Splitting on patterns that only can match the empty string (like r'\b' or r'(?<=-)') never worked, while splitting Starting since Python 3.5 splitting on a pattern that only can match the empty string raises a ValueError (this never worked), and splitting a pattern that can match the empty string but not only emits a FutureWarning. This taken developers a time for replacing their patterns r'\s*' to r'\s+' as they should be. Now I have created a final patch [9] that makes re.split() splitting on zero-width patterns. >>> re.split(r'\b', 'Self-Defence Class') ['', 'Self', '-', 'Defence', ' ', 'Class', ''] >>> re.split(r'\s+|(?<=-)', 'Self-Defence Class') ['Self-', 'Defence', 'Class'] >>> re.split(r'\s*', 'Self-Defence Class') ['', 'S', 'e', 'l', 'f', '-', 'D', 'e', 'f', 'e', 'n', 'c', 'e', 'C', 'l', 'a', 's', 's', ''] The latter case the result is differ too much from the previous result, and this likely not what the author wanted to get. But users had two Python releases for fixing their code. FutureWarning is not silent by default. Because these patterns produced errors or warnings in the recent two releases, we don't need an additional parameter for compatibility. But the problem was not just with re.split(). Other functions also worked not good with patterns that can match the empty string. >>> re.findall(r'^|\w+', 'Self-Defence Class') ['', 'elf', 'Defence', 'Class'] >>> list(re.finditer(r'^|\w+', 'Self-Defence Class')) [<re.Match object; span=(0, 0), match=''>, <re.Match object; span=(1, 4), match='elf'>, <re.Match object; span=(5, 12), match='Defence'>, <re.Match object; span=(13, 18), match='Class'>] >>> re.sub(r'(^|\w+)', r'<\1>', 'Self-Defence Class') '<>S<elf>-<Defence> <Class>' After matching the empty string the following character will be skipped and will be not included in the next match. My patch fixes these functions too. >>> re.findall(r'^|\w+', 'Self-Defence Class') ['', 'Self', 'Defence', 'Class'] >>> list(re.finditer(r'^|\w+', 'Self-Defence Class')) [<re.Match object; span=(0, 0), match=''>, <re.Match object; span=(0, 4), match='Self'>, <re.Match object; span=(5, 12), match='Defence'>, <re.Match object; span=(13, 18), match='Class'>] >>> re.sub(r'(^|\w+)', r'<\1>', 'Self-Defence Class') '<><Self>-<Defence> <Class>' I think this change don't need preliminary warnings, because it change the behavior of more rarely used patterns. No re tests have been broken. I was needed to add new tests for detecting the behavior change. But there is one spoonful of tar in a barrel of honey. I didn't expect this, but this change have broken a pattern used with re.sub() in the doctest module: r'(?m)^\s*?$'. This was fixed by replacing it with r'(?m)^[^\S\n]+?$'). I hope that such cases are pretty rare and think this is an avoidable breakage. The new behavior of re.split() matches the behavior of regex.split() with the VERSION1 flag, the new behavior of re.findall() and re.finditer() matches the behavior of corresponding functions in the regex module (independently from the version flag). But the new behavior of re.sub() doesn't match exactly the behavior of regex.sub() with any version flag. It differs from the old behavior as you can see in the example above, but is closer to it that to regex.sub() with VERSION1. This allowed to avoid braking existing tests for re.sub(). >>> regex.sub(r'(\W+|(?<=-))', r':', 'Self-Defence Class') 'Self:Defence:Class' >>> regex.sub(r'(?V1)(\W+|(?<=-))', r':', 'Self-Defence Class') 'Self::Defence:Class' >>> re.sub(r'(\W+|(?<=-))', r':', 'Self-Defence Class') 'Self:Defence:Class' As re.split() it never matches the empty string adjacent to the previous match. re.findall() and re.finditer() only don't match the empty string adjacent to the previous empty string match. In the regex module regex.sub() is mutually consistent with regex.findall() and regex.finditer() (with the VERSION1 flag), but regex.split() is not consistent with them. In the re module re.split() and re.sub() will be mutually consistent, as well as re.findall() and re.finditer(). This is more backward compatible. And I don't know reasons for preferring the behavior of re.findall() and re.finditer() over the behavior of re.split() in this corner case. Would be nice to get this change in 3.7.0a3 for wider testing. Please make a review of the patch [9] or tell your thoughts about this change. [1] https://docs.python.org/3/library/re.html [2] https://pypi.python.org/pypi/regex/ [3] https://mail.python.org/pipermail/python-dev/2004-August/047272.html [4] https://bugs.python.org/issue852532 [5] https://bugs.python.org/issue988761 [6] https://bugs.python.org/issue1647489 [7] https://bugs.python.org/issue3262 [8] https://bugs.python.org/issue25054 [9] https://github.com/python/cpython/pull/4471

3 5

PEP 565: show DeprecationWarning in __main__ (round 2)
by Nick Coghlan 28 Nov '17

28 Nov '17

This is a new version of the proposal to show DeprecationWarning in __main__. The proposal itself hasn't changed (it's still recommending a new entry in the default filter list), but there have been several updates to the PEP text based on further development work and comments in the initial thread: - there's now a linked issue and reference implementation - it turns out we don't currently support the definition of module based filters at startup time, so I've explicitly noted the relevant enhancement that turned out to be necessary (allowing plain-string-or-compiled-regex in stored filter definitions where we currently only allow compiled regexes) - I've noted the intended changes to the warnings-related documentation - I've noted a couple of other relevant changes that Victor already implemented for 3.7 - I've noted that the motivation for the change in 2.7 & 3.1 covered all Python applications, not just developer tools (developer tools just provide a particularly compelling example of why "revert to the Python 2.6 behaviour" isn't a good answer) Cheers, Nick. ================= PEP: 565 Title: Show DeprecationWarning in __main__ Author: Nick Coghlan <ncoghlan(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Nov-2017 Python-Version: 3.7 Post-History: 12-Nov-2017, 25-Nov-2017 Abstract ======== In Python 2.7 and Python 3.2, the default warning filters were updated to hide DeprecationWarning by default, such that deprecation warnings in development tools that were themselves written in Python (e.g. linters, static analysers, test runners, code generators), as well as any other applications that merely happened to be written in Python, wouldn't be visible to their users unless those users explicitly opted in to seeing them. However, this change has had the unfortunate side effect of making DeprecationWarning markedly less effective at its primary intended purpose: providing advance notice of breaking changes in APIs (whether in CPython, the standard library, or in third party libraries) to users of those APIs. To improve this situation, this PEP proposes a single adjustment to the default warnings filter: displaying deprecation warnings attributed to the main module by default. This change will mean that code entered at the interactive prompt and code in single file scripts will revert to reporting these warnings by default, while they will continue to be silenced by default for packaged code distributed as part of an importable module. The PEP also proposes a number of small adjustments to the reference interpreter and standard library documentation to help make the warnings subsystem more approachable for new Python developers. Specification ============= The current set of default warnings filters consists of:: ignore::DeprecationWarning ignore::PendingDeprecationWarning ignore::ImportWarning ignore::BytesWarning ignore::ResourceWarning The default ``unittest`` test runner then uses ``warnings.catch_warnings()`` ``warnings.simplefilter('default')`` to override the default filters while running test cases. The change proposed in this PEP is to update the default warning filter list to be:: default::DeprecationWarning:__main__ ignore::DeprecationWarning ignore::PendingDeprecationWarning ignore::ImportWarning ignore::BytesWarning ignore::ResourceWarning This means that in cases where the nominal location of the warning (as determined by the ``stacklevel`` parameter to ``warnings.warn``) is in the ``__main__`` module, the first occurrence of each DeprecationWarning will once again be reported. This change will lead to DeprecationWarning being displayed by default for: * code executed directly at the interactive prompt * code executed directly as part of a single-file script While continuing to be hidden by default for: * code imported from another module in a ``zipapp`` archive's ``__main__.py`` file * code imported from another module in an executable package's ``__main__`` submodule * code imported from an executable script wrapper generated at installation time based on a ``console_scripts`` or ``gui_scripts`` entry point definition As a result, API deprecation warnings encountered by development tools written in Python should continue to be hidden by default for users of those tools While not its originally intended purpose, the standard library documentation will also be updated to explicitly recommend the use of ``FutureWarning`` (rather than ``DeprecationWarning``) for backwards compatibility warnings that are intended to be seen by *users* of an application. This will give the following three distinct categories of backwards compatibility warning, with three different intended audiences: * ``PendingDeprecationWarning``: reported by default only in test runners that override the default set of warning filters. The intended audience is Python developers that take an active interest in ensuring the future compatibility of their software (e.g. professional Python application developers with specific support obligations). * ``DeprecationWarning``: reported by default for code that runs directly in the ``__main__`` module (as such code is considered relatively unlikely to have a dedicated test suite), but relies on test suite based reporting for code in other modules. The intended audience is Python developers that are at risk of upgrades to their dependencies (including upgrades to Python itself) breaking their software (e.g. developers using Python to script environments where someone else is in control of the timing of dependency upgrades). * ``FutureWarning``: always reported by default. The intended audience is users of applications written in Python, rather than other Python developers (e.g. warning about use of a deprecated setting in a configuration file format). Given its presence in the standard library since Python 2.3, ``FutureWarning`` would then also have a secondary use case for libraries and frameworks that support multiple Python versions: as a more reliably visible alternative to ``DeprecationWarning`` in Python 2.7 and versions of Python 3.x prior to 3.7. Documentation Updates ===================== The current reference documentation for the warnings system is relatively short on specific *examples* of possible settings for the ``-W`` command line option or the ``PYTHONWARNINGS`` environment variably that achieve particular end results. The following improvements are proposed as part of the implementation of this PEP: * Explicitly list the following entries under the description of the ``PYTHONWARNINGS`` environment variable:: PYTHONWARNINGS=error # Convert to exceptions PYTHONWARNINGS=always # Warn every time PYTHONWARNINGS=default # Warn once per call location PYTHONWARNINGS=module # Warn once per calling module PYTHONWARNINGS=once # Warn once per Python process PYTHONWARNINGS=ignore # Never warn * Explicitly list the corresponding short options (``-We``, ``-Wa``, ``-Wd``, ``-Wm``,``-Wo``, ``-Wi``) for each of the warning actions listed under the ``-W`` command line switch documentation * Explicitly list the default filter set in the ``warnings`` module documentation, using the ``action::category`` and ``action::category:module`` notation * Explicitly list the following snippet in the ``warnings.simplefilter`` documentation as a recommended approach to turning off all warnings by default in a Python application while still allowing them to be turned back on via ``PYTHONWARNINGS`` or the ``-W`` command line switch:: if not sys.warnoptions: warnings.simplefilter("ignore") None of these are *new* (they already work in all still supported Python versions), but they're not especially obvious given the current structure of the related documentation. Reference Implementation ======================== A reference implementation is available in the PR [4_] linked from the related tracker issue for this PEP [5_]. As a side-effect of implementing this PEP, the internal warnings filter list will start allowing the use of plain strings as part of filter definitions (in addition to the existing use of compiled regular expressions). When present, the plain strings will be compared for exact matches only. This approach allows the new default filter to be added during interpreter startup without requiring early access to the ``re`` module. Motivation ========== As discussed in [1_] and mentioned in [2_], Python 2.7 and Python 3.2 changed the default handling of ``DeprecationWarning`` such that: * the warning was hidden by default during normal code execution * the ``unittest`` test runner was updated to re-enable it when running tests The intent was to avoid cases of tooling output like the following:: $ devtool mycode/ /usr/lib/python3.6/site-packages/devtool/cli.py:1: DeprecationWarning: 'async' and 'await' will become reserved keywords in Python 3.7 async = True ... actual tool output ... Even when `devtool` is a tool specifically for Python programmers, this is not a particularly useful warning, as it will be shown on every invocation, even though the main helpful step an end user can take is to report a bug to the developers of ``devtool``. The warning is even less helpful for general purpose developer tools that are used across more languages than just Python, and almost entirely \*un\*helpful for applications that simply happen to be written in Python, and aren't necessarily intended for a developer audience at all. However, this change proved to have unintended consequences for the following audiences: * anyone using a test runner other than the default one built into ``unittest`` (the request for third party test runners to change their default warnings filters was never made explicitly, so many of them still rely on the interpreter defaults that are designed to suit deployed applications) * anyone using the default ``unittest`` test runner to test their Python code in a subprocess (since even ``unittest`` only adjusts the warnings settings in the current process) * anyone writing Python code at the interactive prompt or as part of a directly executed script that didn't have a Python level test suite at all In these cases, ``DeprecationWarning`` ended up become almost entirely equivalent to ``PendingDeprecationWarning``: it was simply never seen at all. Limitations on PEP Scope ======================== This PEP exists specifically to explain both the proposed addition to the default warnings filter for 3.7, *and* to more clearly articulate the rationale for the original change to the handling of DeprecationWarning back in Python 2.7 and 3.2. This PEP does not solve all known problems with the current approach to handling deprecation warnings. Most notably: * the default ``unittest`` test runner does not currently report deprecation warnings emitted at module import time, as the warnings filter override is only put in place during test execution, not during test discovery and loading. * the default ``unittest`` test runner does not currently report deprecation warnings in subprocesses, as the warnings filter override is applied directly to the loaded ``warnings`` module, not to the ``PYTHONWARNINGS`` environment variable. * the standard library doesn't provide a straightforward way to opt-in to seeing all warnings emitted *by* a particular dependency prior to upgrading it (the third-party ``warn`` module [3_] does provide this, but enabling it involves monkeypatching the standard library's ``warnings`` module). * re-enabling deprecation warnings by default in __main__ doesn't help in handling cases where software has been factored out into support modules, but those modules still have little or no automated test coverage. Near term, the best currently available answer is to run such applications with ``PYTHONWARNINGS=default::DeprecationWarning`` or ``python -W default::DeprecationWarning`` and pay attention to their ``stderr`` output. Longer term, this is really a question for researchers working on static analysis of Python code: how to reliably find usage of deprecated APIs, and how to infer that an API or parameter is deprecated based on ``warnings.warn`` calls, without actually running either the code providing the API or the code accessing it While these are real problems with the status quo, they're excluded from consideration in this PEP because they're going to require more complex solutions than a single additional entry in the default warnings filter, and resolving them at least potentially won't require going through the PEP process. For anyone interested in pursuing them further, the first two would be ``unittest`` module enhancement requests, the third would be a ``warnings`` module enhancement request, while the last would only require a PEP if inferring API deprecations from their contents was deemed to be an intractable code analysis problem, and an explicit function and parameter marker syntax in annotations was proposed instead. The CPython reference implementation will also include the following related changes in 3.7: * a new ``-X dev`` command line option that combines several developer centric settings (including ``-Wd``) into one command line flag: https://bugs.python.org/issue32043 * changing the behaviour in debug builds to show more of the warnings that are off by default in regular interpeter builds: https://bugs.python.org/issue32088 References ========== .. [1] stdlib-sig thread proposing the original default filter change (https://mail.python.org/pipermail/stdlib-sig/2009-November/000789.html) .. [2] Python 2.7 notification of the default warnings filter change (https://docs.python.org/3/whatsnew/2.7.html#changes-to-the-handling-of-depr…) .. [3] Emitting warnings based on the location of the warning itself (https://pypi.org/project/warn/) .. [4] GitHub PR for PEP 565 implementation (https://github.com/python/cpython/pull/4458) .. [5] Tracker issue for PEP 565 implementation (https://bugs.python.org/issue31975) .. [6] python-dev discussion thread for this PEP (https://mail.python.org/pipermail/python-dev/2017-November/150477.html) Copyright ========= This document has been placed in the public domain. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 4

Using async/await in place of yield expression
by David Mertz 28 Nov '17

28 Nov '17

Changing subject line because this is way off to the side. Guido and Nathaniel point out that you can do everything yield expressions do with async/await *without* an explicit event loop. While I know that is true, it feels like the best case is adding fairly considerable ugliness to the code in the process. > On Sat, Nov 25, 2017 at 3:37 PM, Guido van Rossum <guido(a)python.org> > wrote: > > Maybe you didn't realize async/await don't need an event loop? Driving an > > async/await-based coroutine is just as simple as driving a > yield-from-based > > one (`await` does exactly the same thing as `yield from`). > > On Sun, Nov 26, 2017 at 12:29 PM, Nathaniel Smith <njs(a)pobox.com> wrote: > Technically anything you can write with yield/yield from could also be > written using async/await and vice-versa, but I think it's actually > nice to have both in the language. > Here is some code which is definitely "toy", but follows a pattern pretty similar to things I really code using yield expressions: In [1]: from itertools import takewhile In [2]: def injectable_fib(a=1, b=2): ...: while True: ...: new = yield a ...: if new is not None: ...: a, b = new ...: a, b = b, a+b ...: In [3]: f = injectable_fib() In [4]: list(takewhile(lambda x: x<200, f)) Out[4]: [1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144] In [5]: f.send((100,200)) Out[5]: 200 In [6]: list(takewhile(lambda x: x<1000, f)) Out[6]: [300, 500, 800] Imagining that 'yield' vanished from the language tomorrow, and I wanted to write the same thing with async/await, I think the best I can come up with is... actually, I just don't know who to do it without any `yield`. I can get as far as a slightly flawed: In [9]: async def atakewhile(pred, coro): ...: l = [] ...: async for x in coro: ...: if pred(x): ...: return l ...: l.append(x) But I just have no idea what would go in the body of async def afib_injectable(): (that is, if I'm prohibited a `yield` in there) -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

10 13