Mailman 3 January 2020 - Python-Dev

Mixed Python/C debugging
by Skip Montanaro April 26, 2023

April 26, 2023

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum March 29, 2023

March 29, 2023

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (… [View More]flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido) [View Less]

7 17

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com May 18, 2021

May 18, 2021

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: … [View More]$Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: [View Less]

8 14

Boundaries between numbers and identifiers
by Serhiy Storchaka April 15, 2021

April 15, 2021

In Python 2.5 `0or[]` was accepted by the Python parser. It became an error in 2.6 because "0o" became recognizing as an incomplete octal number. `1or[]` still is accepted. On other hand, `1if 2else 3` is accepted despites the fact that "2e" can be recognized as an incomplete floating point number. In this case the tokenizer pushes "e" back and returns "2". Shouldn't it do the same with "0o"? It is possible to make `0or[]` be parseable again. Python implementation is able to tokenize … [View More]

8 10

The Python 2 death march
by Benjamin Peterson April 16, 2020

April 16, 2020

Hi all, It's finally time to schedule the last releases in Python 2's life. There will be two more releases of Python 2.7: Python 2.7.17 and Python 2.7.18. Python 2.7.17 release candidate 1 will happen on October 5th followed by the final release on October 19th. I'm going to time Python 2.7.18 to coincide with PyCon 2020 in April, so attendees can enjoy some collective catharsis. We'll still say January 1st is the official EOL date. Thanks to Sumana Harihareswara, there's now a FAQ about … [View More]

24 38

Last call for comments on PEP 573 (Module State Access from C Extension Methods)
by Stefan Behnel March 11, 2020

March 11, 2020

Hi all, I think that PEP 573 is ready to be accepted, to greatly improve the state of extension modules in CPython 3.9. https://www.python.org/dev/peps/pep-0573/ It has come a long way since the original proposal and went through several iterations and discussions by various interested people, effectively reducing its scope quite a bit. So this is the last call for comments on the latest version of the PEP, before I will pronounce on it. Please keep the discussion in this thread. Stefan

3 7

Stalemate on bringing back PEP 523 support into Python 3.8
by Brett Cannon March 10, 2020

March 10, 2020

An unfortunate side-effect of making PyInterpreterState in Python 3.8 opaque is it removed [PEP 523](https://www.python.org/dev/peps/pep-0523/) support. https://www.python.org/dev/peps/pep-0523/ was opened to try and fix this, but there seems to be a stalemate in the issue. A key question is at what API level should setting the frame evaluation function live at? No one is suggesting the stable ABI, but there isn't agreement between CPython or the internal API (there's also seems to be a … [View More]

7 9

Request to postpone some Python 3.9 incompatible changes to Python 3.10
by Victor Stinner March 7, 2020

March 7, 2020

Hi, Python 3.9 introduces many small incompatible changes which broke tons of Python projects, including popular projects, some of them being unmaintained but still widely used (like nose, last release in 2015). Miro and me consider that Python 3.9 is pushing too much pressure on projects maintainers to either abandon Python 2.7 right now (need to update the CI, the documentation, warn users, etc.), or to introduce a *new* compatibility layer to support Python 3.9: layer which would be … [View More]dropped as soon as Python 2.7 support will be dropped (soon-ish). Python 3.9 is too early to accumulate so many incompatible changes on purpose, some incompatible changes like the removal of collections aliases to ABC should be reverted, and reapplied on Python 3.10. Python 3.9 would be the last release which still contain compatibility layers for Python 2.7. Said differently, we request to maintain the small compatibility layer in Python for one more cycle, instead of requesting every single project maintainer to maintain it on their side. We consider that the general maintenance burden is lower if it's kept in Python for now. == Fedora COPR notify packages broken by Python 3.9 == In Python 3.9, Victor introduced tons of incompatible changes at the beginning of the devcycle. His plan was to push as many as possible, and later decide what to do... This time has come :-) He wrote PEP 606 "Python Compatibility Version" and we wrote PEP 608 "Coordinated Python release", but both have been rejected. At least, it seems like most people agreed that having a CI to get notified of broken projects should help. We are updating the future Fedora 33 to replace Python 3.8 with Python 3.9. We are using a tool called "COPR" which is like a sandbox and can be seen as the CI discussed previously. It rebuilds Fedora using Python 3.9 as /usr/bin/python3 (and /usr/bin/python !). We now have a good overview of broken packages and which incompatible changes broke most packages. https://fedoraproject.org/wiki/Changes/Python3.9 - Describes the Fedora change. https://copr.fedorainfracloud.org/coprs/g/python/python3.9/monitor/ - Has package failures. Some packages fail because of broken dependencies. https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON39 - Has open Python 3.9 bug reports for Fedora packages. Some problems have been fixed upstream already before reaching Fedora, most are only fixed when the Fedora maintainers report the problems back to upstream maintainers. Right now, there are 150+ packages broken by Python 3.9 incompatible changes. == Maintenance burden == Many Python projects have not yet removed Python 2 support and Python 2 CI. It's not that they would be in the "we will not drop Python 2 support ever" camp, it's just that they have not done it yet. Removing Python 2 compatibility code from the codebase and removing it from the documentation and metadata and CI is a boring task, doesn't bring anything to users, and it might take a new major release of the library. At this point, we are very early in 2020 to expect most projects to have already done this. At the same time, we would like them to support Python 3.9 as soon in the release cycle as possible. By removing Python 2 compatibility layers from the 3.9 standard library, we are forcing the projects maintainers to re-invent their own compatibility layers and copy-paste stuff like this around. Example: try: from collections.abc import Sequence except ImprotError: # Python 2.7 doesn't have collections.abc from collections import Sequence While if we remove collections.Sequence in 3.10, they will face this decision early in 2021 and they will simply fix their code by adding ".abc" at the proper places, not needing any more compatibility layers. Of course, there will be projects that will still have declared Python 2 support in 2021, but it will arguably not be that many. While it's certainly tempting to have "more pure" code in the standard library, maintaining the compatibility shims for one more release isn't really that big of a maintenance burden, especially when comparing with dozens (hundreds?) of third party libraries essentially maintaining their own. An good example of a broken package is the nose project which is no longer maintained (according to their website): the last release was in 2015. It remains a very popular test runner. According to libraries.io, it has with 3 million downloads per month, 41.7K dependent repositories and 325 dependent packages. We patched nose in Fedora to fix Python 3.5, 3.6, 3.8 and now 3.9 compatibility issues. People installing nose from PyPI with "pip install" get the unmaintained flavor which is currently completely broken on Python 3.9. Someone should take over the nose project and maintain it again, or every single project using nose should pick another tool (unittest, nose2, pytest, whatever else). Both options will take a lot of time. == Request to revert some incompatible changes == See https://docs.python.org/dev/whatsnew/3.9.html#removed Incompatible changes which require "if <python3>: (...) else: (...)" or "try: <python3 code> except (...): <python2 code>": * Removed tostring/fromstring methods in array.array and base64 modules * Removed collections aliases to ABC classes * Removed fractions.gcd() function (which is similar to math.gcd()) * Remove "U" mode of open(): having to use io.open() just for Python 2 makes the code uglier * Removed old plistlib API: 2.7 doesn't have the new API == Kept incompatible changes == Ok-ish incompatible changes (mostly only affects compatiblity libraries like six): * _dummy_thread and dummy_threading modules removed: broke six, nine and future projects. six and nine are already fixed for 3.9. OK incompatible changes (can be replaced by the same code on 2.7 and 3.9): * isAlive() method of threading.Thread has been removed: Thread.is_alive() method is available in 2.7 and 3.9 * xml.etree.ElementTree.getchildren() and xml.etree.ElementTree.getiterator() methods are removed from 3.9, but list()/iter() works in 2.7 and 3.9 == Call to check for DeprecationWarning in your own projects == You must pay attention to DeprecationWarning in Python 3.9: it will be the last "compatibility layer" release, incompatible changes will be reapplied to Python 3.10. For example, you can use the development mode to see DeprecationWarning and ResourceWarning: use the "-X dev" command line option or set the PYTHONDEVMODE=1 environment variable. Or you can use the PYTHONWARNINGS=default environment variable to see DeprecationWarning. You might even want to treat all warnings as errors to ensure that you don't miss any when you run your test suite in your CI. You can use PYTHONWARNINGS=error, and combine it with PYTHONDEVMODE=1. Warnings filters can be used to ignore warnings in third party code, see the documentation: https://docs.python.org/dev/library/warnings.html#the-warnings-filter -- Victor Stinner and Miro Hrončok for Fedora [View Less]

29 65

Comments on PEP 558 (Defined semantics for locals() )
by Mark Shannon Feb. 22, 2020

Feb. 22, 2020

Hi, First of all I want to say that I'm very much in favour of the general idea behind PEP 558. Defined semantics is always better than undefined semantics :) However, I think there are a few changes needed: 1. Don't add anything to the C API, please. Frame attributes can be accessed via `PyObject_GetAttr[String]`. 2. Don't make the behaviour dependent on whether "tracing" is turned on. Doing so forces debuggers to use sys.settrace which is horribly slow. It also makes the … [View More]

2 6

Python Language Summit at PyCon 2020
by Mariatta Feb. 18, 2020

Feb. 18, 2020

(cross posting to python-committers and python-dev) I'm happy to announce that the signups for Python Language Summit at PyCon 2020 is now open. Full details at: https://us.pycon.org/2020/events/languagesummit/ *TL;DR* When: Wednesday, April 15, 2020, 9am–4pm (Note, we’re starting 1 hour earlier than usual!) Where: David L. Lawrence Convention Center, Pittsburgh, PA, room TBD Sign up to attend: https://forms.gle/Fg7ayhYTaY75J1r7A (closes Feb 29th, 2020 AoE) Sign up to discuss a topic: … [View More]

2 3

PEP 458: Secure PyPI downloads with package signing
by Sumana Harihareswara Feb. 12, 2020

Feb. 12, 2020

Hi! I'm forwarding this on behalf of Marina Moore https://github.com/mnm678 . - Sumana Harihareswara --- PEP 458 ( https://www.python.org/dev/peps/pep-0458/ ) proposes using The Update Framework (TUF) to allow users of PyPI to verify that the packages they install originate from PyPI. Implementing this PEP would provide protection in the event of an attack on PyPI, its mirrors, or the network used to install packages. We started this PEP in 2013, and have recently revised it and restarted … [View More]

1 1

extern "C" { ... } in Include/cpython/*.h
by Skip Montanaro Feb. 8, 2020

Feb. 8, 2020

(Apologies. Not sure where to ask this, and I'm not much of a C++ programmer. Maybe I should have just added a comment to the still-open issue.) I just noticed that Nick migrated the guts of Include/frameobject.h to include/cpython/frameobject.h. It's not clear to me that the latter should be #include'd directly from anywhere other than Include/frameobject.h. If that's the case, does the extern "C" stuff still need to be replicated in the lower level file? Won't the scope of the extern "C" … [View More]

3 2

Maintenance of multiprocessing module: there are a few stalled issues/patches
by mailer＠app.tempr.email Jan. 31, 2020

Jan. 31, 2020

Couldn't help but notice that there are quite a few stalled issues/patches, some of them for years even. :-/ So I thought, it might help to bump them here on the mailing list. https://bugs.python.org/issue28053https://github.com/python/cpython/pull/99…

2 1

Summary of Python tracker Issues
by Python tracker Jan. 31, 2020

Jan. 31, 2020

ACTIVITY SUMMARY (2020-01-24 - 2020-01-31) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7300 (+15) closed 43962 (+52) total 51262 (+67) Open issues with patches: 2858 Issues opened (56) ================== #36051: Drop the GIL during large bytes.join operations? https://bugs.python.org/issue36051 reopened by skip.montanaro #39350: Remove … [View More]

1 0

Limited API, MetaClasses, and ExtensionMetaClasses
by Sebastian Berg Jan. 31, 2020

Jan. 31, 2020

Hi all, I may be thinking in the wrong direction, but right now `PyType_Type.tp_new` will resolves the `metaclass` from the bases and call: type = (PyTypeObject *)metatype->tp_alloc(metatype, nslots); where `metatype` is actually resolved from the metatype of the bases. In contrast `PyType_FromSpecWithBases` immediately calls: res = (PyHeapTypeObject*)PyType_GenericAlloc(&PyType_Type, 0); So I am curious whether `PyType_FromSpecWithBases` should not do the same thing, or as … [View More]

1 0

PEP 613 Accepted
by Barry Warsaw Jan. 30, 2020

Jan. 30, 2020

The Steering Council has considered Guido’s request for pronouncement on PEP 613 - Explicit Type Aliases, written by Shannon Zhu and sponsored by Guido van Rossum. The Council unanimously accepts PEP 613. Congratulations! We’ll leave it to the sponsor and author to update the PEP accordingly. -Barry (on behalf of the Python Steering Council)

1 0

Static inline functions in header files, limited API and the stable ABI
by Victor Stinner Jan. 30, 2020

Jan. 30, 2020

Hi, Is it ok to add a static inline function to the limited API? Can it cause ABI issue? Or is it safer to add a regular function? For example, is it safe to add the following function to the limited API? static inline PyObject * _PyObject_CallOneArg(PyObject *func, PyObject *arg) { ... return _PyObject_VectorcallTstate(tstate, func, args, nargsf, NULL); } Or should it be an opaque definition like the following function? PyAPI_FUNC(PyObject *) PyObject_CallNoArgs(PyObject *func); I … [View More]

2 1

Remove COUNT_ALLOCS special build
by Victor Stinner Jan. 29, 2020

Jan. 29, 2020

Hi, I would like to confirm that there is no user of the Python "COUNT_ALLOCS" special build, because I plan to remove it from Python 3.9. If you use it, please raise your hand and explain why other debug tools don't fit your specific use case. -- I'm always annoyed by "#ifdef COUNT_ALLOCS" code which is common in code related to reference counting (object.h, object.c, typeobject.c). It makes the code harder to read and harder to maintain. It's a 27 years old feature which always required to … [View More]

2 1

Question: how to extend standard Python library to add new functionalities?
by daniel.wangksu＠gmail.com Jan. 29, 2020

Jan. 29, 2020

Hi all, I want to add openssl engine in Python SSL library. I know that if the new functions can be added in Python we can inherit existing class (such as ssl). However, It seems the SSL module extends _SSL module which is written in C. I'm not sure how can I extend SSL module in this situation. Any help would be deeply appreciated!

2 1

Typo in Link
by onlinejudge95 Jan. 28, 2020

Jan. 28, 2020

Hi team, First of all is it a right place to ask this question? If not i would love to redirect and apologise for any inconvenience. While i was going through the wiki.python.org links, i saw that the urls( https://wiki.python.org/moin) contains *moin*, is it expected? Thanks, onlinejudge95

3 2

Re: Typo in Link
by Steve Holden Jan. 28, 2020

Jan. 28, 2020

Really webmaster(a)python.org would have been more appropriate, but I've already copied that address on my direct reply to you. The python-dev list is for discussions about the development of the language and its CPython implementation. Kind regards, Steve On Tue, Jan 28, 2020 at 7:20 PM Steve Holden <steve(a)holdenweb.com> wrote: > Hi Steve, > > I believe you will find that wiki.python.org redirects to > wiki.python.org/moin/. While https://wiki.python.org/moin/… [View More]

1 0

Merge bugs.python.org accounts
by Ivan Pozdeev Jan. 28, 2020

Jan. 28, 2020

Who should I contact on subj? BPO UI doesn't have any contact information. -- Regards, Ivan

3 2

Python library for the ServiceNow REST API
by IT Canvass Jan. 28, 2020

Jan. 28, 2020

It should be compatible with modern versions of ServiceNow. If there's API breakage or new features implemented, I usually have that fixed/incorporated into pysnow in a week or two. <a href="https://itcanvass.com/"> servicenow training</a>

4 3

Are PyObject_RichCompareBool shortcuts part of Python or just CPython quirks?
by Tim Peters Jan. 26, 2020

Jan. 26, 2020

PyObject_RichCompareBool(x, y, op) has a (valuable!) shortcut: if x and y are the same object, then equality comparison returns True and inequality False. No attempt is made to execute __eq__ or __ne__ methods in those cases. This has visible consequences all over the place, but they don't appear to be documented. For example, >>> import math >>> ([math.nan] * 5).count(math.nan) 5 despite that `math.nan == math.nan` is False. It's usually clear which methods will be … [View More]

13 34

[RELEASE] Python 3.9.0a3 available for testing
by Łukasz Langa Jan. 25, 2020

Jan. 25, 2020

Go get it here: https://www.python.org/downloads/release/python-390a3/ <https://www.python.org/downloads/release/python-390a3/> This is an early developer preview of Python 3.9 Python 3.9 is still in development. This releasee, 3.9.0a3 is the third of six planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta … [View More]

1 0

Summary of Python tracker Issues
by Python tracker Jan. 24, 2020

Jan. 24, 2020

ACTIVITY SUMMARY (2020-01-17 - 2020-01-24) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7285 (+17) closed 43910 (+55) total 51195 (+72) Open issues with patches: 2852 Issues opened (45) ================== #29258: __init__.py required for pkgutil.walk_packages in python3 https://bugs.python.org/issue29258 reopened by inada.naoki #39353: … [View More]

1 0

Re: SWIFT Confirmation and request for Updated Statement for DEC invoice
by ron＠ronstruckandauto.com Jan. 24, 2020

Jan. 24, 2020

Sir, Please note that and we are Transferring payment again to your Designated bank As advised earlier. Kindly see attached covering letter with Bank Swift for follow up with your bank. Also see below specification Of payment breakdown. USD 3,146.05 USD 28,550.00 Please kindly confirm the attached swifts and Send Updated Statement of Account for our records Purpose. Regards Brian Alex Head Of Finance Treasury Department. Address: ul. 327 Abdrakhmanova (Bayalinov lane) +996 312 37 67 70

1 0

Comparing UUID objects to strings: why not?
by Michael Anckaert Jan. 23, 2020

Jan. 23, 2020

Hello everyone I have a usecase where I'm comparing a UUID instance with string quite often. A case where I have to convert the UUID instance to a string using str(uuid_obj), leading to redundant code: if str(uuid_obj) == uuid: # do stuff... In my experience I can't find a case where you wouldn't want the implicit conversion to string before comparing a UUID with string. So I would like to propose improving the __eq__ method of UUID to: def __eq__(self, other): if … [View More]

4 4

Re: Documenting Python's float.__str__()
by Vedran Čačić Jan. 22, 2020

Jan. 22, 2020

> ints print the same in just about every single programming language that uses base ten Arabic-Hindu digits 0...9. It's kind of a universal. Not actually true: C64's Basic V2 printed positive numbers with space in front. -- Vedran Čačić

1 0

Documenting Python's float.__str__()
by Karl O. Pinc Jan. 22, 2020

Jan. 22, 2020

Hello, There appears to be extremely minimal documentation on how floats are formatted on output. All I really see is that float.__str__() is float.__repr__(). So that means that float->str->float does not result in a different value. It would be nice if the output format for float was documented, to the extent this is possible. #python suggested that I propose a patch, but I see no way to write a documentation patch without having any clue about what Python promises, whether in the … [View More]CPython implementation or as part of a specification for Python. What are the promises Python makes about the str() of a float? Will it produce 1.0 today and 1.0e0 or +1.0 tomorrow? When is the result in exponential notation and when not? Does any of this depend on the underlying OS or hardware or Python implementation? Etc. I'm guessing that Python is consistent with an IEEE 754 "external character sequence", but don't know what the IEEE specification says or whether python conforms. I don't really care whether there's documentation for __str__() or __repr__() or something else. I'm just thinking that there should be some way to guarantee a well defined "useful" float output formatting. By "useful" I mean in exponential notation when non-exponential notation is over-long. I am writing a program that sometimes prints python floats and want to be able to document what is printed. Right now I can't truly guarantee anything, other than the nan and inf and -inf representations. (I feel comfortable with nan and the like because I don't see it likely that their representations will change.) Of course I could always re-implement Python's float.__repr__() in Python so as to have full control, but this should be pointless. Python's output representation is unlikely to change and Python should be able to make sufficient promises about its existing float representation. I suppose there are similar issues with integers, but the varieties of floating point number implementations and the existence of both exponential and non-exponential representations make float particularly problematic and representations potentially mercurial. I also don't know if documentation changes with regard to external representations would require a PEP. I have found the following related information: Use shorter float repr when possible https://bugs.python.org/issue1580 https://github.com/python/cpython/blob/master/Python/pystrtod.c#L831 String conversion and formatting https://docs.python.org/3/c-api/conversion.html sys.float_repr_style https://docs.python.org/3/library/sys.html#sys.float_repr_style object.__str__(self) https://docs.python.org/3/reference/datamodel.html#object.__str__ At the end of the day I don't _really_ care. But having put thought into the matter I care enough to write this email and ask the question. Regards, Karl <kop(a)karlpinc.com> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein [View Less]

6 13

PEP 587
by Zak Mabbott Jan. 21, 2020

Jan. 21, 2020

I am really confused on writing python and what app I need to write it in any tips or any ideas for apps??

2 1

About "python-porting" mail list
by Facundo Batista Jan. 20, 2020

Jan. 20, 2020

Hello! This list (which I co-admin, with Georg) is getting less and less traffic as months pass by. See: https://mail.python.org/pipermail/python-porting/ The interwebs has been collecting ton of resources about porting py2 to 3 during these years. Any not-yet-answered question surely can be done in a list with more participants. Can we kill this list? Thanks! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista

4 7

Re: Questions about CPython's behavior on addition operator -- binary_op1 (abstract.c) and nb_add (typeobject.c)
by Raphaël Monat Jan. 20, 2020

Jan. 20, 2020

> I'm also keen to understand this and think I can elucidate a little, > from my study of the code. The perspective of someone who *didn't* > write it might help, but I defer to the authors' version when it appears. > Thank you! I'd be happy to know the authors' opinion as well. > A crucial observation is that there is only one nb_add slot in a type > definition. Think about adding a int(1) + float(2). Where it lands in > long_add (v->ob_type->tp_as_number->… [View More]nb_add) will return > NotImplemented, because it does not understand the float right-hand > argument, but float_add (w->ob_type->tp_as_number->nb_add) is able to > give an answer, since it can float the int, behaving as > float.__radd__(f, i). So the slots have to implement both __add__ and > __radd__. > I agree. And by default, this is done by expanding the macro "SLOT1BIN(slot_nb_add, nb_add, "__add__", "__radd__")". > The logic is the same as binary_op1, but the function has to deal with > the possibility that one or other type may already provide a special > function wrapper function in its nb_add slot, which is what the test > tp_as_number->SLOTNAME == TESTFUNC is about. TESTFUNC is nearly always > the same as FUNCNAME. Then it also deals with quite a complex decision > in method_is_overloaded(). > I have overlooked this test, and now I'm confused by it. In the definition of slot_nb_add, it seems to check that tp_as_number->nb_add == slot_nb_add. Is it (a convoluted way) to check that tp_as_number->nb_add != NULL? Otherwise, I guess this always hold for standard library modules, but someone may change tp_as_number->nb_add for another function? > The partial repetition of the logic, which I think is now nested > (because binary_op1() may have called slot_nb_add) is necessary to > insert the more complex version into the decision tree. But this is > roughly where my ability to visualise the paths runs out. > I have an hypothesis: when binary_op1(v, w, ...) is called: 1) either slotv (v->ob_type->tp_as_number->nb_add) is defined. This calls v's slot_nb_add. This slot_nb_add may then call either v's "__add__" or w's "__radd__" using the calls to vectorcall_maybe. 2) or slotv is not defined. In that case we start the whole process using w's slot_nb_add, which should be stored in w->ob_type->tp_as_number->nb_add. In particular, does this mean that if slotv is defined, the if(slotw) <https://github.com/python/cpython/blob/3.8/Objects/abstract.c#L813> will never be reached (meaning this if could be changed by an else if)? Letting the case where w is a subtype of v aside for now, this would mean that some tests may be performed twice, but only one slot_nb_add will be called? (of course, this slot_nb_add will be able to call either its __add__ or the __radd__ of the other) > I would not say they are "defined" by the slot_nb_add function. > Rather, if one or other has been defined (in Python), v.__add__ or > w.__radd__ is called by the single function slot_nb_add. They cannot > be called directly from C, but call_maybe() supplies the mechanism. > > A confusing factor is that for types defined in C and filling the > nb_add slot, the slot function (float_add, or whatever) has to be > wrapped by two descriptors that can then sit in the type's dictionary > as "__add__" and "__radd__". In that case these *are* defined by a > wrapped C function, but that function is the function in the > implementation of the type (float_add, say), not slot_nb_add. There > are two kinds of wrapper: one used "Python-side out", so > Python-calling "__add__" leads to nb_add's behaviour, and one "C-side > out" so C-calling via nb_add leads to "__add__". > This makes sense. Thank you! > I *think* it is changed to contain slot_nb_add as defined by the macro. > And then slot_nb_add is able to dispatch to __add__ or __radd__ with vectorcall_maybe. Thanks! When this is all clearer (and hopefully accurate), I'll wrap it up in a blog post or something. -- Raphaël [View Less]

1 0

Remove formatter module from Python 3.9, deprecated for 7 years (Python 3.4)
by Victor Stinner Jan. 17, 2020

Jan. 17, 2020

Hi, I proposed https://bugs.python.org/issue39352 to remove the formatter module. It's deprecated since Python 3.4. The main reason why it's still around is the PEP 4 rule: "In order to facilitate writing code that works in both Python 2 & 3 simultaneously, any module that exists in both Python 3.5 and Python 2.7 will not be removed from the standard library until Python 2.7 is no longer supported as specified by PEP 373." https://www.python.org/dev/peps/pep-0004/#for-modules-existing-in-… [View More]

3 3

Summary of Python tracker Issues
by Python tracker Jan. 17, 2020

Jan. 17, 2020

ACTIVITY SUMMARY (2020-01-10 - 2020-01-17) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7268 (+17) closed 43855 (+63) total 51123 (+80) Open issues with patches: 2847 Issues opened (54) ================== #16576: ctypes: structure with bitfields as argument https://bugs.python.org/issue16576 reopened by vinay.sajip #37751: In codecs, function … [View More]

1 0

Questions about CPython's behavior on addition operator -- binary_op1 (abstract.c) and nb_add (typeobject.c)
by Raphaël Monat Jan. 17, 2020

Jan. 17, 2020

Hi all, I'm looking at CPython's behavior when an addition is called. From what I understand, binary_op1 <https://github.com/python/cpython/blob/3.8/Objects/abstract.c#L786> is eventually called, and it calls either slotv or slotw, which seems to be the binaryfunc defined as nb_add in the field tp_as_number of respectively v / w. I have a few questions: 1) In the default case, tp_as_number->nb_add is defined by the function slot_nb_add <https://github.com/python/cpython/… [View More]

2 1

Windows - rebuilding grammar files
by Abdur-Rahmaan Janhangeer Jan. 14, 2020

Jan. 14, 2020

Greetings list, On windows i want to play with the grammar file, but according to this article <https://realpython.com/cpython-source-code-guide/>: > For Windows, there is no officially supported way of running pgen. However, you can clone my fork <https://github.com/tonybaloney/cpython/tree/pcbuildregen> and run build.bat --regen from within the PCBuild directory. But i don't want to work from his fork. Is there an official way of regenerating the files? I just want to have … [View More]

2 2

Why doesn't venv also install python3*-config?
by Musbur Jan. 13, 2020

Jan. 13, 2020

Hello, I'm experimenting with package development on different versions of Python in different virtualenvs. After running "make" I don't do "make install", but rather I set up virtualenvs by running /path/to/source/python -m venv env_dir. This works for as long as I don't need to compile extensions. Once I do that I'm running into trouble because there is no python3-config binary in the venv, so it uses the "system" python3-config which of course returns results for the /usr(/local)/....… [View More]

10 15

Should we pass tstate in the VECTORCALL calling convention before making it public?
by Victor Stinner Jan. 11, 2020

Jan. 11, 2020

Hi, I started to modify Python internals to pass explicitly the Python thread state ("tstate") to internal C a functions: https://vstinner.github.io/cpython-pass-tstate.html Until subinterpreters will be fully implemented, it's unclear if getting the current Python thread state using _PyThreadState_GET() macro (which uses an atomic read) will remain efficient. For example, right now, the "GIL state" API doesn't support subinterpreters: fixing this may require to add a lock somewhere which may … [View More]make _PyThreadState_GET() less efficient. Sorry, I don't have numbers, since I didn't experiment to implement these changes yet: I was blocked by other issues. We can only guess at this point. To me, it sounds like a good practice to pass the current Python thread state to internal C functions. It seems like most core developers agreed with that in my previous python-dev thread "Pass the Python thread state to internal C functions": https://mail.python.org/archives/list/python-dev@python.org/thread/PQBGECVG… The question is now if we should "propagate" tstate to function calls in the latest VECTORCALL calling convention (which is currently private). Petr Viktorin plans to make VECTORCALL APIs public in Python 3.9, as planned in the PEP 590: https://bugs.python.org/issue39245 I added explicitly Stefan Behnel in copy, since Cython should be directly impacted by such change. Cython is the kind of project which may benefit of having tstate accessible directly. I started to move more and more things from "globals" to "per interpreter". For example, the garbage collector is now "per interpreter" (lives in PyThreadState). Small integer singletons are now also "per singleton": int("1") are now different objects in each interpreter, whereas they were shared previously. Later, even "None" singleton (and all other singletons) should be made "per interpreter". Getting a "per interpreter" object requires to state from the Python thread state: call _PyThreadState_GET(). Avoiding _PyThreadState_GET() calls reduces any risk of making Python slower with incoming subinterpreter changes. For the long term, the goal is that each subinterpreter has its own isolated world: no Python object should be shared, no state should be shared. The intent is to avoid any need of locking, to maximize performance when running interpreters in parallel. Obviously, each interpreter will have its own private GIL ;-) Py_INCREF() and Py_DECREF() would require slow atomic operations if Python objects are shared. If objects are not shared between interpreters, Py_INCREF() and Py_DECREF() can remain as fast as they are today. Any additional locking may kill performance. Victor -- Night gathers, and now my watch begins. It shall not end until my death. [View Less]

6 11

Object deallocation during the finalization of Python program
by Pau Freixes Jan. 11, 2020

Jan. 11, 2020

Hi, Recently I've been facing a really weird bug where a Python program was randomly segfaulting during the finalization, the program was using some C extensions via Cython. Debugging the issue I realized that during the deallocation of one of the Python objects the deallocation function was trying to release a pointer that was surprisingly assigned to NULL. The pointer was at the same time held by another Python object that was an attribute of the Python object that had the deallocation … [View More]

3 3

Requesting PR review on locale module
by Cédric Krier Jan. 11, 2020

Jan. 11, 2020

Hi, Few months ago, I submitted a PR [1] for the bpo [2] about locale.format casting Decimal into float. Has someone some times to finish the review? This change is blocking a bugfix on Tryton [3]. [1] https://github.com/python/cpython/pull/15275 [2] https://bugs.python.org/issue34311 [3] https://bugs.tryton.org/issue8574 Thanks, -- Cédric Krier - B2CK SPRL Email/Jabber: cedric.krier(a)b2ck.com Tel: +32 472 54 46 59 Website: http://www.b2ck.com/

4 4

Is anyone aware of any existing tests for ternary_op in abstract.c?
by Nick Coghlan Jan. 11, 2020

Jan. 11, 2020

[Note: I don't think this is a high priority issue, as it only relates to doing mixed-type modular exponentiation, and that seems like an inherently bad idea anyway. But I'm also curious if anyone is able to explain the current behaviour, as I haven't come up with a better explanation than "It doesn't actually work, but nobody needs it enough to complain about it not working"] While writing up bpo-39302 [1], I tried experimenting at the REPL to see if I could get __pow__ to ever exercise the … [View More]code paths where 3-argument pow() is handled by the exponent or the modulus (as described in the source code and PEP 208), rather than the usual case where it is handled by the base. I failed - the only time I could get my custom __pow__ implementation to run was when it was the base, never when it was the modulus or exponent. This means that, as far as I'm able to tell, ternary_op in abstract.c isn't actually working as intended, or at least, it doesn't work in combination with the way the `nb_power` slot wrapper is implemented in typeobject.c. That slot wrapper only ever calls the Python level function provided by the first operand, even though ternary_op calls 3 different slot implementations, they all ultimately get resolved as attempts to call the same python level function (the `__pow__` implementation on the base). I couldn't find any test cases that exercise this functionality, so even though PEP 208 describes the expected semantics, it isn't clear whether or not the implementation has ever achieved them. This is the session where I tried to see if I could get it to work the way I expected it to work based on the comments in ternary_op: [py3.7]> class _PowArgsMixin: ... def __pow__(*args): ... return tuple(map(type, args)) ... [py3.7]> class NoPow: ... pass ... [py3.7]> class NotImplementedPow: ... def __pow__(*args): ... return NotImplemented ... [py3.7]> class UnrelatedPow(_PowArgsMixin): ... pass ... [py3.7]> class ChildOfNoPow(_PowArgsMixin, NoPow): ... pass ... [py3.7]> class ChildOfNotImplementedPow(_PowArgsMixin, NotImplementedPow): ... pass ... [py3.7]> no_pow = NoPow() [py3.7]> ni_pow = NotImplementedPow() [py3.7]> has_pow = UnrelatedPow() [py3.7]> child_no = ChildOfNoPow() [py3.7]> child_ni = ChildOfNotImplementedPow() [py3.7]> # They all give the expected result as the base ... pow(has_pow, no_pow, no_pow) (<class '__main__.UnrelatedPow'>, <class '__main__.NoPow'>, <class '__main__.NoPow'>) [py3.7]> pow(has_pow, ni_pow, ni_pow) (<class '__main__.UnrelatedPow'>, <class '__main__.NotImplementedPow'>, <class '__main__.NotImplementedPow'>) [py3.7]> pow(child_no, no_pow, no_pow) (<class '__main__.ChildOfNoPow'>, <class '__main__.NoPow'>, <class '__main__.NoPow'>) [py3.7]> pow(child_ni, ni_pow, ni_pow) (<class '__main__.ChildOfNotImplementedPow'>, <class '__main__.NotImplementedPow'>, <class '__main__.NotImplementedPow'>) [py3.7]> # But none of them work when given as the exponent ... pow(no_pow, has_pow, no_pow) Traceback (most recent call last): File "<stdin>", line 2, in <module> TypeError: unsupported operand type(s) for pow(): 'NoPow', 'UnrelatedPow', 'NoPow' [py3.7]> pow(ni_pow, has_pow, ni_pow) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NotImplementedPow', 'UnrelatedPow', 'NotImplementedPow' [py3.7]> pow(no_pow, child_no, no_pow) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NoPow', 'ChildOfNoPow', 'NoPow' [py3.7]> pow(ni_pow, child_ni, ni_pow) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NotImplementedPow', 'ChildOfNotImplementedPow', 'NotImplementedPow' [py3.7]> # And none of them work when given as the modulus either ... pow(no_pow, no_pow, has_pow) Traceback (most recent call last): File "<stdin>", line 2, in <module> TypeError: unsupported operand type(s) for pow(): 'NoPow', 'NoPow', 'UnrelatedPow' [py3.7]> pow(ni_pow, ni_pow, has_pow) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NotImplementedPow', 'NotImplementedPow', 'UnrelatedPow' [py3.7]> pow(no_pow, no_pow, child_no) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NoPow', 'NoPow', 'ChildOfNoPow' [py3.7]> pow(ni_pow, ni_pow, child_ni) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for pow(): 'NotImplementedPow', 'NotImplementedPow', 'ChildOfNotImplementedPow' [1] https://bugs.python.org/issue39302 -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

1 0

Summary of Python tracker Issues
by Python tracker Jan. 10, 2020

Jan. 10, 2020

ACTIVITY SUMMARY (2020-01-03 - 2020-01-10) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7251 (+24) closed 43792 (+66) total 51043 (+90) Open issues with patches: 2846 Issues opened (63) ================== #25872: multithreading traceback KeyError when modifying file https://bugs.python.org/issue25872 reopened by xtreak #27657: urlparse fails … [View More]

1 0

Differences in what lint (e.g., mypy, flake8, etc) require from new code and what Cpython uses/returns - e.g., strings with single/double quotes.
by Michael Jan. 9, 2020

Jan. 9, 2020

Hi all, Last year I was struggling to get some code to pass CI in pypa/packaging. There were other issues, but one that suprised me most was learning to ALWAYS use double quotes (") to get the code to pass the lint check (type checking). Anything using single quotes (') as string delimiters were not accepted as a STRING type. I am not questioning the demands of the lint checker - rather - I am offering my services (aka time) to work through core to make CPython pass it's own (I assume) PEP … [View More]recommendations or guidelines. Again, this is not a bug-report - but an offer to assist with what I see as an inconsistency. If my assistance in this area is appreciated/desired I would appreciate one or more core developers to assist me with managing the PR process. Not for speed, but I would not want to burden just one core developer. My approach would entail opening a number of related 'issues' for different pieces: e.g., Lib, Modules, Python, Documentation where some Lib|Modules|Python pieces might need to be individual 'issues' due to size and/or complexity. I do not see this as happening 'overnight'. so-called simple things to fix: In Documentation: The return value is the result of the evaluated expression. Syntax errors are reported as exceptions. Example: x = 1 eval('x+1') 2 Change the text above to state eval("x+1") - assuming the lint process would no longer accept eval('x+1') as proper typed syntax. And, then, hoping this is still regarded as 'simple' make sure code such as ctypes.util.find_library() is consistent, returning strings terminated by double quotes rather than the single quotes as of now. >>> import ctypes.util >>> ctypes.util.find_library("c") >>> 'libc.a(shr.o)' >>> ctypes.util.find_library("ssl") >>> 'libssl.a(libssl.so)' Something more "complex" may be the list of names dir() returns: >>> >>> import sysconfig >>> dir(sysconfig)) ['_BASE_EXEC_PREFIX', '_BASE_PREFIX', '_CONFIG_VARS', '_EXEC_PREFIX', '_INSTALL_SCHEMES', '_PREFIX', '_PROJECT_BASE', '_PYTHON_BUILD', '_PY_VERSION', '_PY_VERSION_SHORT', '_PY_VERSION_SHORT_NO_DOT', '_SCHEME_KEYS', '_USER_BASE', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_expand_vars', '_extend_dict', '_generate_posix_vars', '_get_default_scheme', '_get_sysconfigdata_name', '_getuserbase', '_init_non_posix', '_init_posix', '_is_python_source_dir', '_main', '_parse_makefile', '_print_dict', '_safe_realpath', '_subst_vars', '_sys_home', 'get_config_h_filename', 'get_config_var', 'get_config_vars', 'get_makefile_filename', 'get_path', 'get_path_names', 'get_paths', 'get_platform', 'get_python_version', 'get_scheme_names', 'is_python_build', 'os', 'pardir', 'parse_config_h', 'realpath', 'scheme', 'sys'] >>> Regards, Michael [View Less]

3 3

Dict __missing__ consistency issues.
by Bar Harel Jan. 9, 2020

Jan. 9, 2020

Hey guys, I was just about to fix dict's get() to call __missing__ if a key doesn't exist (before returning the default value) but realized that although small, that patch can cause future issues. Right now there's an inconsistency: >>> from collections import UserDict >>> class A(dict): ... def __missing__(self, key): ... print(key) ... >>> class B(UserDict): ... def __missing__(self, key): ... print("UserDict", key) ... >>> a = A() >>> b =… [View More]

3 8

Re: Dict __missing__ consistency issues.
by Bar Harel Jan. 8, 2020

Jan. 8, 2020

> > [Bar Harel] > > As for get(). Current implementation of UserDict returns the default > value > > for get() if __missing__() raises KeyError. Which sounds reasonable to > me. > > After all, you would logically expect __missing__ to be called for get() > > as you tried to get a non-existing value. > > > [Ethan Furman] > I disagree. My understanding of the purpose behind get() is to get a value > or return the default specified in the get() … [View More]

1 0

Dict __missing__ consistency issues
by Bar Harel Jan. 8, 2020

Jan. 8, 2020

Hey guys, I was just about to fix dict's get() to call __missing__ if a key doesn't exist (before returning the default value) but realized that although small, that patch can cause future issues. Right now there's an inconsistency: >>> from collections import UserDict >>> class A(dict): ... def __missing__(self, key): ... print(key) ... >>> class B(UserDict): ... def __missing__(self, key): ... print("UserDict", key) ... >>> a = A() >>> b = … [View More]

1 0

Should set objects maintain insertion order too?
by Larry Hastings Jan. 8, 2020

Jan. 8, 2020

As of 3.7, dict objects are guaranteed to maintain insertion order. But set objects make no such guarantee, and AFAIK in practice they don't maintain insertion order either. Should they? I do have a use case for this. In one project I maintain a "ready" list of jobs; I need to iterate over it, but I also want fast lookup because I soemtimes remove jobs when they're subsequently marked "not ready". The existing set object would work fine here, except that it doesn't … [View More]

26 79

Python Documentation and AIX specifics - how to proceed?
by Michael Jan. 7, 2020

Jan. 7, 2020

First - best wishes all for a happy and healthy 2020! As my nickname implies - my primary means to contribute to Python is with regard to AIX. One of the things I recently came across is Misc/README.AIX which was last updated sometime between 2010 and 2014. I am thinking a facelift is in order. Assuming 2010-2011 as the original date - much has changed and many of the comments are no longer accurate. Before saying so, I would like to check here that - having all tests pass on the 3.8 bot … [View More]

3 3

Python-dev mailing lis archives earlier than late April 1999?
by Skip Montanaro Jan. 7, 2020

Jan. 7, 2020

I could swear python-dev was older than late April 1999, yet that's as far back as the MM3 archives go. As evidence, here's an email from Jack Jansen on 28 April 1999 which was a reply to an earlier message not present in the current archive: https://mail.python.org/archives/list/python-dev@python.org/thread/EMH62JYF… Any pointers to older messages appreciated... Skip

5 16

Steering Council Update (November through December)
by Ewa Jodlowska Jan. 6, 2020

Jan. 6, 2020

Hello! Below is an update from the Steering Council. It covers November and December. The update has been added to the Steering Council repo: https://github.com/python/steering-council/blob/master/updates/2020-01-06-s… (added to README as well). ------------------------------------------------------------------------ # Steering Council Community Update for November - December 2019 ## November recap: - Steering Council assigned one another tasks to check-in on and clean up 29 open PEPs in … [View More]November (Nov 5 and Nov 12 meetings). See [PEP repo]( https://github.com/python/peps/) for details. - Steering Council decided that going forward, only one person will need to approve the minutes and the group will try to rotate that responsibility. Additionally, to keep meetings efficient we will: - 1. have a person responsible for the agenda to make sure the agenda is accurate and timed appropriately - 2. have a person run the actual meeting to make sure we stay on topic and cut discussions when topics go over time. - Steering Council discussed ideas on how to lower the count of PRs. It was decided that Brett would start a discussion on Discourse with the idea to close enhancement PRs. This has been done [here]( https://discuss.python.org/t/automatically-close-all-enhancement-prs-for-co…). At a later time, it was decided the idea would be shut down since many folks did not approve. - Steering Council discussed how a hiring plan can coincide with the Vision Deck and how it would work with sponsor/corporate collaborations. The Vision Deck is an overview document the Steering Council is drafting, which will help develop Python's roadmap for the next 5 years. Goal is to have the Vision Deck complete by PyCon 2020. - Steering Council had a brief discussion about encouraging more candidates to run for the next steering council and that everyone would work on raising awareness for this. ## December recap: - The Steering Council reviewed [PEP 584]( https://www.python.org/dev/peps/pep-0584/) (“Add + and += operators to the built-in dict class”) and decided that Guido would let the PEP authors know | and |= was preferred and that the PEP needed some editing. [This was done]( https://mail.python.org/archives/list/python-ideas@python.org/message/P46F7…). At a later time it was decided that Guido will be BDFL-Delegate. - The Steering Council decided it would meet after the 2020 Steering Council vote ended. The Steering Council will meet December 10 and December 17th. The hand-off meeting will happen the first week of January. Ewa will send out a Doodle for the first full week of January once the new Steering Council members are known. It was decided that on January 8th, 2020 the Steering Council will have a hand-off meeting between the 2019 team and 2020 team. - Steering Council decided that we would send out ballots to Marc-Andre Lemburg, Alex Martelli, and Kurt B. Kaiser. [Brett responded to thread]( https://mail.python.org/archives/list/python-committers@python.org/message/…) noting this decision and future plans for [PEP 13]( https://www.python.org/dev/peps/pep-0013/). - Steering Council discussed [PEP 611]( https://www.python.org/dev/peps/pep-0611/). Barry [emailed the python-dev@ list]( https://mail.python.org/archives/list/python-dev@python.org/message/KY46EXG…) with the outcome of the discussion. - Steering Council discussed PyPI’s typosquatting issues. - Group discussed the status of the GitHub migration plan. Ewa drafted a job description for the Project Manager role to help with the GitHub migration for the Dec 17th SC meeting. The Steering Council reviewed it and Ewa scheduled a call with the GitHub team for late January for next steps. - Steering Council decided to keep Zulip since the Packaging-wg plans on using it. We will re-evaluate usage next year. Guido informed the [Discourse thread]( https://discuss.python.org/t/should-we-continue-using-zulip/2816/7). Brett [posted an announcement]( https://python.zulipchat.com/#narrow/stream/116742-core.2Fhelp/topic/Engage…) in Zulip. Thank you, Ewa ----------------------- *As a non-profit organization, the PSF depends on sponsorships and donations to support the Python community. Check out our Annual Impact Report for more details: https://www.python.org/psf/annual-report/2019/ <https://www.python.org/psf/annual-report/2019/>* *Please contribute to PSF; we can't continue our work without your support! https://www.python.org/psf/donations/ <https://www.python.org/psf/donations/>* [View Less]

1 0

Adding a scarier warning to object.__del__?
by Yonatan Zunger Jan. 5, 2020

Jan. 5, 2020

Hey everyone, I just encountered yet another reason to beware of __del__: when it's called during interpreter shutdown, for reasons which are kind of obvious in retrospect, if it calls notify() on a threading.Condition, the waiting thread may or may not ever actually receive it, and so if it does that and then tries to join() the thread the interpreter may hang in a hard-to-debug way. This isn't something that can reasonably be fixed, and (like in most cases involving __del__) there's a very … [View More]

10 16

Summary of Python tracker Issues
by Python tracker Jan. 3, 2020

Jan. 3, 2020

ACTIVITY SUMMARY (2019-12-27 - 2020-01-03) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7227 (+24) closed 43726 (+38) total 50953 (+62) Open issues with patches: 2842 Issues opened (48) ================== #27973: urllib.urlretrieve() fails on second ftp transfer https://bugs.python.org/issue27973 reopened by pablogsal #39114: Python 3.9.0a2 … [View More]

1 0

PEP 558: Defined semantics for locals() (December 2019 edition)
by Nick Coghlan Jan. 2, 2020

Jan. 2, 2020

Hi folks, I've finally updated PEP 558 and it's reference implementation based on Nathaniel's feedback back in May. The latest version of the PEP can now be found at https://www.python.org/dev/peps/pep-0558/, and I've created a discussion thread on Discourse: https://discuss.python.org/t/pep-558-defined-semantics-for-locals/2936 The latest version implements Nathaniel's "independent snapshot" proposal, and I like how that has turned out. The one thing that changed from the May discussion … [View More]

2 3

Use of AT_SECURE on Linux
by Max Mazurov Jan. 1, 2020

Jan. 1, 2020

Hello, list. I noticed that CPython does not sanitize environment when transition between SELinux contexts or AppArmor profiles happens, which I think is a good thing to do in cases when one sandboxed application runs a Python script with different (potentially less strict) sandbox configuration. On Linux, such transition is indicated by LSMs (SELinux, AppArmor) using the AT_SECURE bit in the aux. vector. Consult getauxval(3) in the glibc manual for details. Here is the paragraph describing … [View More]

1 0