Mailman 3 January 2021 - Python-Dev

Mixed Python/C debugging
by Skip Montanaro April 26, 2023

April 26, 2023

Having tried comp.lang.python with no response, I turn here... After at least ten years away from Python's run-time interpreter & byte code compiler, I'm getting set to familiarize myself with that again. This will, I think, entail debugging a mixed Python/C environment. I'm an Emacs user and am aware that GDB since 7.0 has support for debugging at the Python code level. Is Emacs+GDB my best bet? Are there any Python IDEs which support C-level breakpoints and debugging? Thanks, Skip

5 6

PEP 448 review
by Guido van Rossum March 29, 2023

March 29, 2023

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)". I think Georg Brandl nailed it: """ *I like the "sequence and dict flattening" part of the PEP, mostly because itis consistent and should be easy to understand, but the comprehension syntaxenhancements seem to be bad for readability and "comprehending" what the codedoes.The call syntax part is a mixed bag on the one hand it is nice to be consistent with the extended possibilities in literals (flattening), but on the other hand there would be small but annoying inconsistencies anyways (e.g. the duplicate kwarg case above).* """ Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.) For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case. The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me. So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression). The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them: >>> xs = [1, 2, 3] >>> ys = ['abc', 'def'] >>> zs = [99] >>> [*xs, *ys, *zs] [1, 2, 3, 'abc', 'def', 99] >>> But now suppose you have a list of iterables: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xss[0], *xss[1], *xss[2]] [1, 2, 3, 'abc', 'def', 99] >>> Wouldn't it be nice if you could write the latter using a comprehension? >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs for xs in xss] [1, 2, 3, 'abc', 'def', 99] >>> This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.: >>> xss = [[1, 2, 3], ['abc', 'def'], [99]] >>> [*xs[:2] for xs in xss] [1, 2, 'abc', 'def', 99] >>> On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident). All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2. I would also like code reviewers (Benjamin?) to start reviewing the patch <http://bugs.python.org/issue2292>, taking into account that the comprehension part needs to be removed. -- --Guido van Rossum (python.org/~guido)

7 17

PEP 638: Syntactic macros
by Mark Shannon Feb. 5, 2023

Feb. 5, 2023

Hi everyone, I've submitted my PEP on syntactic macros as PEP 638. https://www.python.org/dev/peps/pep-0638/ All comments and suggestions are welcome. Cheers, Mark

16 25

Speeding up CPython
by Mark Shannon May 18, 2021

May 18, 2021

Hi everyone, CPython is slow. We all know that, yet little is done to fix it. I'd like to change that. I have a plan to speed up CPython by a factor of five over the next few years. But it needs funding. I am aware that there have been several promised speed ups in the past that have failed. You might wonder why this is different. Here are three reasons: 1. I already have working code for the first stage. 2. I'm not promising a silver bullet. I recognize that this is a substantial amount of work and needs funding. 3. I have extensive experience in VM implementation, not to mention a PhD in the subject. My ideas for possible funding, as well as the actual plan of development, can be found here: https://github.com/markshannon/faster-cpython I'd love to hear your thoughts on this. Cheers, Mark.

33 77

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com May 18, 2021

May 18, 2021

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: $Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

8 14

Let's Fix Class Annotations -- And Maybe Annotations Generally
by Larry Hastings April 24, 2021

April 24, 2021

Howdy howdy. While working on my PEP I stumbled over a lot of behavior by annotations that I found inconsistent and inconvenient. I think there are several problems here that needs fixing. This discussion will probably evolve into a PEP, and I'll be happy to steer that process. But I'm less certain about what the right thing to do is. (Although I do know what I'd prefer!) So let's talk about it! Annotations are represented in Python as a dictionary. They can be present on functions, classes, and modules as an attribute called "__annotations__". We start with: how do you get the annotations from one of these objects? Surely it's as easy as this line from Lib/inspect.py shows us: return func.__annotations__ And yes, that's best practice for getting an annotation from a function object. But consider this line from Lib/functools.py: ann = getattr(cls, '__annotations__', {}) Huh. Why doesn't it simply look at cls.__annotations__? It's because the language declares that __annotations__ on a class or module is optional. Since cls.__annotations__ may not be defined, evaluating that might throw an exception. Three-argument getattr() is much safer, and I assert it's best practice for getting the annotations from a module. But consider this line from Lib/dataclasses.py: cls_annotations = cls.__dict__.get('__annotations__', {}) And a very similar line from Lib/typing.py: ann = base.__dict__.get('__annotations__', {}) Huh! Why is this code skipping the attribute entirely, and examining cls.__dict__? It's because the getattr() approach has a subtle bug when dealing with classes. Consider this example: class A: ax:int=3 class B(A): pass print(getattr(B, '__annotations__', {})) That's right, B *inherits* A.__annotations__! So this prints {'ax': int}. This *can't* the intended behavior of __annotations__ on classes. It's only supposed to contain annotations for the fields of B itself, not those of one of its randomly-selected base classes. But that's how it behaves today--and people have had to work around this behavior for years. Examining the class dict is, sadly, best practice for getting __annotations__ from a class. So, already: three different objects can have __annotations__, and there are three different best practices for getting their __annotations__. Let's zoom out for a moment. Here's the list of predefined data fields you can find on classes: __annotations__ __bases__ __class__ __dict__ __doc__ __module__ __mro__ __name__ __qualname__ All of these describe metadata about the class. In every case *except one*, the field is mandatory, which also means it's never inherited. And in every case *except one*, you cannot delete the field. (Though you *are* allowed to overwrite some of them.) You guessed it: __annotations__ is the exception. It's optional, and you're allowed to delete it. And these exceptions are causing problems. It seems to me that, if the only way to correctly use a language-defined attribute of classes is by rooting around in its __dict__, the design is a misfire. (Much of the above also applies to modules, too. The big difference: since modules lack inheritance, you don't need to look in their __dict__.) Now consider what happens if my "delayed annotations of annotations using descriptors" PEP is accepted. If that happens, pulling __annotations__ out of the class dict won't work if they haven't been generated yet. So today's "best practice" becomes tomorrow's "this code doesn't work". To correctly examine class annotations, code would have to do something like this, which should work correctly in any Python 3.x version: if (getattr(cls, '__co_annotations__', None) or ('__annotations__' in cls.__dict__)): ann = cls.__annotations__ else: ann = {} This is getting ridiculous. Let's move on to a related topic. For each of the objects that can have annotations, what happens if o.__annotations__ is set, and you "del o.__annotations__", then you access o.__annotations__? It depends on what the object is, because each of them behaves differently. You already know what happens with classes: if any of the base classes has __annotations__ set, you'll get the first one you find in the MRO. If none of the bases have __annotations__ set you'll get an AttributeError. For a module, if you delete it then try to access it, you'll always get an AttributeError. For a function, if you delete it then try to get it, the function will create a new empty dict, store it as its new annotations dict, and return that. Why does it do that? I'm not sure. The relevent PEP (3107) doesn't specify this behavior. So, annotations can be set on three different object types, and each of those three have a different behavior when you delete the annotations then try to get them again. As a final topic: what are the permitted types for __annotations__? If you say "o.__annotations__ = <x>", what types are and aren't allowed for <x>? For functions, __annotations__ may be assigned to either None or a dict (an object that passes PyDict_Check). Anything else throws a TypeError. For classes and modules, no checking is done whatsoever, and you can set __annotations__ on those two to any Python object. While "a foolish consistency is the hobgoblin of little minds", I don't see the benefit of setting a module's __annotations__ to 2+3j. I think it's long past time that we cleaned up the behavior of annotations. They should be simple and consistent across all objects that support them. At the very least, I think we should make cls.__annotations__ required rather than optional, so that it's never inherited. What should its default value be? An empty dict would be more compatible, but None would be cheaper. Note that creating the empty dict on the fly, the way function objects do, wouldn't really help--because current best practice means looking in cls.__dict__. I also think you shouldn't be able to delete __annotations__ on any of the three objects (function, class, module). It should always be set, so that the best practice for accessing annotations on an object is always o.__annotations__. If I could wave my magic wand and do whatever I wanted, I'd change the semantics for __annotations__ to the following: * Functions, classes, and modules always have an __annotations__ member set. * "del o.__annotations__" always throws a TypeError. * The language will set __annotations__ to a dict if the object has annotations, or None if it has no annotations. * You may set __annotations__, but you can only set it to either None or a dict (passes PyDict_Check). * You may only access __annotations__ as an attribute, and because it's always set, best practice is to use "o.__annotations__" (though getattr will always work too). * How __annotations__ is stored is implementation-specific behavior; looking in the relevant __dict__ is unsupported. This would grant sanity and consistency to __annotations__ in a way it's never so far enjoyed. The problem is, it's a breaking change. But the existing semantics are kind of terrible, so at this point my goal is to break them. I think the best practice needs to stop requiring examining cls.__dict__; in fact I'd prefer people stop doing it altogether. If we change the behavior as part of a new release of Python, code that examines annotations on classes can do a version check: if (sys.version_info.major >=3 and sys.version_info.minor >= 10): def get_annotations(o): return o.__annotations__ or {} else: def get_annotations(o): # eight or ten lines of complex code goes here ... Or code can just use inspect.get_type_hints(), which is tied to the Python version anyway and should always do the right thing. //arry/

10 44

Boundaries between numbers and identifiers
by Serhiy Storchaka April 15, 2021

April 15, 2021

In Python 2.5 `0or[]` was accepted by the Python parser. It became an error in 2.6 because "0o" became recognizing as an incomplete octal number. `1or[]` still is accepted. On other hand, `1if 2else 3` is accepted despites the fact that "2e" can be recognized as an incomplete floating point number. In this case the tokenizer pushes "e" back and returns "2". Shouldn't it do the same with "0o"? It is possible to make `0or[]` be parseable again. Python implementation is able to tokenize this example: $ echo '0or[]' | ./python -m tokenize 1,0-1,1: NUMBER '0' 1,1-1,3: NAME 'or' 1,3-1,4: OP '[' 1,4-1,5: OP ']' 1,5-1,6: NEWLINE '\n' 2,0-2,0: ENDMARKER '' On other hand, all these examples look weird. There is an assymmetry: `1or 2` is a valid syntax, but `1 or2` is not. It is hard to recognize visually the boundary between a number and the following identifier or keyword, especially if numbers can contain letters ("b", "e", "j", "o", "x") and underscores, and identifiers can contain digits. On both sides of the boundary can be letters, digits, and underscores. I propose to change the Python syntax by adding a requirement that there should be a whitespace or delimiter between a numeric literal and the following keyword.

8 10

minor PR 22981 waiting review ~20 days: https://github.com/python/cpython/pull/22981
by Alfred Perlstein Feb. 22, 2021

Feb. 22, 2021

PR 22981 is a minor update to doctest to allow it to safely consume modules which contain objects that cause inspect.unwrap to throw. I believe the review comments in the PR have been addressed at this point for ~20 days. The patch is relatively small, reviews are done, but it is unmerged. Is there a way to "bump" the request for review? thank you, -Alfred

2 2

PEP: Deferred Evaluation Of Annotations Using Descriptors
by Larry Hastings Feb. 16, 2021

Feb. 16, 2021

I've written a new PEP. Please find it below. Happy reading! //arry/ ---------- PEP: XXXX Title: Deferred Evaluation Of Annotations Using Descriptors Version: $Revision$ Last-Modified: $Date$ Author: Larry Hastings <larry(a)hastings.org> Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Jan-2021 Abstract ======== As of Python 3.9, Python supports two different behaviors for annotations: * original Python semantics, in which annotations are evaluated at the time they are bound, and * PEP 563 semantics, currently enabled per-module by ``from __future__ import annotations``, in which annotations are converted back into strings and must be parsed by ``eval()`` to be used. Original Python semantics created a circular references problem for static typing analysis. PEP 563 solved that problem, but its novel semantics introduced new problems. This PEP proposes a third way that embodies the best of both previous approaches. It solves the same circular reference problems solved by PEP 563, while preserving Python's original straightforward runtime semantics for annotations. In this new approach, the code to generate the annotations dict is written to its own callable, and ``__annotations__`` is a "data descriptor" which calls the callable once and preserves the result. If accepted, these new semantics for annotations would initially be gated behind ``from __future__ import co_annotations``. However, these semantics would eventually be promoted to be the default behavior. Thus this PEP would *supercede* PEP 563, and PEP 563's behavior would be deprecated and eventually removed. Overview ======== .. note:: The code presented in this section is highly simplified for clarity. The intention is to communicate the high-level concepts involved without getting lost in with the details. The actual details are often quite different. See the Implementation_ section later in this PEP for a much more accurate description of how this PEP works. Consider this example code:: def foo(x: int = 3, y: MyType = None) -> float: ... class MyType: ... foo_y_type = foo.__annotations__['y'] As we see here, annotations are available at runtime through an ``__annotations__`` attribute on functions, classes, and modules. When annotations are specified on one of these objects, ``__annotations__`` is a dictionary mapping the names of the fields to the value specified as that field's annotation. The default behavior in Python 3.9 is to evaluate the expressions for the annotations, and build the annotations dict, at the time the function, class, or module is bound. At runtime the above code actually works something like this:: annotations = {'x': int, 'y': MyType, 'return': float} def foo(x = 3, y = "abc"): ... foo.__annotations__ = annotations class MyType: ... foo_y_type = foo.__annotations__['y'] The crucial detail here is that the values ``int``, ``MyType``, and ``float`` are looked up at the time the function object is bound, and these values are stored in the annotations dict. But this code doesn't run—it throws a ``NameError`` on the first line, because ``MyType`` hasn't been defined yet. PEP 563's solution is to decompile the expressions back into strings, and store those *strings* in the annotations dict. The equivalent runtime code would look something like this:: annotations = {'x': 'int', 'y': 'MyType', 'return': 'float'} def foo(x = 3, y = "abc"): ... foo.__annotations__ = annotations class MyType: ... foo_y_type = foo.__annotations__['y'] This code now runs successfully. However, ``foo_y_type`` is no longer a reference to ``MyType``, it is the *string* ``'MyType'``. The code would have to be further modified to call ``eval()`` or ``typing.get_type_hints()`` to convert the string into a useful reference to the actual ``MyType`` object. This PEP proposes a third approach, delaying the evaluation of the annotations by computing them in their own function. If this PEP was active, the generated code would work something like this:: class function: @property # __annotations__ on a function object is already a # "data descriptor", we're just changing what it does def __annotations__(self): return self.__co_annotations__() # ... def foo_annotations_fn(): return {'x': int, 'y': MyType, 'return': float} def foo(x = 3, y = "abc"): ... foo.__co_annotations__ = foo_annotations_fn class MyType: ... foo_y_type = foo.__annotations__['y'] The important change is that the code constructing the annotations dict now lives in a function—here, called `` foo_annotations__fn()``. But this function isn't called until we ask for the value of ``foo.__annotations__``, and we don't do that until *after* the definition of ``MyType``. So this code also runs successfully, and ``foo_y_type`` now has the correct value, the class ``MyType``. Motivation ========== Python's original semantics for annotations made its use for static type analysis painful due to forward reference problems. This was the main justification for PEP 563, and we need not revisit those arguments here. However, PEP 563's solution was to de-compile code for Python annotations back into strings at compile time, requiring users of annotations to ``eval()`` those strings to turn them back into Python values. This has several drawbacks: * It requires Python implementations to stringize their annotations. This is surprising—unprecedented behavior for a language-level feature. Also, adding this feature to CPython was complicated, and this complicated code would need to be reimplemented independently by every other Python implementation. * It requires a code change every time existing code uses an annotation, to handle converting the stringized annotation back into a useful value. * ``eval()`` is slow. * ``eval()`` isn't always available; it's sometimes removed from Python for space reasons. * In order to evaluate the annotations stored with a class, it requires obtaining a reference to that class's globals, which PEP 563 suggests should be done by looking up that class by name in ``sys.modules``—another surprising requirement for a language-level feature. * It adds an ongoing maintenance burden to Python implementations. Every time the language adds a new feature available in expressions, the implementation's stringizing code must be updated in tandem to support decompiling it. This PEP also solves the forward reference problem outlined in PEP 563 while avoiding the problems listed above: * Python implementations would generate annotations as code objects. This is simpler than stringizing, and is something Python implementations are already quite good at. This means: * alternate implementations would need to write less code to implement this feature, and * the implementation would be simpler overall, which should reduce its ongoing maintenance cost. * Code examining annotations at runtime would no longer need to use ``eval()`` or anything else—it would automatically get the correct values. This is easier, almost certainly faster, and removes the dependency on ``eval()``. Backwards Compatibility ======================= PEP 563 changed the semantics of annotations. When its semantics are active, annotations must assume they will be evaluated in *module-level* scope. They may no longer refer directly to local variables or class attributes. This PEP retains that semantic change, also requiring that annotations be evaluated in *module-level* scope. Thus, code changed so its annotations are compatible with PEP 563 should *already* compatible with this aspect of this PEP and would not need further change. Modules still using stock semantics would have to be revised so its annotations evaluate properly in module-level scope, in the same way they would have to be to achieve compatibility with PEP 563. PEP 563 also requires using ``eval()`` or ``typing.get_type_hints()`` to examine annotations. Code updated to work with PEP 563 that calls ``eval()`` directly would have to be updated simply to remove the ``eval()`` call. Code using ``typing.get_type_hints()`` would continue to work unchanged, though future use of that function would become optional in most cases. Because this PEP makes the same backwards-compatible change to annotation scoping as PEP 563, this PEP will be initially gated with a per-module ``from __future__ import co_annotations`` before it eventually becomes the default behavior. Apart from these two changes already discussed: * the evaluation of values in annotation dicts will be delayed until the ``__annotations__`` attribute is evaluated, and * annotations are now evaluated in module-level scope, this PEP preserves nearly all existing behavior of annotations dicts. Specifically: * Annotations dicts are mutable, and any changes to them are preserved. * The ``__annotations__`` attribute can be explicitly set, and any value set this way will be preserved. * The ``__annotations__`` attribute can be deleted using the ``del`` statement. However, there are two uncommon interactions possible with class and module annotations that work today—both with stock semantics, and with PEP 563 semantics—that would no longer work when this PEP was active. These two interactions would have to be prohibited. The good news is, neither is common, and neither is considered good practice. In fact, they're rarely seen outside of Python's own regression test suite. They are: * *Code that sets annotations from inside any kind of flow control statement.* It's currently possible to set module and class attributes with annotations inside an ``if`` or ``try`` statement, and it works as one would expect. It's untenable to support this behavior when this PEP is active. * *Code in module or class scope that references or modifies the local* ``__annotations__`` *dict directly.* Currently, when setting annotations on module or class attributes, the generated code simply creates a local ``__annotations__`` dict, then sets mappings in it as needed. It's also possible for user code to directly modify this dict, though this doesn't seem like it's an intentional feature. Although it'd be possible to support this after a fashion when this PEP was active, the semantics would likely be surprising and wouldn't make anyone happy. Note that these are both also pain points for static type checkers, and are unsupported by those checkers. It seems reasonable to declare that both are at the very least unsupported, and their use results in undefined behavior. It might be worth making a small effort to explicitly prohibit them with compile-time checks. There's one more idiom that's actually somewhat common when dealing with class annotations, and which will become more problematic when this PEP is active: code often accesses class annotations via ``cls.__dict__.get("__annotations__", {})`` rather than simply ``cls.__annotations__``. It's due to a flaw in the original design of annotations themselves. This topic will be examined in a separate discussion; the outcome of that discussion will likely guide the future evolution of this PEP. Mistaken Rejection Of This Approach In November 2017 ==================================================== During the early days of discussion around PEP 563, using code to delay the evaluation of annotations was briefly discussed, in a November 2017 thread in ``comp.lang.python-dev``. At the time the technique was termed an "implicit lambda expression". Guido van Rossum—Python's BDFL at the time—replied, asserting that these "implicit lambda expression" wouldn't work, because they'd only be able to resolve symbols at module-level scope: IMO the inability of referencing class-level definitions from annotations on methods pretty much kills this idea. https://mail.python.org/pipermail/python-dev/2017-November/150109.html This led to a short discussion about extending lambda-ized annotations for methods to be able to refer to class-level definitions, by maintaining a reference to the class-level scope. This idea, too, was quickly rejected. PEP 563 summarizes the above discussion here: https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-functi… What's puzzling is PEP 563's own changes to the scoping rules of annotations—it *also* doesn't permit annotations to reference class-level definitions. It's not immediately clear why an inability to reference class-level definitions was enough to reject using "implicit lambda expressions" for annotations, but was acceptable for stringized annotations. In retrospect there was probably a pivot during the development of PEP 563. It seems that, early on, there was a prevailing assumption that PEP 563 would support references to class-level definitions. But by the time PEP 563 was finalized, this assumption had apparently been abandoned. And it looks like "implicit lambda expressions" were never reconsidered in this new light. PEP 563 semantics have shipped in three major Python releases. These semantics are now widely used in organizations depending on static type analysis. Evaluating annotations at module-level scope is clearly acceptable to all interested parties. Therefore delayed evaluation of annotations with code using the same scoping rules is obviously also completely viable. .. _Implementation: Implementation ============== There's a prototype implementation of this PEP, here: https://github.com/larryhastings/co_annotations/ As of this writing, all features described in this PEP are implemented, and there are some rudimentary tests in the test suite. There are still some broken tests, and the repo is many months behind. from __future__ import co_annotations ------------------------------------- In the prototype, the semantics presented in this PEP are gated with: from __future__ import co_annotations __co_annotations__ ------------------ Python supports runtime metadata for annotations for three different types: function, classes, and modules. The basic approach to implement this PEP is much the same for all three with only minor variations. With this PEP, each of these types adds a new attribute, ``__co_annotations__``, with the following semantics: * ``__co_annotations__`` is always set, and may contain either ``None`` or a callable. * ``__co_annotations__`` cannot be deleted. * ``__annotations__`` and ``__co_annotations__`` can't both be set to a useful value simultaneously: * If you set ``__annotations__`` to a dict, this also sets ``__co_annotations__`` to None. * If you set ``__co_annotations__`` to a callable, this also deletes ``__annotations__`` Internally, ``__co_annotations__`` is a "data descriptor", where functions are called whenever user code gets, sets, or deletes the attribute. In all three cases, the object has a separate internal place to store the current value of the ``__co_annotations__`` attribute. ``__annotations__`` is also reimplemented as a data descriptor, with its own separate internal storage for its internal value. The code implementing the "get" for ``__annotations__`` works something like this:: if (the internal value is set) return the internal annotations dict if (__co_annotations__ is not None) call the __co_annotations__ function if the result is a dict: store the result as the internal value set __co_annotations__ to None return the internal value do whatever this object does when there are no annotations Unbound code objects -------------------- When Python code defines one of these three objects with annotations, the Python compiler generates a separate code object which builds and returns the appropriate annotations dict. The "annotation code object" is then stored *unbound* as the internal value of ``__co_annotations__``; it is then bound on demand when the user asks for ``__annotations__``. This is an important optimization, for both speed and memory consumption. Python processes rarely examine annotations at runtime. Therefore, pre-binding these code objects to function objects would be a waste of resources in nearly all cases. Note that user code isn't permitted to see these unbound code objects. If the user gets the value of ``__co_annotations__``, and the internal value of ``__co_annotations__`` is an unbound code object, it is bound, and the resulting function object is stored as the new value of ``__co_annotations__``. The annotations function ------------------------ Annotations functions take no arguments and must return a dict (or subclass of dict). The bytecode generated for annotations code objects always uses the ``BUILD_CONST_KEY_MAP`` opcode to build the dict. Stock and PEP 563 semantics only uses this bytecode for function annotations; for class and module annotations, they generate a longer and slightly-less-efficient stanza of bytecode. Also, when generating the bytecode for an annotations code object, all ``LOAD_*`` opcodes are forced to be ``LOAD_GLOBAL``. Function Annotations -------------------- When compiling a function, the CPython bytecode compiler visits the annotations for the function all in one place, starting with ``compiler_visit_annotations()``. If there are any annotations, they create the scope for the annotations function on demand, and ``compiler_visit_annotations()`` assembles it. The code object is passed in in place of the annotations dict for the ``MAKE_FUNCTION`` bytecode. ``MAKE_FUNCTION`` supports a new bit in its oparg bitfield, ``0x10``, which tells it to expect a ``co_annotations`` code object on the stack. The bitfields for ``annotations`` (``0x04``) and ``co_annotations`` (``0x10``) are mutually exclusive. When binding an unbound annotation code object, a function will use its own ``__globals__`` as the new function's globals. One quirk of Python: you can't actually remove the annotations from a function object. If you delete the ``__annotations__`` attribute of a function, then get its ``__annotations__`` member, it will create an empty dict and use that as its ``__annotations__``. Naturally the implementation of this PEP maintains this quirk. Class Annotations ----------------- When compiling a class body, the compiler maintains two scopes: one for the normal class body code, and one for annotations. (This is facilitated by four new functions: ``compiler.c`` adds ``compiler_push_scope()`` and ``compiler_pop_scope()``, and ``symtable.c`` adds ``symtable_push_scope()`` and ``symtable_pop_scope()``.) Once the code generator reaches the end of the class body, but before it generates the bytecode for the class body, it assembles the bytecode for ``__co_annotations__``, then assigns that to ``__co_annotations__`` using ``STORE_NAME``. It also sets a new ``__globals__`` attribute. Currently it does this by calling ``globals()`` and storing the result. (Surely there's a more elegant way to find the class's globals--but this was good enough for the prototype.) When binding an unbound annotation code object, a class will use the value of this ``__globals__`` attribute. When the class drops its reference to the unbound code object--either because it has bound it to a function, or because ``__annotations__`` has been explicitly set--it also deletes its ``__globals__`` attribute. As discussed above, examination / modification of ``__annotations__`` from within the class body is no longer supported. Also, any flow control (``if`` / ``try``) around declarations of members with annotations is unsupported. If you delete the ``__annotations__`` attribute of a class, then get its ``__annotations__`` member, it will return the annotations dict of the first base class with annotations set. If no base classes have annotations set, it will raise ``AttributeError``. Although it's an implementation-specific detail, currently classes store the internal value of ``__co_annotations__`` in their ``tp_dict`` under the same name. Module Annotations ------------------ Module annotations work much the same as class annotations. The main difference is, a module uses its own dict as the ``__globals__`` when binding the function. If you delete the ``__annotations__`` attribute of a class, then get its ``__annotations__`` member, the module will raise ``AttributeError``. Interactive REPL Shell ---------------------- Everything works the same inside Python's interactive REPL shell, except for module annotations in the interactive module (``__main__``) itself. Since that module is never "finished", there's no specific point where we can compile the ``__co_annotations__`` function. For the sake of simplicity, in this case we forego delayed evaluation. Module-level annotations in the REPL shell will continue to work exactly as they do today, evaluating immediately and setting the result directly inside the ``__annotations__`` dict. (It might be possible to support delayed evaluation here. But it gets complicated quickly, and for a nearly-non-existent use case.) Local Annotations Inside Functions ---------------------------------- Python supports syntax for local variable annotations inside functions. However, these annotations have no runtime effect. Thus this PEP doesn't need to do anything to support them. Performance ----------- Performance with this PEP should be favorable. In general, resources are only consumed on demand—"you only pay for what you use". There are three scenarios to consider: * the runtime cost when annotations aren't defined, * the runtime cost when annotations are defined but *not* referenced, and * the runtime cost when annotations are defined *and* referenced. We'll examine each of these scenarios in the context of all three semantics for annotations: stock, PEP 563, and this PEP. When there are no annotations, all three semantics have the same runtime cost: zero. No annotations dict is created and no code is generated for it. This requires no runtime processor time and consumes no memory. When annotations are defined but not referenced, the runtime cost of Python with this PEP should be slightly faster than either original Python semantics or PEP 563 semantics. With those, the annotations dicts are built but never examined; with this PEP, the annotations dicts won't even be built. All that happens at runtime is the loading of a single constant (a simple code object) which is then set as an attribute on an object. Since the annotations are never referenced, the code object is never bound to a function, the code to create the dict is never executed, and the dict is never constructed. When annotations are both defined and referenced, code using this PEP should be much faster than code using PEP 563 semantics, and roughly the same as original Python semantics. PEP 563 semantics requires invoking ``eval()`` for every value inside an annotations dict, which is much slower. And, as already mentioned, this PEP generates more efficient bytecode for class and module annotations than either stock or PEP 563 semantics. Memory use should also be comparable in all three scenarios across all three semantic contexts. In the first and third scenarios, memory usage should be roughly equivalent in all cases. In the second scenario, when annotations are defined but not referenced, using this PEP's semantics will mean the function/class/module will store one unused code object; with the other two semantics, they'll store one unused dictionary. For Future Discussion ===================== __globals__ ----------- Is it permissable to add the ``__globals__`` reference to class objects as proposed here? It's not clear why this hasn't already been done; PEP 563 could have made use of class globals, but instead makes do with looking up classes inside ``sys.modules``. Yet Python seems strangely allergic to adding a ``__globals__`` reference to class objects. If adding ``__globals__`` to class objects is indeed a bad idea (for reasons I don't know), here are two alternatives as to how classes could get a reference to their globals for the implementation of this PEP: * The generate code for a class could bind its annotations code object to a function at the time the class is bound, rather than waiting for ``__annotations__`` to be referenced, making them an exception to the rule (even though "special cases aren't special enough to break the rules"). This would result in a small additional runtime cost when annotations were defined but not referenced on class objects. Honestly I'm more worried about the lack of symmetry in semantics. (But I wouldn't want to pre-bind all annotations code objects, as that would become much more costly for function objects, even as annotations are rarely used at runtime.) * Use the class's ``__module__`` attribute to look up its module by name in ``sys.modules``. This is what PEP 563 advises. While this is passable for userspace or library code, it seems like a little bit of a code smell for this to be defined semantics baked into the language itself. Also, the prototype gets globals for class objects by calling ``globals()`` then storing the result. I'm sure there's a much faster way to do this, I just didn't know what it was when I was prototyping. I'm sure we can revise this to something much faster and much more sanitary. I'd prefer to make it completely internal anyway, and not make it visible to the user (via this new __globals__ attribute). There's possibly already a good place to put it anyway--``ht_module``. Bikeshedding the name --------------------- During most of the development of this PEP, user code actually could see the raw annotation code objects. ``__co_annotations__`` could only be set to a code object; functions and other callables weren't permitted. In that context the name ``co_annotations`` makes a lot of sense. But with this last-minute pivot where ``__co_annotations__`` now presents itself as a callable, perhaps the name of the attribute and the name of the ``from __future__ import`` needs a re-think. Acknowledgements ================ Thanks to Barry Warsaw, Eric V. Smith, and Mark Shannon for feedback and encouragement. Thanks in particular to Mark Shannon for two key suggestions—build the entire annotations dict inside a single code object, and only bind it to a function on demand—that quickly became among the best aspects of this proposal. Copyright ========= This document is placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

14 84

Comments on PEP 558
by Mark Shannon Feb. 8, 2021

Feb. 8, 2021

Hi Nick, Regarding `f_locals` PEP 558 states: """ Instead of being a direct reference to the internal dynamic snapshot used to populate the independent snapshots returned by locals(), frame.f_locals will be updated to instead return a dedicated proxy type (implemented as a private subclass of the existing types.MappingProxyType) that has two internal attributes not exposed as part of the Python runtime API: * mapping: an implicitly updated snapshot of the function local variables and closure references, as well as any arbitrary items that have been set via the mapping API, even if they don't have storage allocated for them on the underlying frame * frame: the underlying frame that the snapshot is for """ This seems rather complex, and consequently fragile. I fear that this is just going to result in different bugs, rather than fixing the bugs it proposes to fix. Why not just make `f_local` a direct view on the underlying frame? It would be simpler to understand, more robust, and should perform better. Cheers, Mark.

3 13

Dusting off PEP 533?
by Paul Bryan Feb. 8, 2021

Feb. 8, 2021

I've now experienced an issue I believe that PEP 533 was intended to address: When an asynchronous context manager is created within an asynchronous generator, if the generator is not iterated fully, the context manager will not exit until the event loop cancels the task by raising a CancelledError, long after the context manager is assumed to be out of scope. Per PEP 525, I can call aclose coroutine method to cleanup the generator, but it requires the code iterating to be aware that that closing the generator is necessary. Any appetite for putting PEP 533 back on the table to address this kind of issue?

4 4

PR review request: shutil permission errors
by Winson Luk Feb. 6, 2021

Feb. 6, 2021

Hi folks, I've had an open PR for a few weeks that still needs a review: https://github.com/python/cpython/pull/24001 This PR fixes a shutil.move() bug when permission to a directory is denied. Thanks, Winson

1 1

PEP 624: Remove Py_UNICODE encoder APIs
by Inada Naoki Feb. 4, 2021

Feb. 4, 2021

Hi, folks. Since the previous discussion was suspended without consensus, I wrote a new PEP for it. (Thank you Victor for reviewing it!) This PEP looks very similar to PEP 623 "Remove wstr from Unicode", but for encoder APIs, not for Unicode object APIs. URL (not available yet): https://www.python.org/dev/peps/pep-0624/ --- PEP: 624 Title: Remove Py_UNICODE encoder APIs Author: Inada Naoki <songofacandy(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 06-Jul-2020 Python-Version: 3.11 Abstract ======== This PEP proposes to remove deprecated ``Py_UNICODE`` encoder APIs in Python 3.11: * ``PyUnicode_Encode()`` * ``PyUnicode_EncodeASCII()`` * ``PyUnicode_EncodeLatin1()`` * ``PyUnicode_EncodeUTF7()`` * ``PyUnicode_EncodeUTF8()`` * ``PyUnicode_EncodeUTF16()`` * ``PyUnicode_EncodeUTF32()`` * ``PyUnicode_EncodeUnicodeEscape()`` * ``PyUnicode_EncodeRawUnicodeEscape()`` * ``PyUnicode_EncodeCharmap()`` * ``PyUnicode_TranslateCharmap()`` * ``PyUnicode_EncodeDecimal()`` * ``PyUnicode_TransformDecimalToASCII()`` .. note:: `PEP 623 <https://www.python.org/dev/peps/pep-0623/>`_ propose to remove Unicode object APIs relating to ``Py_UNICODE``. On the other hand, this PEP is not relating to Unicode object. These PEPs are split because they have different motivation and need different discussion. Motivation ========== In general, reducing the number of APIs that have been deprecated for a long time and have few users is a good idea for not only it improves the maintainability of CPython, but it also helps API users and other Python implementations. Rationale ========= Deprecated since Python 3.3 --------------------------- ``Py_UNICODE`` and APIs using it are deprecated since Python 3.3. Inefficient ----------- All of these APIs are implemented using ``PyUnicode_FromWideChar``. So these APIs are inefficient when user want to encode Unicode object. Not used widely --------------- When searching from top 4000 PyPI packages [1]_, only pyodbc use these APIs. * ``PyUnicode_EncodeUTF8()`` * ``PyUnicode_EncodeUTF16()`` pyodbc uses these APIs to encode Unicode object into bytes object. So it is easy to fix it. [2]_ Alternative APIs ================ There are alternative APIs to accept ``PyObject *unicode`` instead of ``Py_UNICODE *``. Users can migrate to them. ========================================= ========================================== Deprecated API Alternative APIs ========================================= ========================================== ``PyUnicode_Encode()`` ``PyUnicode_AsEncodedString()`` ``PyUnicode_EncodeASCII()`` ``PyUnicode_AsASCIIString()`` \(1) ``PyUnicode_EncodeLatin1()`` ``PyUnicode_AsLatin1String()`` \(1) ``PyUnicode_EncodeUTF7()`` \(2) ``PyUnicode_EncodeUTF8()`` ``PyUnicode_AsUTF8String()`` \(1) ``PyUnicode_EncodeUTF16()`` ``PyUnicode_AsUTF16String()`` \(3) ``PyUnicode_EncodeUTF32()`` ``PyUnicode_AsUTF32String()`` \(3) ``PyUnicode_EncodeUnicodeEscape()`` ``PyUnicode_AsUnicodeEscapeString()`` ``PyUnicode_EncodeRawUnicodeEscape()`` ``PyUnicode_AsRawUnicodeEscapeString()`` ``PyUnicode_EncodeCharmap()`` ``PyUnicode_AsCharmapString()`` \(1) ``PyUnicode_TranslateCharmap()`` ``PyUnicode_Translate()`` ``PyUnicode_EncodeDecimal()`` \(4) ``PyUnicode_TransformDecimalToASCII()`` \(4) ========================================= ========================================== Notes: (1) ``const char *errors`` parameter is missing. (2) There is no public alternative API. But user can use generic ``PyUnicode_AsEncodedString()`` instead. (3) ``const char *errors, int byteorder`` parameters are missing. (4) There is no direct replacement. But ``Py_UNICODE_TODECIMAL`` can be used instead. CPython uses ``_PyUnicode_TransformDecimalAndSpaceToASCII`` for converting from Unicode to numbers instead. Plan ==== Python 3.9 ---------- Add ``Py_DEPRECATED(3.3)`` to following APIs. This change is committed already [3]_. All other APIs have been marked ``Py_DEPRECATED(3.3)`` already. * ``PyUnicode_EncodeDecimal()`` * ``PyUnicode_TransformDecimalToASCII()``. Document all APIs as "will be removed in version 3.11". Python 3.11 ----------- These APIs are removed. * ``PyUnicode_Encode()`` * ``PyUnicode_EncodeASCII()`` * ``PyUnicode_EncodeLatin1()`` * ``PyUnicode_EncodeUTF7()`` * ``PyUnicode_EncodeUTF8()`` * ``PyUnicode_EncodeUTF16()`` * ``PyUnicode_EncodeUTF32()`` * ``PyUnicode_EncodeUnicodeEscape()`` * ``PyUnicode_EncodeRawUnicodeEscape()`` * ``PyUnicode_EncodeCharmap()`` * ``PyUnicode_TranslateCharmap()`` * ``PyUnicode_EncodeDecimal()`` * ``PyUnicode_TransformDecimalToASCII()`` Alternative ideas ================= Instead of just removing deprecated APIs, we may be able to use thier names with different signature. Make some private APIs public ------------------------------ ``PyUnicode_EncodeUTF7()`` doesn't have public alternative APIs. Some APIs have alternative public APIs. But they are missing ``const char *errors`` or ``int byteorder`` parameters. We can rename some private APIs and make them public to cover missing APIs and parameters. ============================= ================================ Rename to Rename from ============================= ================================ ``PyUnicode_EncodeASCII()`` ``_PyUnicode_AsASCIIString()`` ``PyUnicode_EncodeLatin1()`` ``_PyUnicode_AsLatin1String()`` ``PyUnicode_EncodeUTF7()`` ``_PyUnicode_EncodeUTF7()`` ``PyUnicode_EncodeUTF8()`` ``_PyUnicode_AsUTF8String()`` ``PyUnicode_EncodeUTF16()`` ``_PyUnicode_EncodeUTF16()`` ``PyUnicode_EncodeUTF32()`` ``_PyUnicode_EncodeUTF32()`` ============================= ================================ Pros: * We have more consistent API set. Cons: * We have more public APIs to maintain. * Existing public APIs are enough for most use cases, and ``PyUnicode_AsEncodedString()`` can be used in other cases. Replace ``Py_UNICODE*`` with ``Py_UCS4*`` ----------------------------------------- We can replace ``Py_UNICODE`` (typedef of ``wchar_t``) with ``Py_UCS4``. Since builtin codecs support UCS-4, we don't need to convert ``Py_UCS4*`` string to Unicode object. Pros: * We have more consistent API set. * User can encode UCS-4 string in C without creating Unicode object. Cons: * We have more public APIs to maintain. * Applications which uses UTF-8 or UTF-32 can not use these APIs anyway. * Other Python implementations may not have builtin codec for UCS-4. * If we change the Unicode internal representation to UTF-8, we need to keep UCS-4 support only for these APIs. Replace ``Py_UNICODE*`` with ``wchar_t*`` ----------------------------------------- We can replace ``Py_UNICODE`` to ``wchar_t``. Pros: * We have more consistent API set. * Backward compatible. Cons: * We have more public APIs to maintain. * They are inefficient on platforms ``wchar_t*`` is UTF-16. It is because built-in codecs supports only UCS-1, UCS-2, and UCS-4 input. Rejected ideas ============== Using runtime warning --------------------- These APIs doesn't release GIL for now. Emitting a warning from such APIs is not safe. See this example. .. code-block:: PyObject *u = PyList_GET_ITEM(list, i); // u is borrowed reference. PyObject *b = PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(u), PyUnicode_GET_SIZE(u), NULL); // Assumes u is still living reference. PyObject *t = PyTuple_Pack(2, u, b); Py_DECREF(b); return t; If we emit Python warning from ``PyUnicode_EncodeUTF8()``, warning filters and other threads may change the ``list`` and ``u`` can be a dangling reference after ``PyUnicode_EncodeUTF8()`` returned. Additionally, since we are not changing behavior but removing C APIs, runtime ``DeprecationWarning`` might not helpful for Python developers. We should warn to extension developers instead. Discussions =========== * `Plan to remove Py_UNICODE APis except PEP 623 <https://mail.python.org/archives/list/python-dev@python.org/thread/S7KW2U6I…>`_ * `bpo-41123: Remove Py_UNICODE APIs except PEP 623: <https://bugs.python.org/issue41123>`_ References ========== .. [1] Source package list chosen from top 4000 PyPI packages. (https://github.com/methane/notes/blob/master/2020/wchar-cache/package_list.…) .. [2] pyodbc -- Don't use PyUnicode_Encode API #792 (https://github.com/mkleehammer/pyodbc/pull/792) .. [3] Uncomment Py_DEPRECATED for Py_UNICODE APIs (GH-21318) (https://github.com/python/cpython/commit/9c3840870814493fed62e140cfa43c2883…) Copyright ========= This document has been placed in the public domain. -- Inada Naoki <songofacandy(a)gmail.com>

9 27

Upcoming 3.7.10 and 3.6.13 Security Releases
by Ned Deily Feb. 3, 2021

Feb. 3, 2021

We are planning to produce the next security-fix rollup releases for Python 3.7.x and 3.6.x on 2021-01-15. The most recent releases for these versions were on 2020-08-17. There has not been a lot of activity for either branch since then. Core developers: if you know of any additional security issues that should be addressed in these releases, please mark the relevant bpo issues as "release blocker" and, if possible, submit PRs for review prior to the end of 2021-01-14 AOE. Thanks! -- Ned Deily nad(a)python.org -- []

5 6

Why aren't we allowing the use of C11?
by Mark Shannon Feb. 1, 2021

Feb. 1, 2021

Hi everyone, PEP 7 says that C code should conform to C89 with a subset of C99 allowed. It's 2021 and all the major compilers support C11 (ignoring the optional parts). C11 has support for thread locals, static asserts, and anonymous structs and unions. All useful features. Is there a good reason not to start using C11 now? Cheers, Mark.

14 17

More comments on PEP 558
by Mark Shannon Jan. 30, 2021

Jan. 30, 2021

Hi Nick, A couple more issues with PEP 558, as I see it. Lack of specification --------------------- There is no specification section in PEP 558. Much of PEP 558 describes the differences between the current behavior and what PEP 558 proposes. These differences are often intermingled with a discussion of the flaws with the current behavior, which makes it difficult to understand what the proposed behavior is. I could not implement PEP 558 from the PEP alone. Cycles and the cost of maintaining the f_locals cache ----------------------------------------------------- PEP 558 proposes that f_locals acts as a proxy mapping to the locals in a frame *and* that the frame caches the proxy. This obviously creates a cycle, which needs to be broken by the cycle GC. Although the cycle only exists if f_locals is actually used, there is still some overhead for *every* call and return, as the f_locals field needs to be initialized on call and checked for non-NULL on return. Cheers, Mark.

2 1

What is the pyclbr public API?
by Terry Reedy Jan. 30, 2021

Jan. 30, 2021

pyclbr is the stdlib module browser (once just class browser, hence the name). The doc https://docs.python.org/3/library/pyclbr.html#module-pyclbr, which I revised in 2017, documents readline and readline_ex as the public call interface. The functions return a hierarchical structure that includes Function and Class instances. (Both subclass pyclbr._Object.) The doc I revised already omitted signatures for these classes and just listed the attributes of instances. Just before that listing, I added "Users are not expected to create instances of these classes." https://bugs.python.org/issue38307 and https://github.com/python/cpython/pull/24348 propose to add an end_lineno attribute. Since pyclbr is now, since last November, based on the ast tree and since the relevant ast nodes have an end_line attribute, the proposal amounts to copying that attribute, along with others, instead of ignoring it. The patch proposes to add 'end_lineno' after existing start 'lineno' in the _Object, Function, and Class signatures. I prefer doing this, rather than adding the new parameter at the end of the list, because a) being the the logical place would make the code easier to read, and b) new names as the end of the signature, follow optional arguments, would have to be optional, whereas My question is whether the omission of signatures and my added sentence sufficiently defines the signatures of these classes (in particular, the argument order) as private to approve the patch as is? -- Terry Jan Reedy

2 7

Improve CPython tracing performance
by Fabio Zadrozny Jan. 30, 2021

Jan. 30, 2021

Hi all, Right now, when a debugger is active, the number of local variables can affect the tracing speed quite a lot. For instance, having tracing setup in a program such as the one below takes 4.64 seconds to run, yet, changing all the variables to have the same name -- i.e.: change all assignments to `a = 1` (such that there's only a single variable in the namespace), it takes 1.47 seconds (in my machine)... the higher the number of variables, the slower the tracing becomes. ``` import time t = time.time() def call(): a = 1 b = 1 c = 1 d = 1 e = 1 f = 1 def noop(frame, event, arg): return noop import sys sys.settrace(noop) for i in range(1_000_000): call() print('%.2fs' % (time.time() - t,)) ``` This happens because `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are called inside the `call_trampoline` ( https://github.com/python/cpython/blob/master/Python/sysmodule.c#L946). So, I'd like to simply remove those calls. Debuggers can call `PyFrame_LocalsToFast` when needed -- otherwise mutating non-current frames doesn't work anyways. As a note, pydevd already has such a call: https://github.com/fabioz/PyDev.Debugger/blob/0d4d210f01a1c0a8647178b2e665b… and PyPy also has a counterpart. As for `PyFrame_FastToLocalsWithError`, I don't really see any reason to call it at all. i.e.: something as the code below prints the `a` variable from the `main()` frame regardless of that and I checked all pydevd tests and nothing seems to be affected (it seems that accessing f_locals already does this: https://github.com/python/cpython/blob/cb9879b948a19c9434316f8ab6aba9c4601a…, so, I don't see much reason to call it at all). ``` def call(): import sys frame = sys._getframe() print(frame.f_back.f_locals) def main(): a = 1 call() if __name__ == '__main__': main() ``` Does anyone see any issue with this? If it's non controversial, is a PEP needed or just an issue to track it would be enough to remove those 2 lines? Thanks, Fabio

3 9

Summary of Python tracker Issues
by Python tracker Jan. 29, 2021

Jan. 29, 2021

ACTIVITY SUMMARY (2021-01-22 - 2021-01-29) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7588 (+10) closed 47211 (+50) total 54799 (+60) Open issues with patches: 3032 Issues opened (40) ================== #33387: Simplify bytecodes for try-finally, try-except and with blocks https://bugs.python.org/issue33387 reopened by iritkatriel #36675: Doctest directives and comments missing from code samples https://bugs.python.org/issue36675 reopened by mdk #43006: Changed behaviour of inspect.signature() in Python 3.10 https://bugs.python.org/issue43006 opened by Zac Hatfield-Dodds #43007: set_wakeup_fd() only works in main thread https://bugs.python.org/issue43007 opened by MeneMeneTekel #43009: Port curses capi pointer array to a struct https://bugs.python.org/issue43009 opened by shihai1991 #43010: @functools.wraps and abc.abstractmethod interoperability https://bugs.python.org/issue43010 opened by erezinman #43012: Remove pathlib accessors https://bugs.python.org/issue43012 opened by barneygale #43013: IDLE: update code, mostly by cleanups of 2.x or 2to3 artifact https://bugs.python.org/issue43013 opened by terry.reedy #43015: Add str.replaceall? https://bugs.python.org/issue43015 opened by Nathaniel Manista #43016: Improve tests for curses https://bugs.python.org/issue43016 opened by serhiy.storchaka #43017: Improve error message in the parser when using un-parenthesise https://bugs.python.org/issue43017 opened by pablogsal #43019: wait_for(to_thread)) does not work as expected. Extra document https://bugs.python.org/issue43019 opened by synchronizing #43022: Unable to dynamically load functions from python3.dll https://bugs.python.org/issue43022 opened by paul.moore #43024: improve signature (in help, etc) for functions taking sentinel https://bugs.python.org/issue43024 opened by iritkatriel #43026: Missing words renders meaning unclear in fcntl.html https://bugs.python.org/issue43026 opened by EzraBC #43027: Calling _PyBytes_Resize() on 1-byte bytes may raise error https://bugs.python.org/issue43027 opened by malin #43029: unittest: Add assertUniqeIn https://bugs.python.org/issue43029 opened by rousseldenis #43030: signed/unsigned mismatch in Py_UNICODE_ISSPACE macro https://bugs.python.org/issue43030 opened by doko #43034: Python tutorial misleads users about floor division behavior https://bugs.python.org/issue43034 opened by jessevsilverman #43035: FileNotFoundError in distutils\file_util.py copy_tree https://bugs.python.org/issue43035 opened by knzivid #43036: TOS-behaviour documentation is inconsistent https://bugs.python.org/issue43036 opened by xmorel #43039: tempfile.TemporaryDirectory() name string is incorrect https://bugs.python.org/issue43039 opened by eosborne #43041: copying WeakValueDictionary is not iteration safe https://bugs.python.org/issue43041 opened by djromberg #43042: tutorial ambiguous about creation of local symbol table for re https://bugs.python.org/issue43042 opened by jessevsilverman #43043: Python tutorial could make rules for default argument definiti https://bugs.python.org/issue43043 opened by jessevsilverman #43045: Instructions for installing pip on Ubuntu/WSL2 is incomplete https://bugs.python.org/issue43045 opened by lennart.borgman #43046: argparse: capturing actions https://bugs.python.org/issue43046 opened by monkeyman79 #43047: logging.config formatters documentation is out of sync with co https://bugs.python.org/issue43047 opened by iwienand #43048: Printing RecursionError results in RecursionError https://bugs.python.org/issue43048 opened by vlad2 #43049: Use io.IncrementalNewlineDecoder for doctest newline conversio https://bugs.python.org/issue43049 opened by pdonis #43050: threading timer memory leak https://bugs.python.org/issue43050 opened by fengjiang #43052: _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0 https://bugs.python.org/issue43052 opened by isuruf #43053: Speed up math.isqrt, again https://bugs.python.org/issue43053 opened by juraj.sukop #43054: What does the existence of a struct in a header file imply abo https://bugs.python.org/issue43054 opened by Mark.Shannon #43056: Use of dicts in sections 4.2 and 4.7 of Python tutorial a bit https://bugs.python.org/issue43056 opened by jessevsilverman #43058: setting a logging Handler name https://bugs.python.org/issue43058 opened by bcohen #43059: sqlite3: Externally developed? https://bugs.python.org/issue43059 opened by user1347091 #43060: Convert _decimal C API from pointer array to struct https://bugs.python.org/issue43060 opened by erlendaasland #43062: Non-integer values in collections.Counter https://bugs.python.org/issue43062 opened by HuangFuSL #43063: zipfile.Path / importlib.resources raises KeyError if a file w https://bugs.python.org/issue43063 opened by The Compiler Most recent 15 issues with no replies (15) ========================================== #43062: Non-integer values in collections.Counter https://bugs.python.org/issue43062 #43060: Convert _decimal C API from pointer array to struct https://bugs.python.org/issue43060 #43058: setting a logging Handler name https://bugs.python.org/issue43058 #43056: Use of dicts in sections 4.2 and 4.7 of Python tutorial a bit https://bugs.python.org/issue43056 #43052: _dyld_shared_cache_contains_path needs SYSTEM_VERSION_COMPAT=0 https://bugs.python.org/issue43052 #43049: Use io.IncrementalNewlineDecoder for doctest newline conversio https://bugs.python.org/issue43049 #43047: logging.config formatters documentation is out of sync with co https://bugs.python.org/issue43047 #43045: Instructions for installing pip on Ubuntu/WSL2 is incomplete https://bugs.python.org/issue43045 #43043: Python tutorial could make rules for default argument definiti https://bugs.python.org/issue43043 #43042: tutorial ambiguous about creation of local symbol table for re https://bugs.python.org/issue43042 #43036: TOS-behaviour documentation is inconsistent https://bugs.python.org/issue43036 #43027: Calling _PyBytes_Resize() on 1-byte bytes may raise error https://bugs.python.org/issue43027 #43026: Missing words renders meaning unclear in fcntl.html https://bugs.python.org/issue43026 #43019: wait_for(to_thread)) does not work as expected. Extra document https://bugs.python.org/issue43019 #43016: Improve tests for curses https://bugs.python.org/issue43016 Most recent 15 issues waiting for review (15) ============================================= #43059: sqlite3: Externally developed? https://bugs.python.org/issue43059 #43049: Use io.IncrementalNewlineDecoder for doctest newline conversio https://bugs.python.org/issue43049 #43047: logging.config formatters documentation is out of sync with co https://bugs.python.org/issue43047 #43046: argparse: capturing actions https://bugs.python.org/issue43046 #43030: signed/unsigned mismatch in Py_UNICODE_ISSPACE macro https://bugs.python.org/issue43030 #43029: unittest: Add assertUniqeIn https://bugs.python.org/issue43029 #43024: improve signature (in help, etc) for functions taking sentinel https://bugs.python.org/issue43024 #43017: Improve error message in the parser when using un-parenthesise https://bugs.python.org/issue43017 #43016: Improve tests for curses https://bugs.python.org/issue43016 #43013: IDLE: update code, mostly by cleanups of 2.x or 2to3 artifact https://bugs.python.org/issue43013 #43009: Port curses capi pointer array to a struct https://bugs.python.org/issue43009 #42999: `pathlib.Path.link_to()` documentation is misleading https://bugs.python.org/issue42999 #42997: Improve error message for missing : before suites https://bugs.python.org/issue42997 #42995: Add PurePath.with_suffix_appended() https://bugs.python.org/issue42995 #42994: Missing MIME types for opus, AAC, 3gpp and 3gpp2 https://bugs.python.org/issue42994 Top 10 most discussed issues (10) ================================= #42973: argparse: mixing optional and positional arguments... not agai https://bugs.python.org/issue42973 17 msgs #3353: make built-in tokenizer available via Python C API https://bugs.python.org/issue3353 15 msgs #43013: IDLE: update code, mostly by cleanups of 2.x or 2to3 artifact https://bugs.python.org/issue43013 13 msgs #43022: Unable to dynamically load functions from python3.dll https://bugs.python.org/issue43022 12 msgs #42967: [security] urllib.parse.parse_qsl(): Web cache poisoning - `; https://bugs.python.org/issue42967 11 msgs #43046: argparse: capturing actions https://bugs.python.org/issue43046 7 msgs #43053: Speed up math.isqrt, again https://bugs.python.org/issue43053 7 msgs #43006: Changed behaviour of inspect.signature() in Python 3.10 https://bugs.python.org/issue43006 6 msgs #43059: sqlite3: Externally developed? https://bugs.python.org/issue43059 6 msgs #38307: Add .end_lineno attribute to pyclbr _Objects https://bugs.python.org/issue38307 5 msgs Issues closed (49) ================== #2636: Adding a new regex module (compatible with re) https://bugs.python.org/issue2636 closed by vstinner #23544: IDLE hangs when selecting Stack View with debug active https://bugs.python.org/issue23544 closed by terry.reedy #27772: Refer to actual format string when creating “zero padding” https://bugs.python.org/issue27772 closed by serhiy.storchaka #29076: Mac installer shell updater script silently fails if default s https://bugs.python.org/issue29076 closed by ned.deily #33289: tkinter askcolor returning floats for r, g, b values instead o https://bugs.python.org/issue33289 closed by serhiy.storchaka #36086: Split IDLE into separate feature in Windows installer https://bugs.python.org/issue36086 closed by rhettinger #36379: nb_inplace_pow is always called with an invalid argument https://bugs.python.org/issue36379 closed by gregory.p.smith #38250: enum.Flag should be more set-like https://bugs.python.org/issue38250 closed by ethan.furman #40745: Typo in library/typing https://bugs.python.org/issue40745 closed by iritkatriel #41459: pickle.load raises SystemError on malformed input https://bugs.python.org/issue41459 closed by eric.smith #41798: [C API] Revisit usage of the PyCapsule C API with multi-phase https://bugs.python.org/issue41798 closed by shihai1991 #42300: Typo in translation to portuguese https://bugs.python.org/issue42300 closed by iritkatriel #42383: Pdb does not correclty restart the target if it changes the cu https://bugs.python.org/issue42383 closed by gvanrossum #42384: Inconsistent sys.path between python and pdb https://bugs.python.org/issue42384 closed by gvanrossum #42843: What min_sphinx for Python 3.10 https://bugs.python.org/issue42843 closed by mdk #42869: pydoc does not append .html to documentation https://bugs.python.org/issue42869 closed by mdk #42901: [Enum] move member creation to __set_name__ in order to suppor https://bugs.python.org/issue42901 closed by ethan.furman #42915: enum.Flag ~ bitwise negation is very slow and can't be defined https://bugs.python.org/issue42915 closed by ethan.furman #42948: bytearray.copy is undocumented https://bugs.python.org/issue42948 closed by wim.glenn #42955: Add sys.stdlib_module_names: list of stdlib module names (Pyth https://bugs.python.org/issue42955 closed by vstinner #42957: os.readlink produces wrong result on windows https://bugs.python.org/issue42957 closed by steve.dower #42977: Tkinter Optionmenu Too Narrow on Mac https://bugs.python.org/issue42977 closed by serhiy.storchaka #42979: _zoneinfo: zoneinfomodule_exec() doesn't check for PyDateTime_ https://bugs.python.org/issue42979 closed by vstinner #42985: AMD64 Arch Linux Asan 3.x fails: command timed out: 1200 secon https://bugs.python.org/issue42985 closed by vstinner #42992: Tkinter bbox coordinates incorrectly drawn https://bugs.python.org/issue42992 closed by terry.reedy #43002: Exception chaining accepts exception classes https://bugs.python.org/issue43002 closed by steven.daprano #43003: Parts of the API will be removed in Python 4.0 ? https://bugs.python.org/issue43003 closed by terry.reedy #43004: No type variables left in collections.abc.Callable https://bugs.python.org/issue43004 closed by tyler.yep #43005: Package Relative Imports - double dot doc example not working https://bugs.python.org/issue43005 closed by gvanrossum #43008: IDLE ignores sys.excepthook in normal, subprocess mode https://bugs.python.org/issue43008 closed by terry.reedy #43011: DeprecationWarnings in test_ctypes https://bugs.python.org/issue43011 closed by benjamin.peterson #43014: tokenize spends a lot of time in `re.compile(...)` https://bugs.python.org/issue43014 closed by BTaskaya #43018: unwanted label showing up in ttk.LabeledScale https://bugs.python.org/issue43018 closed by serhiy.storchaka #43020: str.lower method with "İ" character https://bugs.python.org/issue43020 closed by steven.daprano #43021: Unpacking tuple argument in combination with inline if stateme https://bugs.python.org/issue43021 closed by steven.daprano #43023: Remove a redundant check in _PyBytes_Resize() https://bugs.python.org/issue43023 closed by malin #43025: Use normal 'i' character to denote imaginary part of complex n https://bugs.python.org/issue43025 closed by eric.smith #43028: seeking past the end of a file unexpected behavior https://bugs.python.org/issue43028 closed by cotton.seed #43031: aarch64 RHEL7 LTO + PGO 3.7: "make" hangs when running test_as https://bugs.python.org/issue43031 closed by vstinner #43032: Size of pie chart in matplotlib (frame affects it) https://bugs.python.org/issue43032 closed by mark.dickinson #43033: Incorrect handling of PyObject_SetAttrString() in the _zoneinf https://bugs.python.org/issue43033 closed by serhiy.storchaka #43037: Importlib reload by module name (String) https://bugs.python.org/issue43037 closed by brett.cannon #43038: ensurepip: tries to use setup.py/setup.cfg https://bugs.python.org/issue43038 closed by ned.deily #43040: random.py randrange() is very slow if the range is a power of https://bugs.python.org/issue43040 closed by rhettinger #43044: Python 2.7 documentation links to 404 pages when the library w https://bugs.python.org/issue43044 closed by vstinner #43051: Pip Permissions Error overwriting pip directory https://bugs.python.org/issue43051 closed by steve.dower #43055: Inconsistent behaviour when using walrus operator with 'and'/' https://bugs.python.org/issue43055 closed by eric.smith #43057: Timezone 'Etc/GMT-5' functions as Timezone 'Etc/GMT+5' https://bugs.python.org/issue43057 closed by mark.dickinson #43061: subprocess: feature request: Get only the stdout of the last s https://bugs.python.org/issue43061 closed by r.david.murray

1 0

New sys.module_names attribute in Python 3.10: list of all stdlib modules
by Victor Stinner Jan. 28, 2021

Jan. 28, 2021

Hi, I just added a new sys.module_names attribute, list (technically a frozenset) of all stdlib module names: https://bugs.python.org/issue42955 There are multiple use cases: * Group stdlib imports when reformatting a Python file, * Exclude stdlib imports when computing dependencies. * Exclude stdlib modules when listing extension modules on crash or fatal error, only list 3rd party extension (already implemented in master, see bpo-42923 ;-)). * Exclude stdlib modules when tracing the execution of a program using the trace module. * Detect typo and suggest a fix: ImportError("No module named maths. Did you mean 'math'?",) (test the nice friendly-traceback project!). Example: >>> 'asyncio' in sys.module_names True >>> 'numpy' in sys.module_names False >>> len(sys.module_names) 312 >>> type(sys.module_names) <class 'frozenset'> >>> sorted(sys.module_names)[:10] ['__future__', '_abc', '_aix_support', '_ast', '_asyncio', '_bisect', '_blake2', '_bootsubprocess', '_bz2', '_codecs'] >>> sorted(sys.module_names)[-10:] ['xml.dom', 'xml.etree', 'xml.parsers', 'xml.sax', 'xmlrpc', 'zipapp', 'zipfile', 'zipimport', 'zlib', 'zoneinfo'] The list is opinionated and defined by its documentation: A frozenset of strings containing the names of standard library modules. It is the same on all platforms. Modules which are not available on some platforms and modules disabled at Python build are also listed. All module kinds are listed: pure Python, built-in, frozen and extension modules. Test modules are excluded. For packages, only sub-packages are listed, not sub-modules. For example, ``concurrent`` package and ``concurrent.futures`` sub-package are listed, but not ``concurrent.futures.base`` sub-module. See also the :attr:`sys.builtin_module_names` list. The design (especially, the fact of having the same list on all platforms) comes from the use cases list above. For example, running isort should produce the same output on any platform, and not depend if the Python stdlib was splitted into multiple packages on Linux (which is done by most popular Linux distributions). The list is generated by the Tools/scripts/generate_module_names.py script: https://github.com/python/cpython/blob/master/Tools/scripts/generate_module… When you add a new module, you must run "make regen-module-names, otherwise a pre-commit check will fail on your PR ;-) The list of Windows extensions is currently hardcoded in the script (contributions are welcomed to discover them, since the list is short and evolves rarely, I didn't feel the need to spend time that on that). Currently (Python 3.10.0a4+), there are 312 names in sys.module_names, stored in Python/module_names.h: https://github.com/python/cpython/blob/master/Python/module_names.h It was decided to include "helper" modules like "_aix_support" which is used by sysconfig. But test modules like _testcapi are excluded to make the list shorter (it's rare to run the CPython test suite outside Python). There are 83 private modules, name starting with an underscore (exclude _abc but also __future__): >>> len([name for name in sys.module_names if not name.startswith('_')]) 229 This new attribute may help to define "what is the Python stdlib" ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death.

13 34

Does the C-API include C structs not explicitly documented?
by Mark Shannon Jan. 27, 2021

Jan. 27, 2021

Hi everyone, Are C structs part of the C-API if the struct is only included in a header, but not documented? I have always assumed not, but it isn't clear. This question arose in https://github.com/python/cpython/pull/24298 Obviously, any structs listed in https://docs.python.org/3/c-api/structures.html are part of the API Thoughts? Cheers, Mark.

2 1

Resurrecting PEP 558 (defined semantics for locals())
by Mark Shannon Jan. 27, 2021

Jan. 27, 2021

Hi, PEP 558 seems to be dormant, if not abandoned. There are at least two open issues for bugs resulting from the currently weird and inefficient behavior of `f_locals` and `locals()`. See https://bugs.python.org/issue30744 for an example of undesirable behaviour. PEP 588, or something like it, would fix those. I'd be happy to take over the PEP, or write a new one. I like PEP 588, although I would propose a simplification. The PEP mentions "tracing mode" and changes behavior according to whether a program is in "tracing mode" or not. I'd like to remove this distinction. Cheers, Mark.

3 5

Re: Speeding up CPython
by Bruno Cabral Jan. 25, 2021

Jan. 25, 2021

Hello Everyone, I´m very curious about this proposal, but unfortunately it has been a while since I heard any news about this project. Does anyone know what happened? Best Regards, Bruno

3 2

Tff rvt
by Swati Singh Jan. 23, 2021

Jan. 23, 2021

-- *Life is short .... Live it to it's full potential :) -Swati *

1 0

Please explain how to migrate when a function is removed, thanks ;-)
by Victor Stinner Jan. 22, 2021

Jan. 22, 2021

Hi, We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to build. I'm not here to talk about DeprecationWarning, but how we communicate on incompatible changes made on purpose. For example, What's New in Python 3.8 announces: "In asyncio, the explicit passing of a loop argument has been deprecated and will be removed in version 3.10". As expected, the parameter was removed in Python 3.10 (bpo-42392), but I cannot see anything in What's New in Python 3.10. The problem is that I don't know how to fix projects broken by this change. I'm not complaining about this specific change, but more generally. I strongly suggest to well document incompatible changes. The bare minimum is to mention them in "Porting to Python 3.10" section: https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10 A link to the bpo sometimes helps to understand how to port code. But I would really appreciate if authors of incompatible changes would explain how to add Python 3.10 support to existing projects, without losing support for older Python versions. Not just "this function is now removed, good luck!" :-) I didn't touch asyncio for at least 1 year, so I don't know what happens if I remove a loop argument. Does an application remain compatible with Python 3.6 without passing loop? I know that I made multiple incompatible changes myself and I'm sure that the documentation should probably be enhanced, but I also expect others to help on that! ;-) Please think about people who have to port 4000 Python projects to Python 3.10! --- The best would be to ship a tool which adds Python 3.10 support to existing projects without losing support with Python 3.6. Maybe something like https://github.com/asottile/pyupgrade could be used for that? pyupgrade looks more specific to the Python syntax, than the usage of the stdlib. I wrote such tool to add Python 3.10 support to C extensions without losing support with Python 2.7. It relies on a header file (pythoncapi_compat.h) which provides new C API functions on old Python versions. https://github.com/pythoncapi/pythoncapi_compat For example, it replaces "obj->ob_refcnt = refcnt;" with "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python 3.8 and older (function added to Python 3.9) Victor -- Night gathers, and now my watch begins. It shall not end until my death.

4 7

The SC is accepting PEP 632: deprecating distutils
by Brett Cannon Jan. 22, 2021

Jan. 22, 2021

On behalf of the SC, I'm happy to announce that we have chosen to accept PEP 632. Congrats, Steve, and thanks for the work on the PEP! I'll let Steve outline what the next steps are for implementing the PEP.

2 1

Summary of Python tracker Issues
by Python tracker Jan. 22, 2021

Jan. 22, 2021

ACTIVITY SUMMARY (2021-01-15 - 2021-01-22) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7578 (+32) closed 47161 (+35) total 54739 (+67) Open issues with patches: 3037 Issues opened (52) ================== #42604: EXT_SUFFIX too short on FreeBSD and AIX https://bugs.python.org/issue42604 reopened by vstinner #42937: 192.0.0.8 (IPv4 dummy address) considered globally reachable https://bugs.python.org/issue42937 opened by cdirkx #42939: Linux's chattr i.e. ioctl(FS_IOC_SETFLAGS) is not supported in https://bugs.python.org/issue42939 opened by socketpair #42940: Incorrect behavior of inspect.signature(f).bind https://bugs.python.org/issue42940 opened by slavkostrov #42941: Infinite loop in asyncio sslproto https://bugs.python.org/issue42941 opened by linxy95 #42943: singledispatchmethod should expose registry of all known overl https://bugs.python.org/issue42943 opened by Ilya.Kulakov #42946: email: ValueError in get_section when parsing header with non- https://bugs.python.org/issue42946 opened by The Compiler #42947: email: Handling when both extended/ascii parameter (filename*/ https://bugs.python.org/issue42947 opened by The Compiler #42948: bytearray.copy is undocumented https://bugs.python.org/issue42948 opened by wim.glenn #42949: pdb & multiprocessing.Pool: AttributeError: module '__main__' https://bugs.python.org/issue42949 opened by macfreek #42950: Incorrect exception behavior in handling recursive call. https://bugs.python.org/issue42950 opened by xxm #42952: Incorrect handling of EC_KEY_new_by_curve_name() in the _ssl m https://bugs.python.org/issue42952 opened by ZackerySpytz #42955: Add sys.module_names: list of stdlib module names (Python and https://bugs.python.org/issue42955 opened by vstinner #42956: Easy conversion between range and slice https://bugs.python.org/issue42956 opened by nemeskeyd #42957: os.readlink produces wrong result on windows https://bugs.python.org/issue42957 opened by simon mackenzie #42958: filecmp.cmp(shallow=True) isn't actually shallow when only mti https://bugs.python.org/issue42958 opened by AlexVndnblcke #42960: resources module, FreeBSD update adding RLIMIT_KQUEUES constan https://bugs.python.org/issue42960 opened by devnexen #42961: Use-after-free (of a heap type) during finalization https://bugs.python.org/issue42961 opened by bstaletic #42962: Windows: SystemError during os.kill(..., signal.CTRL_C_EVENT) https://bugs.python.org/issue42962 opened by William.Schwartz #42963: [multiprocessing] Calling pool.terminate() from an error_callb https://bugs.python.org/issue42963 opened by sjelin #42964: Draft PEP blob etc https://bugs.python.org/issue42964 opened by larry #42966: argparse: customizable help formatter https://bugs.python.org/issue42966 opened by monkeyman79 #42967: [security] urllib.parse.parse_qsl(): Web cache poisoning - `; https://bugs.python.org/issue42967 opened by AdamGold #42968: multiprocessing handle leak on Windows when child process is k https://bugs.python.org/issue42968 opened by dgrunwald #42969: pthread_exit & PyThread_exit_thread from PyEval_RestoreThread https://bugs.python.org/issue42969 opened by gregory.p.smith #42970: File path with blank causes open error in 3.8, not in 3.7 https://bugs.python.org/issue42970 opened by dday52 #42971: Some errnos for BSD/OSX are missing from errno module https://bugs.python.org/issue42971 opened by ngie #42972: [C API] Heap types (PyType_FromSpec) must fully implement the https://bugs.python.org/issue42972 opened by vstinner #42973: argparse: mixing optional and positional arguments... not agai https://bugs.python.org/issue42973 opened by monkeyman79 #42974: tokenize reports incorrect end col offset and line string when https://bugs.python.org/issue42974 opened by romanows #42976: __text_signature__ parser silently drops arguments with certai https://bugs.python.org/issue42976 opened by Antony.Lee #42977: Tkinter Optionmenu Too Narrow on Mac https://bugs.python.org/issue42977 opened by zjdavid #42979: _zoneinfo: zoneinfomodule_exec() doesn't check for PyDateTime_ https://bugs.python.org/issue42979 opened by vstinner #42980: argparse: GNU-style help formatter https://bugs.python.org/issue42980 opened by will #42981: Error messages raised by _hashlib_scrypt_impl() are slightly m https://bugs.python.org/issue42981 opened by illia-v #42982: Update suggested number of iterations for pbkdf2_hmac() https://bugs.python.org/issue42982 opened by illia-v #42985: AMD64 Arch Linux Asan 3.x fails: command timed out: 1200 secon https://bugs.python.org/issue42985 opened by vstinner #42986: pegen parser: Crash on SyntaxError with f-string on Windows https://bugs.python.org/issue42986 opened by neonene #42988: Information disclosure via pydoc -p: /getfile?key=path allows https://bugs.python.org/issue42988 opened by hroncok #42990: Improve the C code for calling Python code: _PyEval_EvalCode() https://bugs.python.org/issue42990 opened by Mark.Shannon #42991: support for splitting multichannel audio fragments in audioop https://bugs.python.org/issue42991 opened by Th4R4 #42992: Tkinter bbox coordinates incorrectly drawn https://bugs.python.org/issue42992 opened by rhoffmann #42993: doc xml.etree.ElementTree.ElementTree.write does not mention a https://bugs.python.org/issue42993 opened by buhtz #42994: Missing MIME types for opus, AAC, 3gpp and 3gpp2 https://bugs.python.org/issue42994 opened by nbeals #42995: Add PurePath.with_suffix_appended() https://bugs.python.org/issue42995 opened by bgerrity #42997: Improve error message for missing : before suites https://bugs.python.org/issue42997 opened by pablogsal #42998: pathlib.Path: add `username` argument to `home()` https://bugs.python.org/issue42998 opened by barneygale #42999: `pathlib.Path.link_to()` documentation is misleading https://bugs.python.org/issue42999 opened by barneygale #43000: All SSL requests fail with WRONG_VERSION_NUMBER when a packet https://bugs.python.org/issue43000 opened by darrenrs #43001: python3.9.1 test_embed test_tabnanny failed https://bugs.python.org/issue43001 opened by asholomitskiy84 #43002: Exception chaining accepts exception classes https://bugs.python.org/issue43002 opened by cool-RR #43003: Parts of the API will be removed in Python 4.0 ? https://bugs.python.org/issue43003 opened by MichaelClerx Most recent 15 issues with no replies (15) ========================================== #43003: Parts of the API will be removed in Python 4.0 ? https://bugs.python.org/issue43003 #43002: Exception chaining accepts exception classes https://bugs.python.org/issue43002 #43001: python3.9.1 test_embed test_tabnanny failed https://bugs.python.org/issue43001 #42999: `pathlib.Path.link_to()` documentation is misleading https://bugs.python.org/issue42999 #42994: Missing MIME types for opus, AAC, 3gpp and 3gpp2 https://bugs.python.org/issue42994 #42993: doc xml.etree.ElementTree.ElementTree.write does not mention a https://bugs.python.org/issue42993 #42991: support for splitting multichannel audio fragments in audioop https://bugs.python.org/issue42991 #42981: Error messages raised by _hashlib_scrypt_impl() are slightly m https://bugs.python.org/issue42981 #42979: _zoneinfo: zoneinfomodule_exec() doesn't check for PyDateTime_ https://bugs.python.org/issue42979 #42976: __text_signature__ parser silently drops arguments with certai https://bugs.python.org/issue42976 #42966: argparse: customizable help formatter https://bugs.python.org/issue42966 #42963: [multiprocessing] Calling pool.terminate() from an error_callb https://bugs.python.org/issue42963 #42960: resources module, FreeBSD update adding RLIMIT_KQUEUES constan https://bugs.python.org/issue42960 #42949: pdb & multiprocessing.Pool: AttributeError: module '__main__' https://bugs.python.org/issue42949 #42947: email: Handling when both extended/ascii parameter (filename*/ https://bugs.python.org/issue42947 Most recent 15 issues waiting for review (15) ============================================= #42999: `pathlib.Path.link_to()` documentation is misleading https://bugs.python.org/issue42999 #42997: Improve error message for missing : before suites https://bugs.python.org/issue42997 #42995: Add PurePath.with_suffix_appended() https://bugs.python.org/issue42995 #42994: Missing MIME types for opus, AAC, 3gpp and 3gpp2 https://bugs.python.org/issue42994 #42991: support for splitting multichannel audio fragments in audioop https://bugs.python.org/issue42991 #42990: Improve the C code for calling Python code: _PyEval_EvalCode() https://bugs.python.org/issue42990 #42988: Information disclosure via pydoc -p: /getfile?key=path allows https://bugs.python.org/issue42988 #42986: pegen parser: Crash on SyntaxError with f-string on Windows https://bugs.python.org/issue42986 #42982: Update suggested number of iterations for pbkdf2_hmac() https://bugs.python.org/issue42982 #42981: Error messages raised by _hashlib_scrypt_impl() are slightly m https://bugs.python.org/issue42981 #42980: argparse: GNU-style help formatter https://bugs.python.org/issue42980 #42974: tokenize reports incorrect end col offset and line string when https://bugs.python.org/issue42974 #42973: argparse: mixing optional and positional arguments... not agai https://bugs.python.org/issue42973 #42967: [security] urllib.parse.parse_qsl(): Web cache poisoning - `; https://bugs.python.org/issue42967 #42958: filecmp.cmp(shallow=True) isn't actually shallow when only mti https://bugs.python.org/issue42958 Top 10 most discussed issues (10) ================================= #42955: Add sys.module_names: list of stdlib module names (Python and https://bugs.python.org/issue42955 24 msgs #42988: Information disclosure via pydoc -p: /getfile?key=path allows https://bugs.python.org/issue42988 13 msgs #42967: [security] urllib.parse.parse_qsl(): Web cache poisoning - `; https://bugs.python.org/issue42967 10 msgs #42887: 100000 assignments of .__sizeof__ cause a segfault on del https://bugs.python.org/issue42887 7 msgs #42962: Windows: SystemError during os.kill(..., signal.CTRL_C_EVENT) https://bugs.python.org/issue42962 7 msgs #42985: AMD64 Arch Linux Asan 3.x fails: command timed out: 1200 secon https://bugs.python.org/issue42985 7 msgs #41701: Buildbot web page: connection lost after 1 minute, then displa https://bugs.python.org/issue41701 6 msgs #42606: Support POSIX atomicity guarantee of O_APPEND on Windows https://bugs.python.org/issue42606 6 msgs #42800: Traceback objects allow accessing frame objects without trigge https://bugs.python.org/issue42800 6 msgs #33289: tkinter askcolor returning floats for r, g, b values instead o https://bugs.python.org/issue33289 5 msgs Issues closed (35) ================== #36769: doc Document that fnmatch.filter supports any kind of iterable https://bugs.python.org/issue36769 closed by iritkatriel #40176: unterminated string literal tokenization error messages could https://bugs.python.org/issue40176 closed by BTaskaya #40304: Classes created using type() don't need to explicitly inherit https://bugs.python.org/issue40304 closed by terry.reedy #41036: Visit the type of instance of heap types if tp_traverse is not https://bugs.python.org/issue41036 closed by vstinner #41545: gc API requiring matching number of gc.disable - gc.enable cal https://bugs.python.org/issue41545 closed by Yonatan Goldschmidt #41995: five possible Null Pointer Dereference bugs. https://bugs.python.org/issue41995 closed by vstinner #42005: profile/cProfile CLI should catch BrokenPipeError https://bugs.python.org/issue42005 closed by vstinner #42048: Document Argument Clinic's defining_class converter https://bugs.python.org/issue42048 closed by erlendaasland #42163: _replace() no longer works on platform.uname_result objects https://bugs.python.org/issue42163 closed by jaraco #42189: copy.deepcopy() no longer works on platform.uname_result objec https://bugs.python.org/issue42189 closed by jaraco #42447: robotsparser deny all with some rules https://bugs.python.org/issue42447 closed by terry.reedy #42531: importlib.resources.path() raises TypeError for packages witho https://bugs.python.org/issue42531 closed by jaraco #42659: Objects of uname_result Class Cannot be Successfully Pickled https://bugs.python.org/issue42659 closed by jaraco #42780: os.set_inheritable() fails for O_PATH file descriptors on Linu https://bugs.python.org/issue42780 closed by vstinner #42856: ensurepip: add configure --with-wheel-pkg-dir=PATH to get whee https://bugs.python.org/issue42856 closed by vstinner #42864: Improve error messages regarding unclosed parentheses https://bugs.python.org/issue42864 closed by pablogsal #42923: Py_FatalError(): dump the list of extension modules https://bugs.python.org/issue42923 closed by vstinner #42925: Error trace of else inside class https://bugs.python.org/issue42925 closed by Mark.Shannon #42931: Include randbytes in random.__all__ https://bugs.python.org/issue42931 closed by rhettinger #42938: [security] ctypes double representation BoF https://bugs.python.org/issue42938 closed by benjamin.peterson #42942: Feature request: Add decdigest() to hashlib https://bugs.python.org/issue42942 closed by 2d4d #42944: random.Random.sample bug when counts is not None https://bugs.python.org/issue42944 closed by rhettinger #42945: weakref.finalize documentation contradicts itself RE: finalize https://bugs.python.org/issue42945 closed by ryan.a.heisler #42951: Random and infinite loop in dealing with recursion error for " https://bugs.python.org/issue42951 closed by Mark.Shannon #42953: After the subclass of enum.Enum is imported in some diffenent https://bugs.python.org/issue42953 closed by ethan.furman #42954: new "if-as" syntax https://bugs.python.org/issue42954 closed by steven.daprano #42959: AttributeError: module 'signal' has no attribute 'SIGHUP' https://bugs.python.org/issue42959 closed by gvanrossum #42965: Optional callable raises TypeError https://bugs.python.org/issue42965 closed by Ofekmeister #42975: urllib.parse.parse_qsl(): add an option to not consider semico https://bugs.python.org/issue42975 closed by vstinner #42978: Improve error message when "self" is missing from the method d https://bugs.python.org/issue42978 closed by pablogsal #42983: sys.argv incorrectly parses command lines with args in environ https://bugs.python.org/issue42983 closed by gvanrossum #42984: Python 3.7 and qt Libraries https://bugs.python.org/issue42984 closed by gvanrossum #42987: HTTP header injection in urllib on windows https://bugs.python.org/issue42987 closed by vstinner #42989: Bug https://bugs.python.org/issue42989 closed by zach.ware #42996: hashlib documentation references an obsolete RFC 2898 https://bugs.python.org/issue42996 closed by illia-v

1 0

pathlib.Path: inconsistent symlink_to() and link_to()
by Barney Gale Jan. 22, 2021

Jan. 22, 2021

Hi, Pathlib's symlink_to() and link_to() methods have different argument orders, so: a.symlink_to(b) # Creates a symlink from A to B a.link_to(b) # Creates a hard link from B to A I don't think link_to() was intended to be implemented this way, as the docs say "Create a hard link pointing to a path named target.". It's also inconsistent with everything else in pathlib, most obviously symlink_to(). Bug report here: https://bugs.python.org/issue39291 This /really/ irks me. Apparently it's too late to fix link_to(), so I'd like to suggest we add a new hardlink_to() method that matches the symlink_to() argument order. link_to() then becomes deprecated/undocumented. Any thoughts? Barney

4 5

PEP 651: Robust Overflow Handling, version 2
by Mark Shannon Jan. 22, 2021

Jan. 22, 2021

Hi, I've updated the PEP in light of my experiments and feedback. The new API is simpler and a bit more backwards compatible. https://www.python.org/dev/peps/pep-0651/ Cheers, Mark.

2 2

Doctest does not work with mock.call PR #22981
by Alfred Perlstein Jan. 21, 2021

Jan. 21, 2021

Hello, There's been a bug open where doctest can break if there are proxy objects that fail to unwrap (https://bugs.python.org/issue35753) since python 3.7, this includes when importing 'call' from the 'mock' module. Does someone have time to review PR 22981 (https://github.com/python/cpython/pull/22981). Last feedback from folks with merge capabilities seems to be from November 8th. I've replied a few times in December on the PR but there is no response. -Alfred

1 0

PEP 651 -- Robust Overflow Handling
by Mark Shannon Jan. 20, 2021

Jan. 20, 2021

Hi everyone, It's time for yet another PEP :) Fortunately, this one is a small one that doesn't change much. It's aim is to make the VM more robust. Abstract ======== This PEP proposes that machine stack overflow is treated differently from runaway recursion. This would allow programs to set the maximum recursion depth to fit their needs and provide additional safety guarantees. The following program will run safely to completion: sys.setrecursionlimit(1_000_000) def f(n): if n: f(n-1) f(500_000) The following program will raise a StackOverflow, without causing a VM crash: sys.setrecursionlimit(1_000_000) class X: def __add__(self, other): return self + other X() + 1 ----------- The full PEP can be found here: https://www.python.org/dev/peps/pep-0651 As always, comments are welcome. Cheers, Mark.

7 21

Advantages of pattern matching - a simple comparative analysis
by Brian Coleman Jan. 19, 2021

Jan. 19, 2021

Take as an example a function designed to process a tree of nodes similar to that which might be output by a JSON parser. There are 4 types of node: - A node representing JSON strings - A node representing JSON numbers - A node representing JSON arrays - A node representing JSON dictionaries The function transforms a tree of nodes, beginning at the root node, and proceeding recursively through each child node in turn. The result is a Python object, with the following transformation applied to each node type: - A JSON string `->` Python `str` - A JSON number `->` Python `float` - A JSON array `->` Python `list` - A JSON dictionary `->` Python `dict` I have implemented this function using 3 different approaches: - The visitor pattern - `isinstance` checks against the node type - Pattern matching Here is the implementation using the visitor pattern: ``` from typing import List, Tuple class NodeVisitor: def visit_string_node(self, node: StringNode): pass def visit_number_node(self, node: NumberNode): pass def visit_list_node(self, node: ListNode): pass def visit_dict_node(self, node: DictNode): pass class Node: def visit(visitor: NodeVisitor): raise NotImplementedError() class StringNode(Node): value: str def visit(self, visitor: NodeVisitor): visitor.visit_string_node(self) class NumberNode(Node): value: str def visit(self, visitor: NodeVisitor): visitor.visit_number_node(self) class ListNode(Node): children: List[Node] def visit(self, visitor: NodeVisitor): visitor.visit_list_node(self) class DictNode(Node): children: List[Tuple[str, Node]] def visit(self, visitor: NodeVisitor): visitor.visit_dict_node(self) class Processor(NodeVisitor): def process(root_node: Node): return root_node.visit(self) def visit_string_node(self, node: StringNode): return node.value def visit_number_node(self, node: NumberNode): return float(node.value) def visit_list_node(self, node: ListNode): return [child_node.visit(self) for child_node in node.children] def visit_dict_node(self, node: DictNode): return {key: child_node.visit(self) for key, child_node in node.children} def process(root_node: Node): processor = Processor() return processor.process(root_node) ``` Here is the implementation using `isinstance` checks against the node type: ``` from typing import List, Tuple class Node: pass class StringNode(Node): value: str class NumberNode(Node): value: str class ListNode(Node): children: List[Node] class DictNode(Node): children: List[Tuple[str, Node]] def process(root_node: Node): def process_node(node: Node): if isinstance(node, StringNode): return node.value elif isinstance(node, NumberNode): return float(node.value) elif isinstance(node, ListNode): return [process_node(child_node) for child_node in node.children] elif isinstance(node, DictNode): return {key: process_node(child_node) for key, child_node in node.children} else: raise Exception('Unexpected node') return process_node(root_node) ``` Finally here is the implementation using pattern matching: ``` from typing import List, Tuple class Node: pass class StringNode(Node): value: str class NumberNode(Node): value: str class ListNode(Node): children: List[Node] class DictNode(Node): children: List[Tuple[str, Node]] def process(root_node: Node): def process_node(node: Node): match node: case StringNode(value=str_value): return str_value case NumberNode(value=number_value): return float(number_value) case ListNode(children=child_nodes): return [process_node(child_node) for child_node in child_nodes] case DictNode(children=child_nodes): return {key: process_node(child_node) for key, child_node in child_nodes} case _: raise Exception('Unexpected node') return process_node(root_node) ``` Here are the lengths of the different implementations: - Pattern matching `->` 37 lines - `isinstance` checks `->` 36 lines - The visitor pattern `->` 69 lines The visitor pattern implementation is by far the most verbose solution, weighing in at almost twice the length of the alternative implementations due to the large amount of boilerplate that is necessary to achieve double dispatch. The pattern matching and `isinstance` check implementations are very similar in length for this trivial example. In each implementation, there are 2 operations performed on each node. - Determine the type of the node - Destructure the node to extract the desired data The visitor pattern and `isinstance` check implementations separate these 2 operations, whereas the pattern matching approach combines the operations together. I believe that it is the declarative nature of pattern matching, where the operations of determining the type of the node and destructuring the node are combined into a single clause, which allows pattern matching to express a concise solution to the problem. In this trivial example, the advantage of pattern matching over the alternative of using a sequence of `if`-`elif`-`else` statements is not as obvious as it would be when compared to a more complex example, where a sub-tree of nodes might be matched based on their type and be destructured in a single clause. I have seen elsewhere an argument that pattern matching should not be accepted into Python as it introduces a pseudo-DSL that is separate from the rest of the language. I agree that pattern matching might be viewed as a pseudo-DSL, but I believe that it is a good thing, if it allows the solution to certain classes of problems to be expressed in a concise manner. People often raise similar objections to operator overloading in other languages, whereas the presence of operator overloading in Python allows mathematical expressions involving custom numeric types such as vectors to be expressed in a natural way. Furthermore, Python has a regular expression module which implements it's own DSL for the purpose of matching string patterns. Regular expressions, in a similar way to pattern matching, allow string patterns to be expressed in a concise and declarative manner. I really hope that the Steering Council accepts pattern matching into Python. I think that it allows for processing of heterogeneous graphs of objects using recursion in a concise, declarative manner. I would like to thank the authors of the Structural Pattern Matching PEP for their hard work in designing this feature and developing an implementation of it. I believe that it will be a wonderful addition to the language that I am very much looking forward to using.

12 25

Should python installation modify /etc/paths.d on Mac OS X?
by Peng Yu Jan. 19, 2021

Jan. 19, 2021

Hi, The /etc/paths.d directory on Mac OS X contains the following directory. Some packages have their PATH variables set there. For Python, I have to set the PATH manually in my ~/.bashrc. Should the Python developers consider setting PATH in /etc/paths.d as well? ==> /etc/paths.d/100-rvictl <== /Library/Apple/usr/bin ==> /etc/paths.d/40-XQuartz <== /opt/X11/bin ==> /etc/paths.d/com.vmware.fusion.public <== /Applications/VMware Fusion.app/Contents/Public -- Regards, Peng

4 4

Unification of the Mac builds?
by Chris Barker Jan. 18, 2021

Jan. 18, 2021

Sorry if I'm out of the loop here, but with Apple's new chip coming out, we need new a build configuration (which I think has already been started, if not done). Perhaps we could take this opportunity to better modularize / unify the build setup? As it was last I checked, you really had only two options: 1) Ignore anything mac specific, and get a "unix" build. 2) Get a full "Framework" build, optionally with Universal support. It would be nice to keep the Framework structure independent of the Mac-specific features, if possible. In particular, I'd love to get be able to get the "pythonw" executable wrapper in an otherwise standard unix build [*]. It would also be nice if it were possible to get universal binaries in a "unix style" build. (option 3 would be to simply abandon the Framework Build altogether -- it's still not clear to me what this really buys mac users) Any chance of this happening? I'm afraid I know nothing of autoconf, so can't be much help, but I'd be willing to help out with testing, or documenting, or anything else that I have the skills to do. Thanks, -Chris [*] The pythonw issue has been a thorn in the side of conda for years. conda uses a standard unix build on the Mac, for consistency with other unix systems. But there is no configuration set up to build the pythonw wrapper outside of a "Framework" build. So instead, conda has creates its own "pythonw" wrapper -- but that is a bash script that re-directs to a different executable. This works fine on the command line (or #! line), but it does not work with setuptools' entry_points. And the setuptools community hasn't shown any interest in hacking around it. Anyway, few enough people develop Desktop apps (particularly with conda) so this has lingered, but it would be nice to fix. -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

3 5

seg fault in 3.10a4
by Robin Becker Jan. 17, 2021

Jan. 17, 2021

Not sure if this is where I should report, but I have a segfault in reportlab userguide creation with 3.10a4. I am getting this info from coredumpctl running archlinux with Linux hostname 5.10.6-arch1-1 #1 SMP PREEMPT Sat, 09 Jan 2021 18:22:35 +0000 x86_64 GNU/Linux the error is Message: Process 1530711 (python) of user 1000 dumped core. Stack trace of thread 1530711: #0 0x00007f7c9bf72615 raise (libc.so.6 + 0x3d615) #1 0x00007f7c9bf5b862 abort (libc.so.6 + 0x26862) #2 0x00007f7c9bfb45e8 __libc_message (libc.so.6 + 0x7f5e8) #3 0x00007f7c9bfbc27a malloc_printerr (libc.so.6 + 0x8727a) #4 0x00007f7c9bfbd64c _int_free (libc.so.6 + 0x8864c) #5 0x00005631260a01dd n/a (/home/robin/devel/reportlab/.py310/bin/python310 + 0x1401dd) 1) I compile this from source using the standard LTO method my script looks like ########################################### CFLAGS="-march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fno-semantic-interposition" LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -rpath=$HOME/LOCAL/3.10.0a4/lib -fno-semantic-interposition" ./configure \ --prefix=$HOME/LOCAL/3.10.0a4 \ --with-computed-gotos \ --enable-optimizations \ --with-lto \ --enable-ipv6 \ --with-system-expat \ --with-dbmliborder=gdbm:ndbm \ --with-system-ffi \ --with-system-libmpdec \ --enable-loadable-sqlite-extensions && make EXTRA_CFLAGS="$CFLAGS" && make install ########################################### 1) can I get more information using a different compile setup 2) or can I get more information with the python run. I just used > $ python genuserguide.py > Built story contains 1843 flowables... > Saved "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" > free(): invalid pointer > Aborted (core dumped) 3) I am running in an old style virtual environment generated using a modified virtualenv 16.2.0 (to fix wrong usage of sys.version[:3] in path names) 4) All other tests seem fine including tests of image generation and the accelerated strinGWidths etc etc so I am a bit mystified. -- Robin Becker

4 5

__init_subclass__ and metaclasses
by Ethan Furman Jan. 16, 2021

Jan. 16, 2021

PEP 487 introduced __init_subclass__ and __set_name__, and both of those were wins for the common cases of metaclass usage. Unfortunately, the implementation of PEP 487 with regards to __init_subclass__ has made the writing of correct metaclasses significantly harder, if not impossible. The cause is that when a metaclass calls type.__new__ to actually create the class, type.__new__ calls the __init_subclass__ methods of the new class' parents, passing it the newly created, but incomplete, class. In code: ``` class Meta(type): # def __new__(mcls, name, bases, namespace, **kwds): # create new class, which will call __init_subclass__ and __set_name__ new_class = type.__new__(mcls, name, bases, namespace, **kwds) # finish setting up class new_class.some_attr = 9 ``` As you can deduce, when the parent __init_subclass__ is called with the new class, `some_attr` has not been added yet -- the new class is incomplete. For Enum, this means that __init_subclass__ doesn't have access to the new Enum's members (they haven't beet added yet). For ABC, this means that __init_subclass__ doesn't have access to __abstract_methods__ (it hasn't been created yet). Because Enum is pure Python code I was able to work around it: - remove new __init_subclass__ (if it exists) - insert dummy class with a no-op __init_subclass__ - call type.__new__ - save any actual __init_subclass__ - add back any new __init_subclass__ - rewrite the new class' __bases__, removing the no-op dummy class - finish creating the class - call the parent __init_subclass__ with the now complete Enum class I have not been able to work around the problem for ABC. Two possible solutions I can think of: - pass a keyword argument to type.__new__ that suppresses the call to __init_subclass__; and - provide a way to invoke new class' parent's __init_subclass__ before returning it or - instead of type.__new__ doing that work, have type.__init__ do it. Thoughts? -- ~Ethan~

6 17

Summary of Python tracker Issues
by Python tracker Jan. 15, 2021

Jan. 15, 2021

ACTIVITY SUMMARY (2021-01-08 - 2021-01-15) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7546 ( -3) closed 47126 (+71) total 54672 (+68) Open issues with patches: 3027 Issues opened (41) ================== #41837: Upgrade installers to OpenSSL 1.1.1i https://bugs.python.org/issue41837 reopened by christian.heimes #42869: pydoc does not append .html to documentation https://bugs.python.org/issue42869 opened by mdk #42871: Regex compilation crashed if I change order of alternatives un https://bugs.python.org/issue42871 opened by Renji #42872: Inconsistent exceptions thrown by pathlib.Path.mkdir on differ https://bugs.python.org/issue42872 opened by Hong Xu #42874: running configure on Solaris 10 gives grep "illegal option" er https://bugs.python.org/issue42874 opened by martin.wheatley.home #42875: argparse incorrectly shows help string on a new line in case o https://bugs.python.org/issue42875 opened by DiPaolo #42878: urllib.request.HTTPPasswordMgr.is_suburi does not care about o https://bugs.python.org/issue42878 opened by Fongeme #42880: ctypes: variadic function call still doesn't work on Apple Sil https://bugs.python.org/issue42880 opened by lazymio #42881: Should typing.get_type_hints change None annotations? https://bugs.python.org/issue42881 opened by larry #42884: array.index() missing start and end https://bugs.python.org/issue42884 opened by daniel.nicorici #42885: Regex performance problem with ^ aka AT_BEGINNING https://bugs.python.org/issue42885 opened by 2d4d #42887: Multiple assignments of attribute "__sizeof__" will cause a se https://bugs.python.org/issue42887 opened by xxm #42888: Not installed “libgcc_s.so.1” causes parser crash. https://bugs.python.org/issue42888 opened by xxm #42889: Incorrect behavior of Python parser after ast node of test pro https://bugs.python.org/issue42889 opened by xxm #42892: AttributeError in email.message.get_body() https://bugs.python.org/issue42892 opened by xavier2 #42893: Strange XPath search behavior of xml.etree.ElementTree.Element https://bugs.python.org/issue42893 opened by robpats #42894: Debugging native Python modules on Windows with Visual Studio https://bugs.python.org/issue42894 opened by jmoguill2 #42896: Solaris 11.4 crle output not handled correctly https://bugs.python.org/issue42896 opened by dmurphy18 #42897: Expose a way to determine if a process has been closed or not https://bugs.python.org/issue42897 opened by petervansickel #42899: Is it legal to eliminate tests of a value, when that test has https://bugs.python.org/issue42899 opened by stestagg #42901: [Enum] move member creation to __set_name__ in order to suppor https://bugs.python.org/issue42901 opened by ethan.furman #42902: a python embedded program may load "C:\Lib\os.py" on windows https://bugs.python.org/issue42902 opened by houjingyi233 #42904: get_type_hints does not provide localns for classes https://bugs.python.org/issue42904 opened by pbryan #42909: Email header with ;;;; stuffing takes very long to parse https://bugs.python.org/issue42909 opened by eriker #42911: Addition chains for pow saves 5-20% time for pow(int,int) https://bugs.python.org/issue42911 opened by jneb #42913: asyncio.ProactorEventLoop mishandles signal wakeup file descri https://bugs.python.org/issue42913 opened by hidmic #42914: pprint numbers with underscore https://bugs.python.org/issue42914 opened by fov #42915: enum.Flag ~ bitwise negation is very slow and can't be defined https://bugs.python.org/issue42915 opened by aspin2 #42916: Support for DICOM image file format in imghdr module https://bugs.python.org/issue42916 opened by claw #42917: Block stack size for frame objects should be dynamically sizab https://bugs.python.org/issue42917 opened by tomkpz #42918: Nested multi-line expression will lead to "compile()" fails https://bugs.python.org/issue42918 opened by xxm #42919: Blank in multiline “if expressions” will lead to EOF error https://bugs.python.org/issue42919 opened by xxm #42921: Inferred Optional type of wrapper function arguments https://bugs.python.org/issue42921 opened by joperez #42923: Py_FatalError(): dump the list of extension modules https://bugs.python.org/issue42923 opened by vstinner #42924: bytearray_repeat copies from ob_bytes instead of ob_start https://bugs.python.org/issue42924 opened by tholl #42925: Error trace of else inside class https://bugs.python.org/issue42925 opened by ChenXuan #42926: Split compiler into code-gen, optimizer and assembler. https://bugs.python.org/issue42926 opened by Mark.Shannon #42927: Inline cache for slots https://bugs.python.org/issue42927 opened by gvanrossum #42929: On Windows, shutil.move doesn't raise FileExistsError if dst e https://bugs.python.org/issue42929 opened by fireattack #42931: Include randbytes in random.__all__ https://bugs.python.org/issue42931 opened by sbalian #42935: Pickle can't import builtins at exit https://bugs.python.org/issue42935 opened by Wicken Most recent 15 issues with no replies (15) ========================================== #42935: Pickle can't import builtins at exit https://bugs.python.org/issue42935 #42931: Include randbytes in random.__all__ https://bugs.python.org/issue42931 #42921: Inferred Optional type of wrapper function arguments https://bugs.python.org/issue42921 #42919: Blank in multiline “if expressions” will lead to EOF error https://bugs.python.org/issue42919 #42916: Support for DICOM image file format in imghdr module https://bugs.python.org/issue42916 #42897: Expose a way to determine if a process has been closed or not https://bugs.python.org/issue42897 #42875: argparse incorrectly shows help string on a new line in case o https://bugs.python.org/issue42875 #42868: SpooledTemporaryFile.__iter__ is not transparent to rollover https://bugs.python.org/issue42868 #42864: Improve error messages regarding unclosed parentheses https://bugs.python.org/issue42864 #42844: Turtle Module -- "onclick" arguments enchancement https://bugs.python.org/issue42844 #42838: Wait for cleanup coroutines before event loop is closed. https://bugs.python.org/issue42838 #42836: docs: struct: clarify struct caching behaviour https://bugs.python.org/issue42836 #42834: [subinterpreters] Convert "global" static variable caches in _ https://bugs.python.org/issue42834 #42833: Lib/urllib/request.py: digest algorithm should be case insensi https://bugs.python.org/issue42833 #42832: classmethod change in 3.9 needs better documentation https://bugs.python.org/issue42832 Most recent 15 issues waiting for review (15) ============================================= #42931: Include randbytes in random.__all__ https://bugs.python.org/issue42931 #42927: Inline cache for slots https://bugs.python.org/issue42927 #42925: Error trace of else inside class https://bugs.python.org/issue42925 #42924: bytearray_repeat copies from ob_bytes instead of ob_start https://bugs.python.org/issue42924 #42923: Py_FatalError(): dump the list of extension modules https://bugs.python.org/issue42923 #42917: Block stack size for frame objects should be dynamically sizab https://bugs.python.org/issue42917 #42911: Addition chains for pow saves 5-20% time for pow(int,int) https://bugs.python.org/issue42911 #42904: get_type_hints does not provide localns for classes https://bugs.python.org/issue42904 #42901: [Enum] move member creation to __set_name__ in order to suppor https://bugs.python.org/issue42901 #42896: Solaris 11.4 crle output not handled correctly https://bugs.python.org/issue42896 #42892: AttributeError in email.message.get_body() https://bugs.python.org/issue42892 #42875: argparse incorrectly shows help string on a new line in case o https://bugs.python.org/issue42875 #42874: running configure on Solaris 10 gives grep "illegal option" er https://bugs.python.org/issue42874 #42869: pydoc does not append .html to documentation https://bugs.python.org/issue42869 #42865: sysconfig appends CFLAGS to LD https://bugs.python.org/issue42865 Top 10 most discussed issues (10) ================================= #42899: Is it legal to eliminate tests of a value, when that test has https://bugs.python.org/issue42899 24 msgs #42880: ctypes: variadic function call still doesn't work on Apple Sil https://bugs.python.org/issue42880 12 msgs #42927: Inline cache for slots https://bugs.python.org/issue42927 11 msgs #42800: Traceback objects allow accessing frame objects without trigge https://bugs.python.org/issue42800 9 msgs #42862: Use functools.lru_cache iso. _sqlite.Cache in sqlite3 module https://bugs.python.org/issue42862 9 msgs #33809: Expose `capture_locals` parameter in `traceback` convenience f https://bugs.python.org/issue33809 8 msgs #40956: Use Argument Clinic in sqlite3 https://bugs.python.org/issue40956 6 msgs #42888: Not installed “libgcc_s.so.1” causes parser crash. https://bugs.python.org/issue42888 6 msgs #42902: a python embedded program may load "C:\Lib\os.py" on windows https://bugs.python.org/issue42902 6 msgs #42843: What min_sphinx for Python 3.10 https://bugs.python.org/issue42843 5 msgs Issues closed (67) ================== #4480: bdist_msi and bdist_wininst are missing an uninstaller icon https://bugs.python.org/issue4480 closed by vstinner #4636: bdist_wininst installer with install script raises exception https://bugs.python.org/issue4636 closed by vstinner #4878: post installer script's message is not shown to user with bdis https://bugs.python.org/issue4878 closed by vstinner #7833: bdist_wininst installers fail to load extensions built with Is https://bugs.python.org/issue7833 closed by vstinner #8171: bdist_wininst builds wrongly for --plat-name=win-amd64 https://bugs.python.org/issue8171 closed by vstinner #9456: Apparent memory leak in PC/bdist_wininst/install.c https://bugs.python.org/issue9456 closed by vstinner #10945: bdist_wininst depends on MBCS codec, unavailable on non-Window https://bugs.python.org/issue10945 closed by vstinner #11598: missing afxres.h error when building bdist_wininst in Visual S https://bugs.python.org/issue11598 closed by vstinner #12200: bdist_wininst install_script not run on uninstall https://bugs.python.org/issue12200 closed by vstinner #13038: bdist_wininst installers should warn if target dir is read-onl https://bugs.python.org/issue13038 closed by vstinner #13123: bdist_wininst uninstaller does not remove pycache directories https://bugs.python.org/issue13123 closed by vstinner #13276: bdist_wininst-created installer does not run the postinstallat https://bugs.python.org/issue13276 closed by vstinner #14877: No option to run bdist_wininst against newer msvc versions on https://bugs.python.org/issue14877 closed by vstinner #15321: bdist_wininst installers may terminate with "close failed in f https://bugs.python.org/issue15321 closed by vstinner #16081: Fix compile warnings in thread_pthread.h https://bugs.python.org/issue16081 closed by iritkatriel #16615: gcc 4.7 unused-but-set warnings https://bugs.python.org/issue16615 closed by iritkatriel #17419: bdist_wininst installer should allow install in user directory https://bugs.python.org/issue17419 closed by vstinner #21865: Improve invalid category exception for warnings.filterwarnings https://bugs.python.org/issue21865 closed by iritkatriel #25077: Compiler warnings: initialization from incompatible pointer ty https://bugs.python.org/issue25077 closed by iritkatriel #26554: PC\bdist_wininst\install.c: Missing call to fclose() https://bugs.python.org/issue26554 closed by vstinner #33065: IDLE debugger: failure stepping through module loading https://bugs.python.org/issue33065 closed by terry.reedy #33066: raise an exception from multiple positions break the traceback https://bugs.python.org/issue33066 closed by iritkatriel #34705: Python 3.8 changes how returns through finally clauses are tra https://bugs.python.org/issue34705 closed by Mark.Shannon #39273: ncurses does not include BUTTON5_* constants https://bugs.python.org/issue39273 closed by serhiy.storchaka #39690: Compiler warnings in unicodeobject.c https://bugs.python.org/issue39690 closed by iritkatriel #40052: Incorrect pointer alignment in _PyVectorcall_Function() of cpy https://bugs.python.org/issue40052 closed by petr.viktorin #41433: Logging libraries BufferingHandler flushed twice at shutdown https://bugs.python.org/issue41433 closed by iritkatriel #41771: bdist_wininst doesn't execute postinstall script https://bugs.python.org/issue41771 closed by vstinner #41994: Refcount issues in import https://bugs.python.org/issue41994 closed by serhiy.storchaka #42693: "if 0:" lines are traced; they didn't use to be https://bugs.python.org/issue42693 closed by Mark.Shannon #42775: __init_subclass__ should be called in __init__ https://bugs.python.org/issue42775 closed by ethan.furman #42802: distutils: Remove bdist_wininst command https://bugs.python.org/issue42802 closed by vstinner #42827: pegen parser: Multiline eval with assignment to function call: https://bugs.python.org/issue42827 closed by lys.nikolaou #42848: asyncio produces an unexpected traceback with recursive __geta https://bugs.python.org/issue42848 closed by gvanrossum #42850: Process hangs when calling urllib.request in a multiprocessing https://bugs.python.org/issue42850 closed by gvanrossum #42851: Subclassing Enum with ipaddress.IPv4Network/IPv6Network raises https://bugs.python.org/issue42851 closed by ethan.furman #42867: Entry Widget not editable on Windows 10, but is on Linux Ubunt https://bugs.python.org/issue42867 closed by terry.reedy #42870: Document changed argparse output wrt optional arguments in Wha https://bugs.python.org/issue42870 closed by gvanrossum #42873: Exponential time and space requirements for compilation of nes https://bugs.python.org/issue42873 closed by mark.dickinson #42876: '''"""''' != '""""' https://bugs.python.org/issue42876 closed by eryksun #42877: TracebackException saves more data than it needs for format https://bugs.python.org/issue42877 closed by gvanrossum #42879: SystemError from class 'int' https://bugs.python.org/issue42879 closed by mark.dickinson #42882: Restarting the interpreter causes UB on 3.10.0a4 https://bugs.python.org/issue42882 closed by vstinner #42883: __init__ function may incur an incorrect behavior if passing https://bugs.python.org/issue42883 closed by serhiy.storchaka #42886: math.log and math.log10 domain error on very large Fractions https://bugs.python.org/issue42886 closed by gvanrossum #42890: bug float to int https://bugs.python.org/issue42890 closed by christian.heimes #42891: segfault with gunicorn and a library made with cython bindings https://bugs.python.org/issue42891 closed by vstinner #42895: Return multi-line list concatenation without parentheses retur https://bugs.python.org/issue42895 closed by gvanrossum #42898: pickle.loads() crashes interpreter on invalid input https://bugs.python.org/issue42898 closed by christian.heimes #42900: Ternary operator precedence relative to bitwise or https://bugs.python.org/issue42900 closed by gvanrossum #42903: optimize lru_cache for functions with no arguments https://bugs.python.org/issue42903 closed by rhettinger #42905: Issue while installing numba inside fastparquet https://bugs.python.org/issue42905 closed by serhiy.storchaka #42906: python3 -m doctest test.py tests the stdlib "time" module inst https://bugs.python.org/issue42906 closed by gvanrossum #42907: Division betwen float and 100 not giving corect result https://bugs.python.org/issue42907 closed by christian.heimes #42908: Incorrect line numbers at end of try-except and with statement https://bugs.python.org/issue42908 closed by Mark.Shannon #42910: Wrong code appears in the type alias introduction https://bugs.python.org/issue42910 closed by Smile-zjk #42912: enum.Flag ~ bitwise negation is very slow https://bugs.python.org/issue42912 closed by aspin2 #42920: How to add end_lineno in pyclbr? https://bugs.python.org/issue42920 closed by gvanrossum #42922: Enlace a descripción de función 'dir' faltante https://bugs.python.org/issue42922 closed by Mariatta #42928: adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_tru https://bugs.python.org/issue42928 closed by gvanrossum #42930: xml.parsers.expat results differ buffer_text and / or buffer_s https://bugs.python.org/issue42930 closed by mfxuus #42932: How to handle import tripartite pacakge issue in unittest scr https://bugs.python.org/issue42932 closed by gvanrossum #42933: OUTPUT WRONG https://bugs.python.org/issue42933 closed by christian.heimes #42934: use TracebackException's new compact param in unittest.TestRes https://bugs.python.org/issue42934 closed by gvanrossum #42936: Decimal module performs wrong floor division with negative num https://bugs.python.org/issue42936 closed by mark.dickinson #1109963: bdist_wininst ignores build_lib from build command https://bugs.python.org/issue1109963 closed by vstinner #809846: bdist_wininst doesn't clean up read-only files in build dir https://bugs.python.org/issue809846 closed by vstinner

1 0

Bumping minimum Sphinx version to 3.2 for cpython 3.10?
by Julien Palard Jan. 15, 2021

Jan. 15, 2021

During the development of cpython 3.10, Sphinx was bumped to 3.2.1. Problem is Sphinx 3 have some incompatibilities with Sphinx 2, some that we could work around, some are bit harder, so we may need to bump `needs_sphinx = '3.2'` (currently it is 1.8). I found two incompatibilities: - We're using :no-trim-doctest-flags:, introduced in Sphinx 3.2.0, if we build with Sphinx < 3.2.0 the blocks using it are dropped by Sphinx (we use it in library/doctest). - double backslashes in domain directives are no longer replaced by single backslashes. But a configuration value (strip_signature_backslash) can be used to have the same behavior on Sphinx 2 and 3. So yes, it's still possible to build the docs with Sphinx 1.8 using `make html SPHINXERRORHANDLING=`, it "works" but: - Doctests code blocks using no-trim-doctest-flags dissapear. - Some functions declarations are lacking a backslash, like print(*objects, sep=' ', end='n', ... Which is bad. For the first one we could still workaround with an ugly `sed -i /no-trim-doctest-flags/d`, or simply stop using it (and reopen bpo-36675). For the 2nd one we could use `strip_signature_backslash` in cpython 3.10 to have the same behavior. So the question is: Should we bump minimum version of Sphinx from 1.8 to 3.2.1 along with Python 3.10? For the moment this is where we go, but if you're having to maintain a release of Python along with Sphinx < 3, please make speak. Conversation has already started here [1] and here [2]. [1]: https://bugs.python.org/issue42843 [2]: https://github.com/python/cpython/pull/24142 -- [Julien Palard](https://mdk.fr)

11 19

PyGC and PyObject_Malloc introspection
by Julien Danjou Jan. 15, 2021

Jan. 15, 2021

Hi there, I've spent quite some time on memory profiling for Python now. I'm struggling to get more information from the allocated memory right now for what looks like a sad reason. :( Supposedly PyObject_Malloc() returns some memory space to store a PyObject. If that was true all the time, that would allow anyone to introspect the allocated memory and understand why it's being used. Unfortunately, this is not the case. Objects whose types are tracked by the GC go through _PyObject_GC_Alloc() which changes the underlying memory structure to be (PyGC_HEAD + PyObject). This is a bummer as there then no safe way that I can think of to know if an allocated memory space is gc-tracked or gc-untracked. It makes it therefore impossible to introspect the memory allocated by PyObject_Malloc(). There are multiple ways to solve this, but I don't see any way right now of doing this without slightly changing CPython. Has anyone any idea on how to workaround this or the kind of change that could be acceptable to mitigate the issue? Thanks! Cheers, -- Julien Danjou # Free Software hacker # https://julien.danjou.info

4 10

3.10 change (?) for __bool__
by Mats Wichmann Jan. 15, 2021

Jan. 15, 2021

Someone reported a testsuite break on stuff I work on (scons) with 3.10a4, and it looks similar to this which appears in the changelog at https://docs.python.org/3.10/whatsnew/changelog.html#changelog bpo-23898: Fix inspect.classify_class_attrs() to support attributes with overloaded __eq__ and __bool__. Patch by Mike Bayer. Except when I go look at that BPO issue, it's old - closed 2015. Is this appearing in the 3.10 changelog in error? Sorry - confused !!! The test in question does indeed touch a class which overrides __bool_ in order to raise an exception (to say "don't do that"), and in the test run the (expected) exception is not raised.

17 36

Re: 3.10 change (?) for __bool__
by Mark Shannon Jan. 13, 2021

Jan. 13, 2021

Hi Rob, On 13/01/2021 12:18 pm, Rob Cliffe wrote: > > > On 12/01/2021 15:53, Mark Shannon wrote: >> >> In master we convert `if x: pass` to `pass` which is equivalent, >> unless bool(x) has side effects the first time it is called. This is a >> recent change. > Can you please confirm that this optimisation ONLY applies to bare > names, i.e. NOT > > if x(): pass > if x.y: pass > if x+y: pass The optimization doesn't apply to the expression, but the test. The optimizer (might) transform if x+y: pass to x+y But the expression is still evaluated. Sorry for the confusion, I should have been clearer in my example. It is the call to `bool()` that *might* be eliminated. Cheers, Mark. > > Thanks > Rob Cliffe

1 0

Heap types (PyType_FromSpec) must fully implement the GC protocol
by Victor Stinner Jan. 12, 2021

Jan. 12, 2021

Hi, In the Python stdlib, many heap types currently don't "properly" (fully?) implement the GC protocol which can prevent to destroy these types at Python exit. As a side effect, some other Python objects can also remain alive, and so are not destroyed neither. There is an on-going effect to destroy all Python objects at exit (bpo-1635741). This problem is getting worse when subinterpreters are involved: Refleaks buildbots failures which prevent to spot other regressions, and so these "leaks" / "GC bugs" must be fixed as soon as possible. In my experience, many leaks spotted by tests using subinterpreters were quite old, it's just that they were ignored previously. It's an hard problem and I don't see any simple/obvious solution right now, except of *workarounds* that I dislike. Maybe the only good solution is to fix all heap types, one by one. == Only the Python stdlib should be affected == PyType_FromSpec() was added to Python 3.2 by the PEP 384 to define "heap types" in C, but I'm not sure if it's popular in practice (ex: Cython doesn't use it, but defines static types). I expect that most types to still be defined the old style (static types) in a vas majority of third party extension modules. To be clear, static types are not affected by this email. Third party extension modules using the limited C API (to use the stable ABI) and PyType_FromSpec() can be affected (if they don't fully implement the GC protocol). == Heap type instances now stores a strong reference to their type == In March 2019, the PyObject_Init() function was modified in bpo-35810 to keep a strong reference (INCREF) to the type if the type is a heap type. The fixed problem was that heap types could be destroyed before the last instance is destroyed. == GC and heap types == The new problem is that most heap types don't collaborate well with the garbage collector. The garbage collector doesn't know anything about Python objects, types, reference counting or anything. It only uses the PyGC_Head header and the traverse functions. If an object holds a strong reference to an object but its type does not define a traverse function, the GC cannot guess/infer this reference. A heap type must respect the following 3 conditions to collaborate with the GC: * Have the Py_TPFLAGS_HAVE_GC flag; * Define a traverse function (tp_traverse) which visits the type: Py_VISIT(Py_TYPE(self)); * Instances must be tracked by the GC. If one of these conditions is not met, the GC can fail to destroy a type during a GC collection. If an instance is kept alive late while a Python interpreter is being deleted, it's possible that the type is never deleted, which can keep indirectly *many* objects alive and so don't delete them neither. In practice, when a type is not deleted, a test using subinterpreter starts to fail on Refleaks buildbot since it leaks references. Without subinterpreters, such leak is simply ignored, whereas this is an on-going effect to delete Python objects at exit (bpo-1635741). == Boring traverse functions == Currently, there is no default traverse implementation which visits the type. For example, I had the implement the following function for _thread.LockType: static int lock_traverse(lockobject *self, visitproc visit, void *arg) { Py_VISIT(Py_TYPE(self)); return 0; } It's a little bit annoying to have to implement the GC protocol whereas a lock cannot contain other Python objects, it's not a container. It's just a thin wrapper to a C lock. There is exactly one strong reference: to the type. == Workaround: loop on gc.collect() == A workaround is to run gc.collect() in a loop until it returns 0 (no object was collected). == Traverse automatically? Nope. == Pablo Galindo attempts to automatically visit the type in the traverse function: https://bugs.python.org/issue40217 https://github.com/python/cpython/commit/0169d3003be3d072751dd14a5c84748ab6… Moreover, What's New in Python 3.9 contains a long section suggesting to implement a traverse function for this problem, but it doesn't suggest to track instances: https://docs.python.org/dev/whatsnew/3.9.html#changes-in-the-c-api This solution causes too many troubles, and so instead, traverse functions were defined on heap types to visit the type. Currently in the master branch, 89 types are defined as heap types on a total of 206 types (117 types are defined statically). I don't think that these 89 heap types respect the 3 conditions to collaborate with the GC. == How should we address this issue? == I'm not sure what should be done. Working around the issue by triggering multiple GC collections? Emit a warning in development mode if a heap type doesn't collaborate well with the GC? If core developers miss these bugs and have troubles to debug them, I expect that extension module authors would suffer even more. == GC+heap type bugs became common == I'm fixing such GC issue for 1 year as part as the work on cleaning Python objects at exit, and also indirectly related to subinterpreters. The behavior is surprising, it's really hard to dig into GC internals and understand what's going on. I wrote an article on this kind of "GC bugs": https://vstinner.github.io/subinterpreter-leaks.html Today, I learnt the hard way that defining a traverse is *not* enough. The type constructor (tp_new) must also track instances! See my fix for _multibytecodec related to CJK codecs: https://github.com/python/cpython/commit/11ef53aefbecfac18b63cee518a7184f77… https://bugs.python.org/issue42866 == Reference cycles are common == The GC only serves to break reference cycles. But reference cycles are rare, right? Well... First of all, most types create reference cycles involing themselves. For example, a type __mro__ tuple contains the type which already creates a ref cycle. Type methods can also contain a reference to the type. => The GC must break the cycle, otherwise the type cannot be destroyed When a function is defined in a Python module, the function __globals__ is the module namespace (module.__dict__) which... contains the function. Defining a function in a Python module also creates a reference cycle which prevents to delete the module namespace. If a function is used as a callback somewhere, the whole module remains "alive" until the reference to the callback is cleared. Example. os.register_at_fork() and codecs.register() callbacks are cleared really late during Python finalization. Currently, it's basically the last objects which are cleared at Python exit. After that, there is exactly one final GC collection. => The GC == Debug GC issues == * gc.get_referents() and gc.get_referrers() can be used to check traverse functions. * gc.is_tracked() can be used to check if the GC tracks an object. * Using the gdb debugger on gc_collect_main() helps to see which objects are collected. See for example the finalize_garbage() functions which calls finalizers on unreachable objects. * The solution is usually a missing traverse functions or a missing Py_VISIT() in an existing traverse function. == __del__ hack for debugging == If you want to play with the issue or if you have to debug a GC issue, you can use an object which logs a message when it's being deleted: class VerboseDel: def __del__(self): print("DELETE OBJECT") obj = VerboseDel() Warning: creating such object in a module also prevents to destroy the module namespace when the last reference to the module is deleted! __del__.__globals__ contains a reference to the module namespace, and obj.__class__ contains a reference to the type... Yeah, ref cycle and GC issues are fun! == Long email == Yeah, I like to put titles in my long emails. Enjoy. Happy hacking! Victor -- Night gathers, and now my watch begins. It shall not end until my death.

8 18

why include cpython/*.h, Need this macro ? # error "this header file must not be included directly"
by junyixie Jan. 11, 2021

Jan. 11, 2021

#ifndef Py_LIMITED_API # define Py_CPYTHON_FILEOBJECT_H # include "cpython/fileobject.h" # undef Py_CPYTHON_FILEOBJECT_H #endif cpython/fileobject.h ``` #ifndef Py_CPYTHON_FILEOBJECT_H # error "this header file must not be included directly" #endif ``` why not use #ifndef #define cpython/fileobject.h #ifndef Py_CPYTHON_FILEOBJECT_H #define Py_CPYTHON_FILEOBJECT_H .... #endif /* Py_CPYTHON_FILEOBJECT_H */

3 2

PEP 642 v3: Explicit patterns for structured pattern matching
by Nick Coghlan Jan. 11, 2021

Jan. 11, 2021

I’ve made a final round of updates to PEP 642 and submitted it to the Steering Council for consideration alongside PEP 634. As usual, the rendered version can be found here: https://www.python.org/dev/peps/pep-0642/ There's a Discourse thread at https://discuss.python.org/t/pep-642-v3-explicit-pattern-syntax-for-structu…, and the rest of the email covers the same points as the opening post in that thread. There are some pretty significant changes relative to v2, although I did already discuss most of them in the v2 thread at https://discuss.python.org/t/pep-642-constraint-pattern-syntax-for-structur… The PEP itself contains a list of major changes relative to PEP 634, so I won’t repeat that here: https://www.python.org/dev/peps/pep-0642/#appendix-c-summary-of-changes-rel… Instead I’ll summarise the parts that I consider most important: * ensuring that all “binding to the right” operations use the as keyword. This drove changes to both mapping patterns and class patterns. * explicitly qualifying both name bindings and value constraints with `as`, `==`, or `is`. This change makes it possible to make pattern matching available to users without having to resolve the thorny questions of what bare names and attribute references should do by default. It also opens up the possibility of potentially adding more value constraint options later (like `in` , `is not`, and `!=`) if those operations seem sufficiently compelling to be worth adding. * explicitly decoupling sequence pattern matching from iterable unpacking. The change to require qualification of name binding operations already breaks the alignment between the two, and that created an opportunity to simplify the grammar by only allowing square bracket based sequence patterns and eliminating both open sequence patterns and parenthesis based sequence patterns * changing class patterns to draw more of their syntactic inspiration from mapping patterns rather than from class instantiation * explicitly representing patterns in the AST, rather than treating patterns as pseudo-expressions all the way through to the code generation layer. Skipping this step makes the code fragile and hard to follow, as there isn’t actually any point in the AST that accepts both expressions and patterns, but with pattern parsing reusing expression nodes, you can’t tell from just looking at the AST which nodes expect subexpressions and which expect subpatterns. I’ll also quote the example match statement from the PEP abstract, which extracts “host” and “port” details from a 2 item sequence, a mapping with “host” and “port” keys, any object with “host” and “port” attributes, or a “host:port” string, treating the “port” as optional in the latter three cases: port = DEFAULT_PORT match expr: case [as host, as port]: pass case {"host" as host, "port" as port}: pass case {"host" as host}: pass case object{.host as host, .port as port}: pass case object{.host as host}: pass case str{} as addr: host, __, optional_port = addr.partition(":") if optional_port: port = optional_port case __ as m: raise TypeError(f"Unknown address format: {m!r:.200}") port = int(port) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

8 28

Extending python's graphlib module
by Touqir Sajed Jan. 10, 2021

Jan. 10, 2021

It was really good to see Python 3.9 adding the topological sort algorithm. What other algorithms might come next to the graphlib module? I think the basic ones like BFS, DFS, Bellman Ford's shortest path algorithm will be quite useful to have. Let me know if contributions are welcome. Thanks!

1 0

unittest of sequence equality
by Alan G. Isaac Jan. 10, 2021

Jan. 10, 2021

The following test fails because because `seq1 == seq2` returns a (boolean) NumPy array whenever either seq is a NumPy array. import unittest import numpy as np unittest.TestCase().assertSequenceEqual([1.,2.,3.], np.array([1.,2.,3.])) I expected `unittest` to rely only on features of a `collections.abc.Sequence`, which based on https://docs.python.org/3/glossary.html#term-sequence, I believe are satisfied by a NumPy array. Specifically, I see no requirement that a sequence implement __eq__ at all much less in any particular way. In short: a test named `assertSequenceEqual` should, I would think, work for any sequence and therefore (based on the available documentation) should not depend on the class-specific implementation of __eq__. Is that wrong? Thank you, Alan Isaac

10 16

Story behind vars() vs .__dict__
by Paul Sokolovsky Jan. 8, 2021

Jan. 8, 2021

Hello, I would easily bet 10 bucks that vars() is the least known, and least used, of the Python builtin functions. My mental model of it was: it was introduced (perhaps in Python3) to "harmonize" all the existing .__dict__ stuff, and provide more abstract interface for it, with .__dict__ patiently waiting its deprecation in CPython 10 or something. Was I wrong with thinking like that, and finding that vars() was already in the docs of CPython 1.4, the earliest doc version hosted at python.org: https://docs.python.org/release/1.4/lib/node26.html#SECTION0033000000000000… So it seems it was around "always", but then the question is why both vars() and .__dict__ are round for so long, and there was no (successful) attempt to stop breaking object encapsulation e.g. on transitioning to Python3000? Anybody has any references to the story of them? I can't find anything relevant so far. I in particular made sure to google for "vars site:python-history.blogspot.com" with 0 hits. Expected clarification on ".__dict__ breaking object encapsulation": Usually, object attributes are used to access "some small piece of data" "stored inside" an object. .__dict__ is not like that. Depending on how you look at it, its either-or: 1. A *metadata* about an object (object's representation as a dict), and then it makes sense to use a dedicated access means for that metadata, exactly what vars() provides. 2. Leak of an implementation detail, because a particular Python implementation uses a dict as internal storage for modules, classes, instances, and then .__dict__ just gives blatant access to this internal storage, bypassing normal object semantics. -- Best regards, Paul mailto:pmiscml@gmail.com

3 4

Summary of Python tracker Issues
by Python tracker Jan. 8, 2021

Jan. 8, 2021

ACTIVITY SUMMARY (2021-01-01 - 2021-01-08) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7549 ( -7) closed 47055 (+71) total 54604 (+64) Open issues with patches: 3026 Issues opened (39) ================== #42807: smtplib send_message should gives more clear exception if the https://bugs.python.org/issue42807 opened by oon #42808: Add PyType_Type.tp_vectorcall for type(obj) performance https://bugs.python.org/issue42808 opened by Dennis Sweeney #42809: Improve pickle tests for recursive data https://bugs.python.org/issue42809 opened by serhiy.storchaka #42812: @overload-ing method of parent class without actual implementa https://bugs.python.org/issue42812 opened by chaim422 #42813: Extra spaces cause unexpected EOF error in "compile" function https://bugs.python.org/issue42813 opened by xxm #42815: new thread doesn't copy context of the parent thread https://bugs.python.org/issue42815 opened by uriyyo #42819: readline 8.1 bracketed paste https://bugs.python.org/issue42819 opened by dtrodrigues #42820: Sphinx conf.py needs_version entry is outdated https://bugs.python.org/issue42820 opened by timhoffm #42823: Incorrect frame.f_lineno when frame.f_trace is set https://bugs.python.org/issue42823 opened by Mark.Shannon #42825: Build libraries with "/OPT:REF" linker optimization on Windows https://bugs.python.org/issue42825 opened by Austin-Lamb #42827: pegen parser: Multiline eval with assignment to function call: https://bugs.python.org/issue42827 opened by mdk #42830: tempfile mkstemp() leaks file descriptors if os.close() is not https://bugs.python.org/issue42830 opened by mieczyslaw.torchala #42831: IDLE fix colours for MacOS dark mode https://bugs.python.org/issue42831 opened by epaine #42832: classmethod change in 3.9 needs better documentation https://bugs.python.org/issue42832 opened by iritkatriel #42833: Lib/urllib/request.py: digest algorithm should be case insensi https://bugs.python.org/issue42833 opened by Pierre.Tardy #42834: [subinterpreters] Convert "global" static variable caches in _ https://bugs.python.org/issue42834 opened by kj #42836: docs: struct: clarify struct caching behaviour https://bugs.python.org/issue42836 opened by uniq10 #42837: Symbol table incorrectly identifies code as a generator, when https://bugs.python.org/issue42837 opened by Mark.Shannon #42838: Wait for cleanup coroutines before event loop is closed. https://bugs.python.org/issue42838 opened by xloem #42839: SourceFileLoader does not (fully) accept path-like objects https://bugs.python.org/issue42839 opened by favonia #42840: `type` takes **kwargs for __init_subclass__ https://bugs.python.org/issue42840 opened by esoma #42843: What min_sphinx for Python 3.10 https://bugs.python.org/issue42843 opened by mdk #42844: Turtle Module -- "onclick" arguments enchancement https://bugs.python.org/issue42844 opened by sc1341 #42848: asyncio produces an unexpected traceback with recursive __geta https://bugs.python.org/issue42848 opened by asleep-cult #42849: pool worker can't be terminated https://bugs.python.org/issue42849 opened by huangzhesi #42850: Process hangs when calling urllib.request in a multiprocessing https://bugs.python.org/issue42850 opened by fakufaku #42851: Subclassing Enum with ipaddress.IPv4Network/IPv6Network raises https://bugs.python.org/issue42851 opened by felixxm #42853: `OverflowError: signed integer is greater than maximum` in ssl https://bugs.python.org/issue42853 opened by amacd31 #42854: OpenSSL 1.1.1: use SSL_write_ex() and SSL_read_ex() https://bugs.python.org/issue42854 opened by christian.heimes #42855: pathlib.exists on Windows raises an exception on URL like/bad https://bugs.python.org/issue42855 opened by gaborjbernat #42856: ensurepip: add configure --with-wheel-pkg-dir=PATH to get whee https://bugs.python.org/issue42856 opened by vstinner #42859: smtplib: recipe for certifying message delivery https://bugs.python.org/issue42859 opened by dgoldenberg123 #42861: ipaddress - add ability to get next closet subnet of any prefi https://bugs.python.org/issue42861 opened by fasial.mahmood94 #42862: Use functools.lru_cache iso. _sqlite.Cache in sqlite3 module https://bugs.python.org/issue42862 opened by erlendaasland #42863: Python venv inconsistent folder structure on windows https://bugs.python.org/issue42863 opened by jmoguill2 #42864: Improve error messages regarding unclosed parentheses https://bugs.python.org/issue42864 opened by pablogsal #42865: sysconfig appends CFLAGS to LD https://bugs.python.org/issue42865 opened by Greentwip #42867: Entry Widget not editable on Windows 10, but is on Linux Ubunt https://bugs.python.org/issue42867 opened by jmccabe #42868: SpooledTemporaryFile.__iter__ is not transparent to rollover https://bugs.python.org/issue42868 opened by jinoh.kang.kr Most recent 15 issues with no replies (15) ========================================== #42868: SpooledTemporaryFile.__iter__ is not transparent to rollover https://bugs.python.org/issue42868 #42864: Improve error messages regarding unclosed parentheses https://bugs.python.org/issue42864 #42849: pool worker can't be terminated https://bugs.python.org/issue42849 #42844: Turtle Module -- "onclick" arguments enchancement https://bugs.python.org/issue42844 #42838: Wait for cleanup coroutines before event loop is closed. https://bugs.python.org/issue42838 #42836: docs: struct: clarify struct caching behaviour https://bugs.python.org/issue42836 #42834: [subinterpreters] Convert "global" static variable caches in _ https://bugs.python.org/issue42834 #42833: Lib/urllib/request.py: digest algorithm should be case insensi https://bugs.python.org/issue42833 #42832: classmethod change in 3.9 needs better documentation https://bugs.python.org/issue42832 #42830: tempfile mkstemp() leaks file descriptors if os.close() is not https://bugs.python.org/issue42830 #42825: Build libraries with "/OPT:REF" linker optimization on Windows https://bugs.python.org/issue42825 #42820: Sphinx conf.py needs_version entry is outdated https://bugs.python.org/issue42820 #42815: new thread doesn't copy context of the parent thread https://bugs.python.org/issue42815 #42813: Extra spaces cause unexpected EOF error in "compile" function https://bugs.python.org/issue42813 #42808: Add PyType_Type.tp_vectorcall for type(obj) performance https://bugs.python.org/issue42808 Most recent 15 issues waiting for review (15) ============================================= #42865: sysconfig appends CFLAGS to LD https://bugs.python.org/issue42865 #42864: Improve error messages regarding unclosed parentheses https://bugs.python.org/issue42864 #42856: ensurepip: add configure --with-wheel-pkg-dir=PATH to get whee https://bugs.python.org/issue42856 #42851: Subclassing Enum with ipaddress.IPv4Network/IPv6Network raises https://bugs.python.org/issue42851 #42848: asyncio produces an unexpected traceback with recursive __geta https://bugs.python.org/issue42848 #42844: Turtle Module -- "onclick" arguments enchancement https://bugs.python.org/issue42844 #42843: What min_sphinx for Python 3.10 https://bugs.python.org/issue42843 #42837: Symbol table incorrectly identifies code as a generator, when https://bugs.python.org/issue42837 #42836: docs: struct: clarify struct caching behaviour https://bugs.python.org/issue42836 #42834: [subinterpreters] Convert "global" static variable caches in _ https://bugs.python.org/issue42834 #42827: pegen parser: Multiline eval with assignment to function call: https://bugs.python.org/issue42827 #42825: Build libraries with "/OPT:REF" linker optimization on Windows https://bugs.python.org/issue42825 #42823: Incorrect frame.f_lineno when frame.f_trace is set https://bugs.python.org/issue42823 #42819: readline 8.1 bracketed paste https://bugs.python.org/issue42819 #42815: new thread doesn't copy context of the parent thread https://bugs.python.org/issue42815 Top 10 most discussed issues (10) ================================= #41798: [C API] Revisit usage of the PyCapsule C API with multi-phase https://bugs.python.org/issue41798 11 msgs #42812: @overload-ing method of parent class without actual implementa https://bugs.python.org/issue42812 10 msgs #39457: Add an autocommit property to sqlite3.Connection with a PEP 24 https://bugs.python.org/issue39457 9 msgs #42848: asyncio produces an unexpected traceback with recursive __geta https://bugs.python.org/issue42848 9 msgs #42867: Entry Widget not editable on Windows 10, but is on Linux Ubunt https://bugs.python.org/issue42867 9 msgs #42804: Unable to compile the cpython code x86 windows https://bugs.python.org/issue42804 8 msgs #42855: pathlib.exists on Windows raises an exception on URL like/bad https://bugs.python.org/issue42855 8 msgs #42115: Caching infrastructure for the evaluation loop: specialised op https://bugs.python.org/issue42115 7 msgs #42725: PEP 563: Should the behavior change for yield/yield from's https://bugs.python.org/issue42725 7 msgs #42837: Symbol table incorrectly identifies code as a generator, when https://bugs.python.org/issue42837 7 msgs Issues closed (68) ================== #4643: cgitb.html fails if getattr call raises exception https://bugs.python.org/issue4643 closed by iritkatriel #7946: Convoy effect with I/O bound threads and New GIL https://bugs.python.org/issue7946 closed by dabeaz #14014: codecs.StreamWriter.reset contract not fulfilled https://bugs.python.org/issue14014 closed by berker.peksag #14655: traceback module docs should show how to print/fomat an except https://bugs.python.org/issue14655 closed by iritkatriel #15106: Potential Bug in errors.c https://bugs.python.org/issue15106 closed by pablogsal #16132: ctypes incorrectly encodes .format attribute of memory views https://bugs.python.org/issue16132 closed by dabeaz #16894: Function attribute access doesn't invoke methods in dict subcl https://bugs.python.org/issue16894 closed by dabeaz #24464: "sqlite3_enable_shared_cache" deprecation warning when compili https://bugs.python.org/issue24464 closed by berker.peksag #24844: Python 3.5rc1 compilation error with Apple clang 4.2 included https://bugs.python.org/issue24844 closed by dabeaz #26407: csv.writer.writerows masks exceptions from __iter__ https://bugs.python.org/issue26407 closed by serhiy.storchaka #27436: Strange code in selectors.KqueueSelector https://bugs.python.org/issue27436 closed by dabeaz #31951: import curses is broken on windows https://bugs.python.org/issue31951 closed by serhiy.storchaka #32631: IDLE: revise zzdummy.py https://bugs.python.org/issue32631 closed by terry.reedy #32810: Expose ags_gen and agt_gen in asynchronous generators https://bugs.python.org/issue32810 closed by dabeaz #38413: Remove or change "Multithreading" section https://bugs.python.org/issue38413 closed by Vladimir Ryabtsev #39068: Base 85 encoding initialization race condition https://bugs.python.org/issue39068 closed by serhiy.storchaka #39508: no module curses error although i downloaded the module - wind https://bugs.python.org/issue39508 closed by serhiy.storchaka #40810: sqlite3 test CheckTraceCallbackContent fails for sqlite v3.7.3 https://bugs.python.org/issue40810 closed by berker.peksag #40822: Drop support for SQLite pre 3.7.15 https://bugs.python.org/issue40822 closed by erlendaasland #40823: Don't use obsolete unittest.makeSuite() in sqlite3 tests https://bugs.python.org/issue40823 closed by berker.peksag #40959: Remove unused and unneeded function declaration from sqlite3 h https://bugs.python.org/issue40959 closed by berker.peksag #41463: Avoid duplicating jump information from opcode.py in compile.c https://bugs.python.org/issue41463 closed by iritkatriel #41837: Upgrade installers to OpenSSL 1.1.1i https://bugs.python.org/issue41837 closed by steve.dower #42164: Python fails to compile in the Fedora Stable LTO buildbots https://bugs.python.org/issue42164 closed by vstinner #42179: Clarify chaining exceptions in tutorial/errors.rst https://bugs.python.org/issue42179 closed by Vladimir Ryabtsev #42257: platform.libc_ver() doesn't consider in case of executable is https://bugs.python.org/issue42257 closed by lemburg #42425: Possible leak in initialization of errmap for OSError https://bugs.python.org/issue42425 closed by serhiy.storchaka #42455: Add decorator_factory function to functools module https://bugs.python.org/issue42455 closed by ncoghlan #42508: macOS IDLE with Tk 8.6.10 in 3.9.1rc1 universal2 installer fai https://bugs.python.org/issue42508 closed by terry.reedy #42513: Socket.recv hangs https://bugs.python.org/issue42513 closed by Barney Stratford #42541: Tkinter colours wrong on MacOS universal2 https://bugs.python.org/issue42541 closed by epaine #42584: Upgrade macOS and Windows installers to use SQLite 3.34.0 https://bugs.python.org/issue42584 closed by erlendaasland #42587: memoryview rely on _Bool undefined behavior (test_buffer fails https://bugs.python.org/issue42587 closed by vstinner #42637: Python macos11.0 release note tkinter problems in news https://bugs.python.org/issue42637 closed by epaine #42681: Incorrect range checks/documentation in curses https://bugs.python.org/issue42681 closed by serhiy.storchaka #42692: Build fails on macOS when compiler doesn't define __has_builti https://bugs.python.org/issue42692 closed by ned.deily #42756: smtplib.LMTP.connect() raises TypeError if `timeout` is not sp https://bugs.python.org/issue42756 closed by corona10 #42759: Take into acount a Tcl interpreter when compare variables and https://bugs.python.org/issue42759 closed by serhiy.storchaka #42772: randrange() mishandles step when stop is None https://bugs.python.org/issue42772 closed by rhettinger #42789: Do not skip test_curses on non-tty https://bugs.python.org/issue42789 closed by serhiy.storchaka #42794: test_nntplib fails on CI https://bugs.python.org/issue42794 closed by vstinner #42798: pip search fails https://bugs.python.org/issue42798 closed by benjamin.peterson #42803: Traced line number is wrong for "if not __debug__" https://bugs.python.org/issue42803 closed by Mark.Shannon #42805: broken pygame integration into wxpython https://bugs.python.org/issue42805 closed by serhiy.storchaka #42806: Incorrect offsets in new parser for f-string substitutions https://bugs.python.org/issue42806 closed by pablogsal #42810: Nested if/else gets phantom else trace (3.10) https://bugs.python.org/issue42810 closed by Mark.Shannon #42811: Update docs for importlib.util.resolve_name() to use __spec__. https://bugs.python.org/issue42811 closed by brett.cannon #42814: Undefined behavior in Objects/genericaliasobject.c https://bugs.python.org/issue42814 closed by vstinner #42816: Add str.split_iter function https://bugs.python.org/issue42816 closed by serhiy.storchaka #42817: timedelta zeropadding hh https://bugs.python.org/issue42817 closed by krasmussen #42818: AttributeError: 'zipimporter' object has no attribute 'find_sp https://bugs.python.org/issue42818 closed by mowgli #42821: HTMLParser: subsequent duplicate attributes should be ignored https://bugs.python.org/issue42821 closed by ezio.melotti #42822: Missing backslashes in documentation for 3.8 and 3.9 https://bugs.python.org/issue42822 closed by mdk #42824: get_type_hints throws for local class reference https://bugs.python.org/issue42824 closed by gvanrossum #42826: typing.Iterable does not need __getitem__() method https://bugs.python.org/issue42826 closed by josh.r #42828: Python readline module https://bugs.python.org/issue42828 closed by gvanrossum #42829: get_type_hints throws for nested class with __future__ annotat https://bugs.python.org/issue42829 closed by gvanrossum #42835: Can't Built libxml2 When Using Python 3.9.x https://bugs.python.org/issue42835 closed by gvanrossum #42841: Add bitwise or operator to collections.abc Mapping and Mutable https://bugs.python.org/issue42841 closed by gvanrossum #42842: module 'logging' has not attribute 'handlers' https://bugs.python.org/issue42842 closed by vinay.sajip #42845: Third Party Module Installation using pip fails in Python 3.10 https://bugs.python.org/issue42845 closed by gvanrossum #42846: Using _multibytecodec module on Windows, test_threading/embed https://bugs.python.org/issue42846 closed by vstinner #42847: Normalise file encodings in Lib/sqlite3/test/ https://bugs.python.org/issue42847 closed by erlendaasland #42852: pprint fails in transformming non-breaking space https://bugs.python.org/issue42852 closed by serhiy.storchaka #42857: Fails to construct paths lead to "python is likely shutting do https://bugs.python.org/issue42857 closed by gvanrossum #42858: Incorrect return value for os.system() in recognizing import e https://bugs.python.org/issue42858 closed by gvanrossum #42860: Incompatible types in Python grammar https://bugs.python.org/issue42860 closed by lys.nikolaou #42866: test test_multibytecodec: Test_IncrementalEncoder.test_subinte https://bugs.python.org/issue42866 closed by vstinner

1 0

Where is the SQLite module maintainer
by Erlend Aasland Jan. 7, 2021

Jan. 7, 2021

Hi, everyone. I'm trying to find a reviewer for this trivial PR: https://github.com/python/cpython/pull/20530 (The PR fixes CheckTraceCallbackContent (in the sqlite3 test suite) for SQLite pre 3.7.15.) I've given up hunting for alternative reviewers (a process I find very uncomfortable, since I feel I'm just bugging people who have too much to do with things they're not interested in), so as a last resort, I'm trying the mailing list: Who can help me review code that touches the sqlite3 module code base? I've received a lot of constructive reviews from Victor Stinner, Dong-he Na and Pablo Galindo on my past sqlite3 PR's; thank you so much! I still feel uncomfortable requesting their review, as none of them are sqlite3 module maintainers. Erlend

5 8

Descriptors in dataclasses fields
by Josue Balandrano Coronel Jan. 4, 2021

Jan. 4, 2021

I've been exploring dataclasses for a few months now and they've proven to be very useful. The only downside is that there's not a simple way to use descriptors. Descriptors only work on class attributes (as per the docs: https://docs.python.org/3/howto/descriptor.html#closing-thoughts). This means that to use a descriptor in a data class we have to use typing.ClassVar like this @dataclass class Item: name: str price: typing.ClassVar[Decimal] = PriceValidator() Which is totally fine because of how descriptors work the previous syntax is a feature in dataclasses, IMHO. But, there's not a straight forward way to pass a value to the descriptor on init. Because ClassVars are not used by @dataclass to do its thing (as per the docs: https://docs.python.org/3/library/dataclasses.html#class-variables) This means that in the example above `price` is not going to be a parameter of the class Item's __init__ method. So the only way to do this is to either create an InitVar field or a regular field and then pass the value to the descriptor in __post_init__ @dataclass class Item: name: str price_val: InitVar[Decimal] price: typing.ClassVar[Decimal] = PriceValidator() def __post_init__(self, price_val: Decimal) -> None: self.price = price_val When using a regular field we can double the field's purpose by making it the field the descriptor is going to use: @dataclass class Item: name: str _price: Decimal price: typing.ClassVar[Decimal] = PriceValidator() def __post_init__(self) -> None: self.price = self._price And then in the descriptor implement __set_name__ like so: def __set_name(self, owner, name): self.name = f"_{name}" Personally, I don't like either option because it adds noice to the data class definition. Using an InitVar is the better option because that variable is clearly defined as init only and it's not present in an instance. Using a regular field adds unnecessary noice to the data class. Also, I think it clashes with the intent of descriptors since they're supposed to manage their data in any way they want. My questions are: - Are my assumptions correct? - Is this the intended behavior? Or what's the preferred way of using a descriptor in a dataclass field? - If this is not intended, could it be possible to add a `descriptor` parameter to the `field` method and treat the field accordingly? I couldn't find any information on the docs or the PEP. I could"ve missed something, sorry if this is the case :) Thanks! -- Josue https://www.rmcomplexity.com

2 3

[RELEASE] Python 3.10.0a4 is now available
by Pablo Galindo Salgado Jan. 4, 2021

Jan. 4, 2021

Happy new year to all of you. I hope you all have a great start of the year! And how to best celebrate that we have left 2020 behind that with a new Python alpha release? :) Go get it here: https://www.python.org/downloads/release/python-3100a4/ *Major new features of the 3.10 series, compared to 3.9* Python 3.10 is still in development. This releasee, 3.10.0a4 is the second of six planned alpha releases. Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process. During the alpha phase, features may be added up until the start of the beta phase (2021-05-03) and, if necessary, may be modified or deleted up until the release candidate phase (2021-10-04). Please keep in mind that this is a preview release and its use is not recommended for production environments. Many new features for Python 3.10 are still being planned and written. Among the new major new features and changes so far: * PEP 623 – Remove wstr from Unicode * PEP 604 – Allow writing union types as X | Y * PEP 612 – Parameter Specification Variables * PEP 626 – Precise line numbers for debugging and other tools. * bpo-38605: from __future__ import annotations (PEP 563) is now the default. * PEP 618 – Add Optional Length-Checking To zip. (Hey, fellow core developer, if a feature you find important is missing from this list, let me know.) The next pre-release of Python 3.10 will be 3.10.0a5, currently scheduled for 2021-02-01. *And now for something completely different* The Majumdar–Papapetrou spacetime <https://en.wikipedia.org/wiki/Sudhansu_Datta_Majumdar#Majumdar%E2%80%93Papa…> is one surprising solution of the coupled Einstein-Maxwell equations that describe a cluster of static charged black holes with the gravitational and the electrostatic forces cancelling each other out. Each one of these many black holes of the multi-black holes system has a spherical topology and follows the Reissner–Nordström metric <https://en.wikipedia.org/wiki/Reissner%E2%80%93Nordstr%C3%B6m_metric>. Unsurprisingly, the movement of a test particle in such spacetime is not only a very chaotic system but also has some fractals <https://arxiv.org/abs/gr-qc/9502014> hiding the complexity of its movement. Regards from cold London, Pablo Galindo Salgado

1 0

Enhancement request for PyUnicode proxies
by Nelson, Karl E. Jan. 4, 2021

Jan. 4, 2021

I was directed to post this request to the general Python development community so hopefully this is on topic. One of the weaknesses of the PyUnicode implementation is that the type is concrete and there is no option for an abstract proxy string to a foreign source. This is an issue for an API like JPype in which java.lang.Strings are passed back from Java. Ideally these would be a type derived from the Unicode type str, but that requires transferring the memory immediately from Java to Python even when that handle is large and will never be accessed from within Python. For certain operations like XML parsing this can be prohibitable, so instead of returning a str we return a JString. (There is a separate issue that Java method names and Python method names conflict so direct inheritance creates some problems.) The JString type can of course be transferred to Python space at any time as both Python Unicode and Java string objects are immutable. However the CPython API which takes strings only accepts the Unicode type objects which have a concrete implementation. It is possible to extend strings, but those extensions do not allow for proxing as far as I can tell. Thus there is no option currently to proxy to a string representation in another language. The concept of the using the duck type ``__str__`` method is insufficient as this indices that an object can become a string, rather than "this object is effectively a string" for the purposes of the CPython API. One way to address this is to use currently outdated copy of READY to extend Unicode objects to other languages. A class like JString would be an unready Unicode object which when READY is called transfers the memory from Java, sets up the flags and sets up a pointer to the code point representation. Unfortunately the READY concept is scheduled for removal and thus the chance to address the needs for proxying a Unicode to another languages representation may be limited. There may be other methods to accomplish this without using the concept of READY. So long as access to the code points go through the Unicode API and the Unicode object can be extended such that the actual code points may be located outside of the Unicode object then a proxy can still be achieved if there are hooks in it to decided when a transfer should be performed. Generally the transfer request only needs to happen once but the key issue being that the number of code points (nor the kind of points) will not be known until the memory is transferred. Java has much the same problem. Although they defined an interface class "java.lang.CharacterArray" the actually "java.lang.String" class is concrete and almost all API methods take a String rather than the base interface even when the base interface would have been adequate. Thus just like Python has difficulty treating a foreign string class as it would a native one, Java cannot treat a Python string as native one as well. So Python strings get represented as CharacterArray type which effectively limits it use greatly. Summary: * A String proxy would need the address of the memory in the "wstr" slot though the code points may be char[], wchar[] or int[] depending the representation in the proxy. * API calls to interpret the data would need to check to see if the data is transferred first, if not it would call the proxy dependent transfer method which is responsible for creating a block of code points and set up flags (kind, ascii, ready, and compact). * The memory block allocated would need to call the proxy dependent destructor to clean up with the string is done. * It is not clear if this would have impact on performance. Python already has the concept of a string which needs actions before it can be accessed, but this is scheduled for removal. Are there any plans currently to address the concept of a proxy string in PyUnicode API?

9 21

Re: [Python-ideas] Re: Add venv activate command to the venv modules
by Paul Sokolovsky Jan. 4, 2021

Jan. 4, 2021

Hello, On Mon, 4 Jan 2021 21:07:23 +0400 Abdur-Rahmaan Janhangeer <arj.python(a)gmail.com> wrote: > @Christopher Barker <pythonchb(a)gmail.com> > > You nailed it! I've used the conda activate command many times. > It should in theory be possible in Python > > @Chris Angelico <rosuav(a)gmail.com> > > Yes mere shell commands passing via Py does not work, > maybe a file way with results in current shell as @M.-A. Lemburg > <mal(a)egenix.com> > points out, venv already generates the files. On windows it has an > activate.bat Knowing that, you're a step away from creating a "subshell" solution for *yourself*. For POSIX systems, that would be: venv-shell.sh: ---- . $1/bin/activate /bins/sh ---- Voila! If you want to add "--shell" switch to "python3 -m venv" with that functionality, to benefit the whole mankind, not just yourself, I'm +1. I'd say, you can start writing a patch for that. But I'd warn you that it likely will be rejected, as it will require a bunch of sustained user support (i.e. it won't work as expected on various OSes and various shells). -- Best regards, Paul mailto:pmiscml@gmail.com

1 0

Review patch fixing packed bitfields in ctypes struct/union
by Filipe Laíns Jan. 2, 2021

Jan. 2, 2021

Hi folks, I know this is a high-volume list so sorry for being yet another voice screaming into your mailbox, but I do not know how else to handle this. A few months ago, I opened a pull request fixing packed bitfields in ctypes struct/union, which right now are being incorrectly created. The offsets and size are all wrong in some cases, fields which should be packed end up with padding between them. bpo: https://bugs.python.org/issue29753 PR: https://github.com/python/cpython/pull/19850 Since ctypes has no maintainer it has been very hard tracking down someone who is up to review this. So, if this issue sparks some interest to you, or you just want to help me out, please have a look :) Cheers, Filipe Laíns

4 4

Summary of Python tracker Issues
by Python tracker Jan. 1, 2021

Jan. 1, 2021

ACTIVITY SUMMARY (2020-12-25 - 2021-01-01) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7556 ( +2) closed 46984 (+59) total 54540 (+61) Open issues with patches: 3033 Issues opened (44) ================== #40521: [subinterpreters] Make free lists and unicode caches per-inter https://bugs.python.org/issue40521 reopened by vstinner #40631: PEG Parser: Cannot used starred expression in parenthesised ex https://bugs.python.org/issue40631 reopened by pablogsal #41224: Document is_annotate() in symtable and update doc strings https://bugs.python.org/issue41224 reopened by serhiy.storchaka #42744: pkg_resources seems to treat python 3.10 as python 3.1 https://bugs.python.org/issue42744 opened by RhinosF1 #42746: python3.7.3 - ssl.SSLContext() - "Killed" https://bugs.python.org/issue42746 opened by hgmmym #42747: Remove Py_TPFLAGS_HAVE_VERSION_TAG flag? https://bugs.python.org/issue42747 opened by vstinner #42752: multiprocessing Queue leaks a file descriptor associated with https://bugs.python.org/issue42752 opened by crazycasta #42756: smtplib.LMTP.connect() raises TypeError if `timeout` is not sp https://bugs.python.org/issue42756 opened by wuestengecko #42757: Class has two prototypes https://bugs.python.org/issue42757 opened by SinTh0r4s #42759: Take into acount a Tcl interpreter when compare variables and https://bugs.python.org/issue42759 opened by serhiy.storchaka #42760: inspect.iscoroutine returns False for asynchronous generator m https://bugs.python.org/issue42760 opened by plammens #42762: infinite loop resulted by "yield" https://bugs.python.org/issue42762 opened by xxm #42763: Exposing a race in the "_warnings" resulting Python parser cra https://bugs.python.org/issue42763 opened by xxm #42765: Introduce new data model method __iter_items__ https://bugs.python.org/issue42765 opened by conqp #42766: urllib.request.HTTPPasswordMgr uses commonprefix instead of co https://bugs.python.org/issue42766 opened by nagdon #42767: Review usage of atomic variables in signamodule.c https://bugs.python.org/issue42767 opened by vstinner #42769: concurrent.futures.ProcessPoolExecutor is unable to forward ex https://bugs.python.org/issue42769 opened by damien.levac #42771: Implement interactive hotkey, Ctrl+L to clear the console in W https://bugs.python.org/issue42771 opened by mixmastamyk #42772: randrange() mishandles step when stop is None https://bugs.python.org/issue42772 opened by rhettinger #42773: build.yml workflow not testing on pushes https://bugs.python.org/issue42773 opened by ammar2 #42775: __init_subclass__ should be called in __init__ https://bugs.python.org/issue42775 opened by ethan.furman #42776: The string find method shows the problem https://bugs.python.org/issue42776 opened by andyye #42777: WindowsPath does not implement is_mount but ntpath implements https://bugs.python.org/issue42777 opened by db #42778: Add follow_symlinks=True parameter to both os.path.samefile() https://bugs.python.org/issue42778 opened by Tom Hale #42780: os.set_inheritable() fails for O_PATH file descriptors on Linu https://bugs.python.org/issue42780 opened by cptpcrd #42782: shutil.move creates a new directory even on failure https://bugs.python.org/issue42782 opened by winsonluk #42783: asyncio.sleep(0) idiom is not documented https://bugs.python.org/issue42783 opened by simonw #42784: issues with object.h includes https://bugs.python.org/issue42784 opened by bigbossbro08 #42785: Support operator module callables in inspect.signature https://bugs.python.org/issue42785 opened by plammens #42787: email.utils.getaddresses improper parsing of unicode realnames https://bugs.python.org/issue42787 opened by konstantin2 #42789: Do not skip test_curses on non-tty https://bugs.python.org/issue42789 opened by serhiy.storchaka #42790: test.regrtest outputs to stdout instead of stderr https://bugs.python.org/issue42790 opened by serhiy.storchaka #42791: There is no way to json encode object to str. https://bugs.python.org/issue42791 opened by kaltovichartyom #42794: test_nntplib fails on CI https://bugs.python.org/issue42794 opened by serhiy.storchaka #42795: Asyncio loop.create_server doesn't bind to any interface if ho https://bugs.python.org/issue42795 opened by plammens #42796: tempfile doesn't seem to play nicely with os.chdir on Windows https://bugs.python.org/issue42796 opened by Gabriele Tornetta #42797: Allow doctest to select tests via -m/--match option https://bugs.python.org/issue42797 opened by costas-basdekis #42798: pip search fails https://bugs.python.org/issue42798 opened by Paul Watson #42799: Please document fnmatch LRU cache size (256) and suggest alter https://bugs.python.org/issue42799 opened by joshtriplett #42800: Traceback objects allow accessing frame objects without trigge https://bugs.python.org/issue42800 opened by ammar2 #42801: Exception catching function crashes on recursive list https://bugs.python.org/issue42801 opened by steven.daprano #42802: distutils: Remove bdist_wininst command https://bugs.python.org/issue42802 opened by vstinner #42803: Traced line number is wrong for "if not __debug__" https://bugs.python.org/issue42803 opened by nedbat #42804: Unable to compile the cpython code x86 windows https://bugs.python.org/issue42804 opened by pbharat Most recent 15 issues with no replies (15) ========================================== #42804: Unable to compile the cpython code x86 windows https://bugs.python.org/issue42804 #42803: Traced line number is wrong for "if not __debug__" https://bugs.python.org/issue42803 #42800: Traceback objects allow accessing frame objects without trigge https://bugs.python.org/issue42800 #42798: pip search fails https://bugs.python.org/issue42798 #42795: Asyncio loop.create_server doesn't bind to any interface if ho https://bugs.python.org/issue42795 #42791: There is no way to json encode object to str. https://bugs.python.org/issue42791 #42790: test.regrtest outputs to stdout instead of stderr https://bugs.python.org/issue42790 #42789: Do not skip test_curses on non-tty https://bugs.python.org/issue42789 #42784: issues with object.h includes https://bugs.python.org/issue42784 #42783: asyncio.sleep(0) idiom is not documented https://bugs.python.org/issue42783 #42777: WindowsPath does not implement is_mount but ntpath implements https://bugs.python.org/issue42777 #42769: concurrent.futures.ProcessPoolExecutor is unable to forward ex https://bugs.python.org/issue42769 #42766: urllib.request.HTTPPasswordMgr uses commonprefix instead of co https://bugs.python.org/issue42766 #42760: inspect.iscoroutine returns False for asynchronous generator m https://bugs.python.org/issue42760 #42757: Class has two prototypes https://bugs.python.org/issue42757 Most recent 15 issues waiting for review (15) ============================================= #42802: distutils: Remove bdist_wininst command https://bugs.python.org/issue42802 #42797: Allow doctest to select tests via -m/--match option https://bugs.python.org/issue42797 #42794: test_nntplib fails on CI https://bugs.python.org/issue42794 #42789: Do not skip test_curses on non-tty https://bugs.python.org/issue42789 #42783: asyncio.sleep(0) idiom is not documented https://bugs.python.org/issue42783 #42782: shutil.move creates a new directory even on failure https://bugs.python.org/issue42782 #42780: os.set_inheritable() fails for O_PATH file descriptors on Linu https://bugs.python.org/issue42780 #42778: Add follow_symlinks=True parameter to both os.path.samefile() https://bugs.python.org/issue42778 #42775: __init_subclass__ should be called in __init__ https://bugs.python.org/issue42775 #42773: build.yml workflow not testing on pushes https://bugs.python.org/issue42773 #42772: randrange() mishandles step when stop is None https://bugs.python.org/issue42772 #42759: Take into acount a Tcl interpreter when compare variables and https://bugs.python.org/issue42759 #42756: smtplib.LMTP.connect() raises TypeError if `timeout` is not sp https://bugs.python.org/issue42756 #42752: multiprocessing Queue leaks a file descriptor associated with https://bugs.python.org/issue42752 #42737: PEP 563: drop annotations for complex assign targets https://bugs.python.org/issue42737 Top 10 most discussed issues (10) ================================= #42794: test_nntplib fails on CI https://bugs.python.org/issue42794 17 msgs #42742: Add abc.Mapping to dataclass https://bugs.python.org/issue42742 9 msgs #40631: PEG Parser: Cannot used starred expression in parenthesised ex https://bugs.python.org/issue40631 8 msgs #42737: PEP 563: drop annotations for complex assign targets https://bugs.python.org/issue42737 8 msgs #42747: Remove Py_TPFLAGS_HAVE_VERSION_TAG flag? https://bugs.python.org/issue42747 8 msgs #1635741: Py_Finalize() doesn't clear all Python objects at exit https://bugs.python.org/issue1635741 8 msgs #29708: support reproducible Python builds https://bugs.python.org/issue29708 7 msgs #40522: [subinterpreters] Get the current Python interpreter state fro https://bugs.python.org/issue40522 7 msgs #42762: infinite loop resulted by "yield" https://bugs.python.org/issue42762 7 msgs #42772: randrange() mishandles step when stop is None https://bugs.python.org/issue42772 6 msgs Issues closed (57) ================== #8481: doc: ctypes no need to explicitly allocate writable memory wit https://bugs.python.org/issue8481 closed by orsenthil #15303: Minor revision to the <BaseWidget._setup> method in Tkinter https://bugs.python.org/issue15303 closed by serhiy.storchaka #23328: urllib.request fails for proxy credentials that contain a '/' https://bugs.python.org/issue23328 closed by orsenthil #27640: Add --disable-test-modules configure option to not build nor i https://bugs.python.org/issue27640 closed by vstinner #27794: setattr a read-only property; the AttributeError should show t https://bugs.python.org/issue27794 closed by rhettinger #30963: xxlimited.c XxoObject_Check should be XxoObject_CheckExact https://bugs.python.org/issue30963 closed by petr.viktorin #32825: warn user of creation of multiple Tk instances https://bugs.python.org/issue32825 closed by serhiy.storchaka #34752: warnings.warn fails silently with unicode input https://bugs.python.org/issue34752 closed by corona10 #37193: Memory leak while running TCP/UDPServer with socketserver.Thre https://bugs.python.org/issue37193 closed by pablogsal #39171: Missing default root in tkinter simpledialog.py https://bugs.python.org/issue39171 closed by serhiy.storchaka #39177: In tkinter, simple dialogs, askstrings, etc. with flexible coo https://bugs.python.org/issue39177 closed by serhiy.storchaka #39465: [subinterpreters] Design a subinterpreter friendly alternative https://bugs.python.org/issue39465 closed by vstinner #40288: [subinterpreters] atexit module should not be loaded more than https://bugs.python.org/issue40288 closed by vstinner #40494: collections.abc.Callable and type variables https://bugs.python.org/issue40494 closed by gvanrossum #41450: OSError is not documented in ssl library, but still can be thr https://bugs.python.org/issue41450 closed by orsenthil #41781: Typos in typing.py https://bugs.python.org/issue41781 closed by shihai1991 #41884: tempfile.py TemporaryDirectory/mkdtemp defaults to mode 0o700, https://bugs.python.org/issue41884 closed by ray-sit #42042: sphinx3 renders diffrently docs.python.org for 3.10 https://bugs.python.org/issue42042 closed by corona10 #42222: Modernize integer test/conversion in randrange() https://bugs.python.org/issue42222 closed by rhettinger #42243: Don't access the module dictionary directly https://bugs.python.org/issue42243 closed by erlendaasland #42318: [tkinter] surrogate pairs in Tcl/Tk string when pasting an emo https://bugs.python.org/issue42318 closed by serhiy.storchaka #42393: Raise overflow errors iso. deprecation warnings in socket.hton https://bugs.python.org/issue42393 closed by serhiy.storchaka #42655: Fix subprocess extra_groups gid conversion https://bugs.python.org/issue42655 closed by serhiy.storchaka #42687: tokenize module does not recognize Barry as FLUFL https://bugs.python.org/issue42687 closed by esoma #42694: Failed test_new_curses_panel in test_curses https://bugs.python.org/issue42694 closed by vstinner #42700: xml.parsers.expat.errors description of codes/messages is flip https://bugs.python.org/issue42700 closed by serhiy.storchaka #42713: Segmentation fault in running eval() with large expression siz https://bugs.python.org/issue42713 closed by terry.reedy #42714: Segmentation fault in running compile() with large expression https://bugs.python.org/issue42714 closed by terry.reedy #42715: Segmentation fault in running exec() with large expression siz https://bugs.python.org/issue42715 closed by terry.reedy #42716: Segmentation fault in running ast.parse() with large expressio https://bugs.python.org/issue42716 closed by terry.reedy #42721: Using of simple dialogs without default root window https://bugs.python.org/issue42721 closed by serhiy.storchaka #42724: Change library name when building. https://bugs.python.org/issue42724 closed by eric.smith #42728: Typo in documentation: importlib.metadata https://bugs.python.org/issue42728 closed by barry #42733: io's r+ mode truncate(0) https://bugs.python.org/issue42733 closed by terry.reedy #42734: Outdated CodeType call in "bogus_code_obj.py" https://bugs.python.org/issue42734 closed by serhiy.storchaka #42740: typing.py get_args and get_origin should support PEP 604 and 6 https://bugs.python.org/issue42740 closed by gvanrossum #42741: Sync 3.9's whatsnew document in 3.10 with 3.9 branch https://bugs.python.org/issue42741 closed by pablogsal #42745: [subinterpreters] Make the type attribute lookup cache per-int https://bugs.python.org/issue42745 closed by vstinner #42748: test_asdl_parser: load_module() method is deprecated https://bugs.python.org/issue42748 closed by corona10 #42749: test_tcl failed on 32-bit POWER6 AIX 3.9: big int issue with 9 https://bugs.python.org/issue42749 closed by vstinner #42750: tkinter.Variable equality inconsistency https://bugs.python.org/issue42750 closed by serhiy.storchaka #42751: Imaplib https://bugs.python.org/issue42751 closed by steven.daprano #42753: "./configure" linux chrome beta https://bugs.python.org/issue42753 closed by eric.smith #42754: Unpacking of literals inside other literals should be optimise https://bugs.python.org/issue42754 closed by pablogsal #42755: sqlite3.Connection.backup default value is diffrent between im https://bugs.python.org/issue42755 closed by pablogsal #42758: pathlib.Path to support the "in" operator (x in y) https://bugs.python.org/issue42758 closed by eric.smith #42761: Why does python's Popen fail to pass environment variables on https://bugs.python.org/issue42761 closed by benjamin.peterson #42764: HTMLParser close() issue https://bugs.python.org/issue42764 closed by aeh #42768: super().__new__() of list expands arguments https://bugs.python.org/issue42768 closed by serhiy.storchaka #42770: Typo in email.headerregistry docs https://bugs.python.org/issue42770 closed by zach.ware #42774: 'ipaddress' module, bad result for 'is_private' on "192.0.0.0" https://bugs.python.org/issue42774 closed by trevormarvin #42779: pow() of huge input does not complete https://bugs.python.org/issue42779 closed by serhiy.storchaka #42781: functools.cached_property docs should explain that it is non-o https://bugs.python.org/issue42781 closed by rhettinger #42786: Different repr for collections.abc.Callable and typing.Callabl https://bugs.python.org/issue42786 closed by Prometheus3375 #42788: Issue with Python’s Floor Division https://bugs.python.org/issue42788 closed by rhettinger #42792: [MacOS] Can't open file in a separate (threading.Thread) threa https://bugs.python.org/issue42792 closed by mark.dickinson #42793: Bug of round function https://bugs.python.org/issue42793 closed by serhiy.storchaka

1 0

Need help with python
by hadi esmaeely Jan. 1, 2021

Jan. 1, 2021

hi my name is hadi i'm from iran (the country which filtering others and be filtered by others) i have started programming with python about 3 months and i'm very interested in learning programming and python language but duo to limitations of technologies and filtering the learning sources in my country i can not find proper source for learning programming(we can use vpn for some of websites but not effective enough). Because of that I must learn and work in another country (In my country, programmers are not valued). I am very interested in learning and immigrating to the Netherlands(my dream country) for work and to meet with you and other great programmers. But I don't know where to begin and how I can learn programming and python language in the right way without wasting time. I have studied 10 hours a day but I cannot conclude results. i have read this books and articles: beginning python from novice to professional django 3 by example antonio mele django for apis william s vincent django for professionals django web development with python - packt fluent python head first python practical python design patterns apress(currently reading) python in a nutshell(currently reading) the self taught programmer python 3 for absolute beginners some django official documents some python official documents but i don't know how to use my learnings from books If you help me to know how I can study in the right way ,I will appreciate you for my whole life thank you and happy new year :)

3 2