Mailman 3 July 2020 - docs

[issue35105] Document that CPython accepts "invalid" identifiers
by STINNER Victor 03 Jul '20

03 Jul '20

New submission from STINNER Victor <vstinner(a)redhat.com>: The Python 3 language has a strict definition for an identifier: https://docs.python.org/dev/reference/lexical_analysis.html#identifiers ... but in practice, it's possible to create "invalid" identifiers using setattr(). Example with PyPy: $ pypy Python 2.7.13 (0e7ea4fe15e82d5124e805e2e4a37cae1a402d4b, Apr 12 2018, 14:50:12) >>>> class A: pass >>>> >>>> a=A() >>>> setattr(a, "1", 2) >>>> getattr(a, "1") 2 >>>> vars(a) {'1': 2} >>>> a.__dict__ {'1': 2} >>>> a.1 File "<stdin>", line 1 a.1 ^ SyntaxError: invalid syntax The exact definition of "identifiers" is a common question. Recent examples: * bpo-25205 * [Python-Dev] Arbitrary non-identifier string keys when using **kwargs https://mail.python.org/pipermail/python-dev/2018-October/155435.html It would be nice to document the answer. Maybe in the Langage Specification, maybe in the setattr() documentation, maybe in a FAQ, maybe everywhere? ---------- assignee: docs@python components: Documentation messages: 328816 nosy: docs@python, serhiy.storchaka, steven.daprano, vstinner priority: normal severity: normal status: open title: Document that CPython accepts "invalid" identifiers versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue35105> _______________________________________

11 21

[issue41133] Insufficient description of cyclic garbage collector for C API
by Daniel Barkalow 01 Jul '20

01 Jul '20

New submission from Daniel Barkalow <barkalow(a)iabervon.org>: Nothing in the documentation gives you enough information to find what you're doing wrong if you have the following bug for an object tracked by the GC: (a) you store a borrowed reference to an object whose lifetime is always longer that the buggy object (and never decrement it); (b) you visit that object in your tp_traverse. Despite the bug, everything works nearly all of the time. However, when it fails, the effect makes no sense from a documentation-only understanding of the garbage collection (unless Python is built with --with-assertions, perhaps). The only sequence in which it fails is: * All of the objects that will reference the victim get moved into an older generation. * The victim gets exactly as many buggy references from objects in its own generation as there are good references, and no good references from objects in its own generation. * Garbage collection runs for only the younger generation. At this point, the victim is marked as garbage despite the fact that there was a good object referencing it at all times. Reading the Python source, it becomes clear that the garbage collector handles references from older generations not by traversing the objects in older generations, but by comparing the count of references from other young objects to the usual reference count. That is, visiting an object you don't hold a reference to may cause that object to get collected, rather than protecting it, even if there are other reasons not to collect it. The best fix would probably be a warning in the documentation for tp_traverse that visiting an object you do not hold a strong reference to can cause inexplicable effects, because the intuitive guess would be that it could only cause memory leaks. It would probably also be worth mentioning in the documentation of the garbage collector something about its algorithm, so people have a hint that references from older objects aren't necessarily sufficient to overcome buggy use of the C API. ---------- assignee: docs@python components: C API, Documentation messages: 372445 nosy: docs@python, iabervon priority: normal severity: normal status: open title: Insufficient description of cyclic garbage collector for C API versions: Python 3.7, Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue41133> _______________________________________

4 4

[issue41184] Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 guidelines for commenting
by Anthropologist 01 Jul '20

01 Jul '20

New submission from Anthropologist <python(a)anthropo.org>: IDLE's Comment Out Region formatting tool currently adds two octothorpe characters followed by no spaces. The Uncomment Region tool removes up to 2 octothorpes but does not remove spaces. The Python Style Guide (PEP 8) specifies that: "An inline comment is a comment on the same line as a statement. Inline comments should be separated by at least two spaces from the statement. They should start with a # and a single space." I propose reconciling these conflicting approaches to commenting out code, either by changing IDLE's behavior (i.e., instead of ##, Comment Out Region should insert # followed by a space, per PEP 8), or by amending the Python Style Guide to provide clear instructions about commenting out code, which is arguably a different use of comments than a single-line comment for the sake of explaining code. If the resolution involves changing IDLE's behavior, the Uncomment Region feature should also be changed to remove the space character after the octothorpe. As it currently stands, this feature fails to uncomment code that has been manually commented out following the guidelines in PEP 8. ---------- assignee: docs@python components: Documentation, IDLE messages: 372757 nosy: anthropologist, docs@python, terry.reedy priority: normal severity: normal status: open title: Reconciling IDLE's Comment Out Region / Uncomment Region with PEP 8 guidelines for commenting type: behavior versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue41184> _______________________________________

3 2

[issue41106] os.scandir() Windows bug dir_entry.stat() not works on file during writing.
by Eryk Sun 01 Jul '20

01 Jul '20

Eryk Sun <eryksun(a)gmail.com> added the comment: > As far as I know os.stat() resets d.stat() maybe should be added > some option to d.stat() to force update(). d.stat(nt_force_update=True). It depends on the filesystem. NTFS will update the directory entry as soon as the link is accessed by CreateFileW. But that's relatively expensive, and actually one of the more expensive steps in an os.stat call. > I am not sure if os.path.getmtime() can reset d.stat(). genericpath.getmtime calls os.stat: https://github.com/python/cpython/blob/d0981e61a5869c48e0a70a51296755839127… lexists, exists, getctime, getatime, getmtime, getsize, isdir, and isfile could be modified to call WinAPI GetFileAttributesExW [1], which is implemented via NtQueryFullAttributesFile [2], an optimized system call to get a file's network-open information. This can be significantly faster than the sequence of system calls that are required by os.stat. Note that this does not update the NTFS directory entry for the accessed link, unlike CreateFileW, but it does return updated information. The GetFileAttributesExW result would be used if the call succeeds and the file isn't a reparse point. Otherwise fall back on os.stat (win32_xstat_impl). If passed an fd, try GetFileInformationByHandleEx to get the FileBasicInfo and FileStandardInfo, or use a single system call via NTAPI NtQueryInformationFile: FileNetworkOpenInformation, which is the same info that GetFileAttributesExW returns. This could be implemented in C as nt._basic_stat(filename, follow_symlinks=True), where follow_symlinks means the expanded set of Windows name-surrogate reparse points. The C implementation would fall back on win32_xstat_impl. Note that a basic stat would not guarantee to return the following fields: st_ino, st_dev, and st_nlink. Alternatively, it could be implemented as a keyword-only basic=True option for os.stat, which would be ignored by POSIX. This way the high-level functions could continue to have a common implementation in genericpath.py. [1] https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfi… [2] https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zw… ---------- _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue41106> _______________________________________

1 0

[issue41153] [easy Doc] "PyPreConfig_InitIsolatedConfig" and "PyPreConfig_InitPythonConfig" are given opposite documentation.
by William Pickard 01 Jul '20

01 Jul '20

New submission from William Pickard <lollol222gg(a)gmail.com>: The initconfig API functions "PyPreConfig_InitPythonConfig" and "PyPreConfig_InitIsolatedConfig" are mistakenly documented for the other method. ---------- assignee: docs@python components: Documentation messages: 372531 nosy: WildCard65, docs@python priority: normal pull_requests: 20358 severity: normal status: open title: [easy Doc] "PyPreConfig_InitIsolatedConfig" and "PyPreConfig_InitPythonConfig" are given opposite documentation. type: enhancement versions: Python 3.10 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue41153> _______________________________________

1 2

[issue41106] os.scandir() Windows bug dir_entry.stat() not works on file during writing.
by Cezary Wagner 01 Jul '20

01 Jul '20

Cezary Wagner <cezary.wagner(a)gmail.com> added the comment: As far as I know os.stat() resets d.stat() maybe should be added some option to d.stat() to force update(). d.stat(nt_force_update=True). I am not sure if os.path.getmtime() can reset d.stat(). os.stat() is 2x times slower than os.path.getmtime() and os.path.getmtime is 16x slower than d.stat(). MAJOR PROBLEM is PERFORMANCE of os.stat() since for directories with 1000 files it takes big number of seconds to read all stats - something wrong is here I think since Windows Explorer is doing it very fast. So I can not use os.stat() ONLY and it complicates code since I need to use os.stat() after d.stat() if files is OLDER THAN because if I use os.stat() the most program time will be these calls. Do you know which code makes such reset of d.stat()? If there is not possible optimization of there is need DOCUMENTATION update because it is really hard to understand why it is not working under windows some REMARKS can help me and others. I have still believe that some optimization is possible for Windows. Maybe it can be force to read stat by os.scandir(force_scan_stat=True) so all directory entries will be have cached stats before d.stat() is called. It can be faster I think since less calls from Python and probably better Windows API for it and same for Linux. I will study C code later if it is possible or write some snippet. ---------- _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue41106> _______________________________________

1 0