Mailman 3 April 2021 - Python-Dev

expanduser('~other') reliability and future
by Barney Gale 28 Apr '21

28 Apr '21

There’s been a bit of discussion on the reliability of getting other users' home directories via os.path.expanduser(), mostly on issue42998 <https://bugs.python.org/issue42998>. In msg390503 <https://bugs.python.org/issue42998#msg390503>, Serhiy Storchaka pointed out that ntpath only makes an educated guess of other user’s home directories. In msg390535 <https://bugs.python.org/issue42998#msg390535>, Eryk Sun supplied an improved ntpath implementation that makes use of winreg and ctypes. In msg391970 <https://bugs.python.org/issue42998#msg391970>, Steve Dower pointed out that, if retrieving another user’s home directory on Windows is unlikely to be made reliable, we should look at deprecating this functionality in expanduser() From a bit of googling, Python seems to be an outlier in having a function to retrieve another user’s home directory. I am encouraged by Eryk’s patch, but a little out of my depth with Windows stuff. Any views on this? Is expanduser(‘~other’) fixable and worth fixing? If not, should we deprecate this functionality? Or something else? Barney

5 4

Revive PEP 396 -- Module Version Numbers ?
by Christopher Barker 27 Apr '21

27 Apr '21

I've already asked on python-ideas, but it was suggested that as there's a PEP already, it's time to move this along. So here's the pitch. Over the years, I've seen __version__ used very broadly but not *quite* in all packages. I've always known it was a convention, not a requirement. But it turns out it's not even a "official" convention. But it really would be nice if there was a consistent way that I could count on to get the version of a package at run time from the package itself. Turns out this was suggested in PEP 396 -- and deferred almost 13 years ago! https://www.python.org/dev/peps/pep-0396/ In the status note, it says: """ Further exploration of the concepts covered in this PEP has been deferred for lack of a current champion interested in promoting the goals of the PEP and collecting and incorporating feedback, and with sufficient available time to do so effectively. """ Well, I may be willing to be that champion, if a core dev is willing to sponsor. And, well, after 13 years, we've seen __version__ be very broadly, though certainly not universally used. Honestly, I haven't looked to see to what extent setuptools supports it, but will, of course, do so if folks think this is worth pursuing. And the PyPA has moved toward "distribution" meta data, preliminarily supported by importlib.metadata.version. So maybe this is a settled issue, and we just need to change the status of the PEP. But for my part, I FAR prefer the version info to be embedded in the code of the package in a simple way, rather than hiding among the metadata, and requiring importlib.metadata.version to get a version at runtime. I note that PEP8 uses __version__ as an example of a "module level dunder" -- but only suggests where it should be put, not how it be used :-) Of course, there will be a need to update the PEP to match current practice, and if it is me doing it, I'd make it very simple So what do you'all think? After thirteen years, it would be nice to put this to bed. I think the next step is to see if I can find a sponsor :-) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

16 48

bz2.BZ2File doesn't support name?
by roy＠panix.com 27 Apr '21

27 Apr '21

I was surprised recently to discover that BZ2File (at least in 3.7) doesn't have a name attribute. Is there some fundamental reason name couldn't be supported, or is it just a bug that it wasn't implemented?

5 6

Deferral of PEP 648 - Extensible customizations of the interpreter at startup
by Barry Warsaw 27 Apr '21

27 Apr '21

Hi Mario, The Python Steering Council today decided that we will defer consideration of PEP 648 to Python 3.11. On March 30, 2021 we sent the following feedback to you via python-dev, which began a discussion thread: https://mail.python.org/archives/list/python-dev@python.org/message/OGYZZZ4… We on the SC extend our thanks again for the PEP, and encourage you to continue to work on this PEP for pronouncement in Python 3.11. Cheers, -Barry (on behalf of the Python Steering Council)

2 1

How to keep Signaling NaN unchanged when convert hex to single-precision float ?
by Huang, Yang 27 Apr '21

27 Apr '21

Hi, Code snippet 1 and 2 convert an hex to float, but the SNaN is changed to QNaN automatically. It follows IEEE 754 behavior. While Code snippet 3 converts an hex to double, and the SNaN keeps. Is there any solution to keep SNaN float unchanged in Code snippet 1 and 2? 1. >>> i = int('7f800001', 16) >>> cp = pointer(c_uint32(i)) >>> fp = cast(cp, POINTER(c_float)) >>> print(fp.contents.value) # nan >>> print(struct.pack(">f", fp.contents.value).hex()) 7fc00001 1. >>> f = struct.unpack('!f', bytes.fromhex('7f800001'))[0] >>> f # nan >>> hex(struct.unpack('<I', struct.pack('<f', f))[0]) '0x7fc00001' 1. >>> i = int('7FF0000000000001', 16) >>> cp = pointer(c_uint64(i)) >>> fp = cast(cp, POINTER(c_double)) >>> print(fp.contents.value) nan >>> print(struct.pack(">d", fp.contents.value).hex()) 7ff0000000000001 Thank you in advance!

1 0

PEP 563 and 649: The Great Compromise
by Larry Hastings 27 Apr '21

27 Apr '21

The heart of the debate between PEPs 563 and 649 is the question: what should an annotation be? Should it be a string or a Python value? It seems people who are pro-PEP 563 want it to be a string, and people who are pro-PEP 649 want it to be a value. Actually, let me amend that slightly. Most people who are pro-PEP 563 don't actually care that annotations are strings, per se. What they want are specific runtime behaviors, and they get those behaviors when PEP 563 turns their annotations into strings. I have an idea--a rough proposal--on how we can mix together aspects of PEP 563 and PEP 649. I think it satisfies everyone's use cases for both PEPs. The behavior it gets us: * annotations can be stored as strings * annotations stored as strings can be examined as strings * annotations can be examined as values The idea: We add a new type of compile-time flag, akin to a "from __future__" import, but not from the future. Let's not call it "from __present__", for now how about "from __behavior__". In this specific case, we call it "from __behavior__ import str_annotations". It behaves much like Python 3.9 does when you say "from __future__ import annotations", except: it stores the dictionary with stringized values in a new member on the function/class/module called "__str_annotations__". If an object "o" has "__str_annotations__", set, you can access it and see the stringized values. If you access "o.__annotations__", and the object has "o.__str_annotations__" set but "o.__annotations__" is not set, it builds (and caches) a new dict by iterating over o.__str_annotations__, calling eval() on each value in "o.__str_annotations__". It gets the globals() dict the same way that PEP 649 does (including, if you compile a class with str_annotations, it sets __globals__ on the class). It does /not/ unset "o.__str_annotations__" unless someone explicitly sets "o.__annotations__". This is so you can write your code assuming that "o.__str_annotations__" is set, and it doesn't explode if somebody somewhere ever looks at "o.__annotations__". (This could lead to them getting out of sync, if someone modified "o.__annotations__". But I suspect practicality beats purity here.) This means: * People who only want stringized annotations can turn it on, and only ever examine "o.__str_annotations__". They get the benefits of PEP 563: annotations don't have to be valid Python values at runtime, just parseable. They can continue doing the "if TYPE_CHECKING:" import thing. * Library code which wants to examine values can examine "o.__annotations__". We might consider amending library functions that look at annotations to add a keyword-only parameter, "str_annotations=False", and if it's true it uses o.__str_annotations__ instead etc etc etc. Also, yes, of course we can keep the optimization where stringized annotations are stored as a tuple containing an even number of strings. Similarly to PEP 649's automatic binding of an unbound code object, if you set "o.__str_annotations__" to a tuple containing an even number of strings, and you then access "o.__str_annotations__", you get back a dict. TBD: how this interacts with PEP 649. I don't know if it means we only do this, or if it would be a good idea to do both this and 649. I just haven't thought about it. (It would be a runtime error to set both "o.__str_annotations__" and "o.__co_annotations__", though.) Well, whaddya think? Any good? I considered calling this "PEP 1212", which is 563 + 649, //arry/

10 16

str() vs format(): trivia question
by Ethan Furman 26 Apr '21

26 Apr '21

urllib.urlencode currently uses `str()` on its non-bytes objects before encoding the result. This causes a compatibility break when integer module constants are converted to IntEnum, as `str(IntEnum.MEMBER)` no longer returns the integer representation; however, `format()` does still return the integer representation. The fix is to add a separate branch to check if the argument is an Enum, and use the value if so -- but it got me wondering: in general, are there differences between calling str() vs calling format() on Python objects? -- ~Ethan~

6 11

Announcing the CPython Docs Workgroup
by Mariatta 26 Apr '21

26 Apr '21

Thanks to everyone for your patience as we worked to get the Docs Workgroup off the ground. As you may have remembered, Carol Willing and Ned Batchelder brought forward the topic of CPython Documentation <https://pyfound.blogspot.com/2020/04/cpython-documentation-next-5-years.html> at last year's Python Language Summit. The topic has been further discussed at last year's virtual Python core dev's sprint, and Carol has worked on the Docs WG charter <https://docs.google.com/document/d/175bjJA7hKRaqvLjiRXSvUxKgGWGJ5RMuW-jYKUs…>. Please see the group charter doc linked above to understand the purpose, goals, group values, and guidelines of the Docs WG. We've been working to move the Docs Workgroup forward. We'll share more details at next month's Python Language Summit. For now, here's some high level information: - The initial workgroup members are: Carol, Mariatta, Ned Batchelder, and Julien Palard. Additionally, the Python Steering Council are permanent members of the WG. - Work on this group will be organized on the python/docs-community GitHub repository. The charter information and meeting notes will be posted there. - The Documentation category on Discourse <https://discuss.python.org/c/core-dev/documentation/26> will be the main communication channel between the WG and the wider Python community - We will scheduling monthly meetings for the team to meet - We’re still working on the logistics on taking in new members to the group. We’ll keep you updated! Mariatta & Carol

1 0

List of stdlib types that have been converted to heap types
by Erlend Aasland 26 Apr '21

26 Apr '21

Ref. https://bugs.python.org/issue43908, I've compiled a list of types that have been converted to heap types. I've posted it on Discourse for readability: https://discuss.python.org/t/list-of-built-in-types-converted-to-heap-types… Note: for now, I didn't bother to trace PyType_Spec variable names to complete type names. Hope it's not too much of a problem. Erlend Here's the plain text version: 0b858cdd5d 2021-01-04 Modules/cjkcodecs/multibytecodec.c - multibytecodec_spec - encoder_spec - decoder_spec - reader_spec - writer_spec c8a87addb1 2021-01-04 Modules/pyexpat.c - _xml_parse_type_spec 75bf107c62 2021-01-02 Modules/arraymodule.c - array_spec - arrayiter_spec dd39123970 2020-12-29 Modules/_functoolsmodule.c - partial_type_spec - keyobject_type_spec - lru_list_elem_type_spec - lru_cache_type_spec 6104013838 2020-12-18 Modules/_threadmodule.c - lock_type_spec - rlock_type_spec - local_dummy_type_spec - local_type_spec a6109ef68d 2020-11-20 Modules/_sre.c - pattern_spec - match_spec - scanner_spec 5c36da78d7 2020-11-20 Modules/_ssl.c - PySSLSocket_spec - PySSLContext_spec - PySSLMemoryBIO_spec - PySSLSession_spec 01c6aa43dc 2020-11-07 Modules/_queuemodule.c - simplequeue_spec 74b4eda98b 2020-11-03 Modules/mmapmodule.c - mmap_object_spec c8c4200b65 2020-10-26 Modules/unicodedata.c - ucd_type_spec 256e54acdb 2020-10-01 Modules/_sqlite - connection_spec - cursor_spec 9031bd4fa4 2020-10-01 Modules/_sqlite - row_spec - stmt_spec cb6db8b6ae 2020-09-29 Modules/_sqlite - type_spec (PrepareProtocolType) a937ab45d6 2020-09-27 Modules/_sqlite - node_spec - cache_spec 83de110dce 2020-09-23 Modules/_lsprof.c - _lsprof_profiler_type_spec 52a2df135c 2020-09-08 Modules/sha256module.c - sha224_type_spec - sha256_type_spec 2aabc3200b 2020-09-07 Modules/overlapped.c - overlapped_type_spec 63f102fe07 2020-09-06 Modules/md5module.c Modules/sha1module.c Modules/sha512module.c - md5_type_spec - sha1_type_spec - sha512_sha384_type_spec - sha512_sha512_type_spec 31967fd8d0 2020-08-27 Modules/_operator.c - itemgetter_type_spec - attrgetter_type_spec - methodcaller_type_spec e087f7cd43 2020-08-13 Modules/_winapi.c - winapi_overlapped_type_spec 1937edd376 2020-06-23 Modules/_lzmamodule.c - lzma_compressor_type_spec - lzma_decompressor_type_spec ec68918795 2020-06-20 Modules/_bz2module.c - bz2_compressor_type_spec - bz2_decompressor_type_spec c4862e333a 2020-06-17 Modules/_gdbmmodule.c - gdbmtype_spec bf69a8f99f 2020-06-16 Modules/_dbmmodule.c - dbmtype_spec d5b3f6b7f9 2020-05-16 Modules/_hashopenssl.c - EVPXOFtype_spec 53e4c91725 2020-03-30 Modules/_abc.c - _abc_data_type_spec 33f15a16d4 2020-03-27 Modules/_json.c - PyScannerType_spec - PyEncoderType_spec b3966639d2 2019-11-05 Modules/posixmodule.c - DirEntryType_spec - ScandirIteratorType_spec df69e75edc 2019-09-25 Modules/_hashopenssl.c - EVPtype_spec f919054e53 2019-09-14 Modules/selectmodule.c - devpoll_Type_spec - kqueue_event_Type_spec - poll_Type_spec - pyEpoll_Type_spec - kqueue_queue_Type_spec 04f0bbfbed 2019-09-13 Modules/_randommodule.c - Random_Type_spec ac46eb4ad6 2019-09-11 Python/Python-ast.c - AST_type_spec - AST_type_spec a1ffad0719 2019-09-10 Modules/zlibmodule.c - Comptype_spec - Decomptype_spec 4f384af067 2019-09-10 Modules/_struct.c - unpackiter_type_spec - PyStructType_spec 11dc6c311d 2012-10-14 Modules/_tkinter.c - PyTclObject_Type_spec - Tktt_Type_spec - Tkapp_Type_spec 3b36fb1f53 2012-06-22 Modules/_ssl.c - sslerror_type_spec bc07cb883e 2012-06-14 Modules/_curses_panel.c - PyCursesPanel_Type_spec

2 1

[Release management team communication] 3.10 feature freeze is 1 week away
by Pablo Galindo Salgado 26 Apr '21

26 Apr '21

Hi everyone, This is the last friendly reminder from the release management team that Python 3.10 feature freeze is a week away (Monday, 3 May 2021). Some important things to have in mind: * We have some release blockers that must be resolved before the release is done: https://bugs.python.org/issue43908 https://bugs.python.org/issue43933 https://bugs.python.org/issue43916 If you want to help making sure that the release is on time, help us resolving those blockers. You can also search the issues in https://bugs.python.org/issue?@template=search&status=1 with the "release-blocker" priority. * Please, don't wait until the last moment (Sunday) to merge PRs into master. This is because if many changes accumulate into a time window and problems are detected, is much more difficult to locate the problematic changes among the rest. If there are any problems with your commits, you also would have the risk of having your changes reverted to unblock the release without any time window to merge them again before feature freeze. Please, also be aware of the following: * No new features or improvements should be merged after the feature freeze. From the devguide (https://devguide.python.org/devcycle/#beta) : > After a first beta release is published, no new features are accepted. Only bug fixes and improvements to documentation and tests can now be committed. > This is when core developers should concentrate on the task of fixing regressions and other new issues filed by users who have downloaded the alpha and beta releases. > Being in beta can be viewed much like being in RC but without the extra overhead of needing commit reviews. * If your change involves some platform-specific behaviour or has a non-trivial amount of C code, make sure you run the buildbots in your Pull Request by using the "test-with-buildbots" label ( https://discuss.python.org/t/now-you-can-test-a-pr-with-the-buildbots-befor… ). Alternatively you could check the buildbots post-merge in the buildbot server: https://buildbot.python.org/all/#/builders?tags=%2B3.x&tags=%2Bstable This is very important because if problems are detected at the time of the release, the release management team may have to revert the changes and therefore those will not be included in Python 3.10. * The master branch will be renamed to main (check our previous communication about how this will be done and how you should proceed https://mail.python.org/archives/list/python-dev@python.org/thread/QWW7KGBW… ) If you have any questions or concerns, please contact me as soon as possible. Regards from sunny London, Pablo Galindo Salgado

1 0