Mailman 3 February 2021 - Python-ideas

Add extend_const action to argparse
by python＠roganartu.com 08 Feb '21

08 Feb '21

Forgive me if this idea has been discussed before, I searched the mailing lists, the CPython repo, and the issue tracker and was unable to find anything. I have found myself a few times in a position where I have a repeated argument that uses the `append` action, along with some convenience arguments that append a specific const to that same dest (eg: `--filter-x` being made equivalent to `--filter x` via `append_const`). This is particularly useful in cli apps that expose some kind of powerful-but-verbose filtering capability, while also providing shorter aliases for common invocations. I'm sure there are other use cases, but this is the one I'm most familiar with. The natural extension to this filtering idea are convenience args that set two const values (eg: `--filter x --filter y` being equivalent to `--filter-x-y`), but there is no `extend_const` action to enable this. While this is possible (and rather straight forward) to add via a custom action, I feel like this should be a built-in action instead. `append` has `append_const`, it seems intuitive and reasonable to expect `extend` to have `extend_const` too (my anecdotal experience the first time I came across this need was that I simply tried using `extend_const` without checking the docs, assuming it already existed). Please see this gist for a working example that may help explain the idea and intended use case more clearly: https://gist.github.com/roganartu/7c2ec129d868ecda95acfbd655ef0ab2

3 2

Type hinting **kwargs
by Paul Bryan 08 Feb '21

08 Feb '21

I'm encountering a situation where it would be far cleaner to use **kwargs (multiple functions taking the same set of parameters); I'd like to have type hints, and avoid repeating multiple function definitions, explicitly enumerating all parameters for each function definition. Therefore, I'd like to have some meaningful type hint for **kwargs. First, I thought TypedDict would be a great fit, until I found this passage in PEP 589: > These features were left out from this PEP, but they are potential > extensions to be added in the future: > ... > TypedDict can't be used for specifying the type of a **kwargs > argument. This would allow restricting the allowed keyword arguments > and their types. According to PEP 484, using a TypedDict type as the > type of **kwargs means that the TypedDict is valid as the value of > arbitrary keyword arguments, but it doesn't restrict which keyword > arguments should be allowed. The syntax **kwargs: Expand[T] has been > proposed for this [11]. > ... > [11] https://github.com/python/mypy/issues/4441 In the mypy issue discussion, there seems to be two issues: 1. Ambiguity of whether the type hint applies to the **kwargs dict vs. each value in the dict. I think this could be a red herring; kwargs presents as a dict and I think TypedDict can unambiguously represent the structure of that dict. I don't see how it should be different than any other dict parameter being defined as a function parameter. 2. A question of whether one needs to assume that for a TypedDict, the type validator should assume total=False. I think what keys are required should be explicitly determined by the TypedDict (required keys inferred from total). I suspect the issue could be that mypy may not be able to determine this during static type analysis? In summary, **kwargs is a dict; TypedDict can specify its structure; static type checker limitations should not necessarily restrict its use in this way. Therefore, I propose a short PEP be drafted to codify the validity of using TypedDict to specify the type of a **kwargs argument. Paul

2 1

PEP for experimental/provisional features
by sascha.schlemmer＠me.com 08 Feb '21

08 Feb '21

Reading the latest discussion about PEP 622/634-636 on the Python Dev mailing list, I've noticed there appears to be confusion about what `__future__` imports are supposed to be used for. Multiple people in that thread (and probably elsewhere in one of the many many threads about these PEPs) have suggested including pattern matching as a "provisional" or "experimental" feature that is enabled by an import from `__future__`. I don't want to discuss whether `__future__` is the correct place for this however (and I also don't want to discuss PEP 622/634-636 specifically). Instead I'd like to propose a `__provisional__` namespace (or whatever color you favor for your bike shed) for new language features/ standard library modules that might undergo breaking changes over a couple of subsequent python releases. This is similar to how many applications start in a version V0.1 and undergo changes until they mature and reach V1.0. From the user perspective this would allow more "adventurous" users to play with the latest features (with the clear warning attached that they might have to adapt their own code over subsequent python releases) while anyone else can just ignore the new feature for now. From the perspective of the language designers the advantage I see is that they can get a first version of a feature out to real users to collect feedback on whether the feature is used as they imagined it being used and change/iterate the design if need be. It is simply unrealistic to get something like `asyncio` or PEP 622/634-636 perfect and feature complete on the first try or otherwise wait indefinitely until something is ready. Some thoughts what `__provisional__` SHOULD and SHOULD NOT be used for: - `__provisional__` SHOULD be useable at the users own discretion (e.g. no use of provisional features in the standard library etc. ) - `__provisional__` SHOULD be a place for features with a vast design space that need to "ripen" as use cases are explored (`asyncio` would have been the poster child candidate for `__provisional__`) - features SHOULD only be placed in `__provisional__` if there is a clear commitment to the feature (the general feature is accepted by the SC before being included in `__provisional__`). `__provisional__` SHOULD NOT be a place where contentious PEPs are thrown into willy-nilly and postpone the decision of whether to accept a PEP or not. Otherwise users would not only have to deal with changes to a feature (they obviously showed interest in using) but also with complete removal/deprecation. - `__provisional__` SHOULD only be used for features that are additive to the language and don't change existing code (which is what imho. what `__future__` is for). - `__provisional__` SHOULD have a clearly defined process (timelines etc.) Alternative ideas that have been mentioned (in threads about PEP 622/634-636 and elsewhere) and why I personally don't think they are sufficient: - "just throw together a library that parses strings and uses `eval`, put it on pypi to see how that goes": For features that require new syntax this just doesn't work imho. Using `pep622_lib.match("put your match statement here")` is a very poor substitute for trying this out due to the influence design choices such as keywords, indentation etc. have on usability. Examples for this are syntax highlighting, autocomplete/autoindent/autoformat and linting which you just don't get this way. - various degrees of: "solve the issue of contention over new languages features once and for all by introducing macros/preprocessors to the language": Against this I have just my personal plea "please don't turn python into lisp/c++".

2 2

Make UTF-8 mode more accessible for Windows users.
by Inada Naoki 07 Feb '21

07 Feb '21

The fact that Python does not use UTF-8 as the default encoding when opening text files is an obstacle for many Windows users, especially beginners in programming. If you search for UnicodeDecodeError, you will see that many Windows users have encountered the problem. This list is only part of many search results. * https://qiita.com/Yuu94/items/9ffdfcb2c26d6b33792e * https://www.mikan-partners.com/archives/3212 * https://teratail.com/questions/268749 * https://github.com/neovim/pynvim/issues/443 * https://www.coder.work/article/1284080 * https://teratail.com/questions/271375 * https://qiita.com/shiroutosan/items/51358b24b0c3defc0f58 * https://github.com/jsvine/pdfplumber/issues/304 * https://ja.stackoverflow.com/questions/69281/73612 * https://trend-tracer.com/pip-error/ Looking at the errors, the following are the most common cases. * UnicodeDecodeError is raised when trying to open a text file written in UTF-8, such as JSON. * UnicodeEncodeError is raised when trying to save text data retrieved from the web, etc. * User run `pip install` and `setup.py` reads README.md or LICENSE file written in UTF-8 without `encoding="UTF-8"` Users can use UTF-8 mode to solve these problems. I wrote a section for UTF-8 mode in the "3. Using Python on Windows" document. https://docs.python.org/3/using/windows.html#utf-8-mode However, UTF-8 mode is still not very well known. How can we make UTF-8 mode more user-friendly? Right now, UTF-8 mode can be enabled using the `-Xutf8` option or the `PYTHONUTF8` environment variable. This is a hurdle for beginners. In particular, Jupyter users may not use the command line at all. Is it possible to enable UTF-8 mode in a configuration file like `pyvenv.cfg`? * User can enable UTF-8 mode per-install, and per-venv. * But difficult to write the setting file when Python is installed for system (not for user), or Windows Store Python * User can still enable UTF-8 mode in venv. But many beginners don't need venv. Is it possible to make it easier to configure? * Put a checkbox in the installer? * Provide a small tool to allow configuration after installation? * python3 -m utf8mode enable|disable? * Accessible only for CLI user * Add "Enable UTF-8 mode" and "Disable UTF-8 mode" to Start menu? Any ideas are welcome. -- Inada Naoki <songofacandy(a)gmail.com>

6 31

Re: [Python-Dev] Re: What's up with assignment expression and tuples?
by Paul Sokolovsky 06 Feb '21

06 Feb '21

Hello, On Sat, 6 Feb 2021 10:46:54 +0200 Serhiy Storchaka <storchaka(a)gmail.com> wrote: > 05.02.21 09:51, Paul Sokolovsky пише: > > a0 = 0 > > b0 = 10 > > while ((a1, b1) := phi([a0, a2], [b0, b2]))[0] < 5: > > a2 = a1 + 1 > > b2 = b1 + 1 > > Such code quickly becomes unreadable. Especially if in real code > function has additional arguments and names are longer that 2-3 > characters. But at least it fully corresponds to the original statements and control flow of the program, that's the point! > The following code is not much larger but more clear and extensible: > > a0 = 0 > b0 = 10 > while True: > a1, b1 := phi([a0, a2], [b0, b2])) > if b1 >= 5: > break > a2 = a1 + 1 > b2 = b1 + 1 As I mentioned in other replies, that's what I've been doing. But that does *NOT* correspond to the original program, or SSA conversion of it. Instead, it's SSA + random munging. And when debugging SSA, you may argue what's worse: to look at 3-stories phi's, or to look at the code structure which doesn't correspond to the input code. But that's not the point. I now in https://bugs.python.org/issue43143 gave more down to earth example: For as long as you agree that following is acceptable: func(a := val) , then I'd say it really doesn't make much sense to argue that following should not be acceptable: min((b, a) := (a, b)) -- Best regards, Paul mailto:pmiscml@gmail.com

1 0

Add a couple of options to open()'s mode parameter to deal with common text encodings
by Ben Rudiak-Gould 05 Feb '21

05 Feb '21

There's a long ongoing thread with the subject "Make UTF-8 mode more accessible for Windows users." There are two obvious problems with UTF-8 mode. First, it applies to entire installations, or at least entire running scripts, including all imported libraries no matter who wrote them, etc., making it a blunt instrument. Second, the problem on Windows isn't that Python happens to use the wrong default encoding, it's that multiple encodings coexist, and you really do have to think each time you en/decode something about which encoding you ought to use. UTF-8 mode doesn't solve that, it just changes the default. It seems as though most of those commenting in the other thread don't actually use Python on Windows. I do, and I can say it's a royal pain to have to write open(path, encoding='utf-8') all the time. If you could write open(path, 'r', 'utf-8'), that would be slightly better, but the third parameter is buffering, not encoding, and open(path, 'r', -1, 'utf-8') is not very readable. UTF-8 mode is somehow worse, because you now have to decide between writing open(path), and having your script be incompatible with non-UTF-8 Windows installations, or writing open(path, encoding='utf-8'), making your script more compatible but making UTF-8 mode pointless. There's a constant temptation to sacrifice portability for convenience - a temptation that Unix users are familiar with, since they omit encoding='utf-8' all the time. My proposal is to add a couple of single-character options to open()'s mode parameter. 'b' and 't' already exist, and the encoding parameter essentially selects subcategories of 't', but it's annoyingly verbose and so people often omit it. If '8' was equivalent to specifying encoding='UTF-8', and 'L' was equivalent to specifying encoding=(the real locale encoding, ignoring UTF-8 mode), that would go a long way toward making open more convenient in the common cases on Windows, and I bet it would encourage at least some of those developing on Unixy platforms to write more portable code also. For other encodings, you can still use 't' (or '') and the encoding parameter. Note that I am not suggesting that 'L' be equivalent to PEP 597's encoding='locale', because that's specified to behave the same as encoding=None, except that it suppresses the warning. I think that's a terrible idea, because it means that open's behavior still depends on the global UTF-8 mode even if you specify the encoding explicitly. This is really a criticism of PEP 597 and not a part of this proposal as such. I think UTF-8 mode was a bad idea (just like a global "binary mode" that interpreted every mode='r' as mode='rb' would have been), and it should be ignored wherever possible. In particular, encoding='locale' should ignore UTF-8 mode. Then 'L' could and should mean encoding='locale'. Obviously the names '8' and 'L' are debatable. 'L' could be argued to be unnecessary if there's a simple way to achieve the same thing with the encoding parameter (which currently there isn't).

6 7