Mailman 3 January 2021 - Python-ideas

Add extend_const action to argparse
by python＠roganartu.com 08 Feb '21

08 Feb '21

Forgive me if this idea has been discussed before, I searched the mailing lists, the CPython repo, and the issue tracker and was unable to find anything. I have found myself a few times in a position where I have a repeated argument that uses the `append` action, along with some convenience arguments that append a specific const to that same dest (eg: `--filter-x` being made equivalent to `--filter x` via `append_const`). This is particularly useful in cli apps that expose some kind of powerful-but-verbose filtering capability, while also providing shorter aliases for common invocations. I'm sure there are other use cases, but this is the one I'm most familiar with. The natural extension to this filtering idea are convenience args that set two const values (eg: `--filter x --filter y` being equivalent to `--filter-x-y`), but there is no `extend_const` action to enable this. While this is possible (and rather straight forward) to add via a custom action, I feel like this should be a built-in action instead. `append` has `append_const`, it seems intuitive and reasonable to expect `extend` to have `extend_const` too (my anecdotal experience the first time I came across this need was that I simply tried using `extend_const` without checking the docs, assuming it already existed). Please see this gist for a working example that may help explain the idea and intended use case more clearly: https://gist.github.com/roganartu/7c2ec129d868ecda95acfbd655ef0ab2

3 2

Make UTF-8 mode more accessible for Windows users.
by Inada Naoki 06 Feb '21

06 Feb '21

The fact that Python does not use UTF-8 as the default encoding when opening text files is an obstacle for many Windows users, especially beginners in programming. If you search for UnicodeDecodeError, you will see that many Windows users have encountered the problem. This list is only part of many search results. * https://qiita.com/Yuu94/items/9ffdfcb2c26d6b33792e * https://www.mikan-partners.com/archives/3212 * https://teratail.com/questions/268749 * https://github.com/neovim/pynvim/issues/443 * https://www.coder.work/article/1284080 * https://teratail.com/questions/271375 * https://qiita.com/shiroutosan/items/51358b24b0c3defc0f58 * https://github.com/jsvine/pdfplumber/issues/304 * https://ja.stackoverflow.com/questions/69281/73612 * https://trend-tracer.com/pip-error/ Looking at the errors, the following are the most common cases. * UnicodeDecodeError is raised when trying to open a text file written in UTF-8, such as JSON. * UnicodeEncodeError is raised when trying to save text data retrieved from the web, etc. * User run `pip install` and `setup.py` reads README.md or LICENSE file written in UTF-8 without `encoding="UTF-8"` Users can use UTF-8 mode to solve these problems. I wrote a section for UTF-8 mode in the "3. Using Python on Windows" document. https://docs.python.org/3/using/windows.html#utf-8-mode However, UTF-8 mode is still not very well known. How can we make UTF-8 mode more user-friendly? Right now, UTF-8 mode can be enabled using the `-Xutf8` option or the `PYTHONUTF8` environment variable. This is a hurdle for beginners. In particular, Jupyter users may not use the command line at all. Is it possible to enable UTF-8 mode in a configuration file like `pyvenv.cfg`? * User can enable UTF-8 mode per-install, and per-venv. * But difficult to write the setting file when Python is installed for system (not for user), or Windows Store Python * User can still enable UTF-8 mode in venv. But many beginners don't need venv. Is it possible to make it easier to configure? * Put a checkbox in the installer? * Provide a small tool to allow configuration after installation? * python3 -m utf8mode enable|disable? * Accessible only for CLI user * Add "Enable UTF-8 mode" and "Disable UTF-8 mode" to Start menu? Any ideas are welcome. -- Inada Naoki <songofacandy(a)gmail.com>

6 31

List .index() Method
by Francis O'Hara Aidoo 31 Jan '21

31 Jan '21

Hello, I'm Francis, a beginner programmer from Ghana, West Africa. I was wondering why the list .index() method doesn't return negative indices as well as well as positive indices. Although ambiguity will make it difficult to implement, in certain cases, it would be useful if it could return the negative index. For instance, if one creates an if statement that checks whether an element is the last item in a list as follows: listy = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] if listy.index(10) == -1: print("Monty Python") I understand that the same effect can be achieved with the index notation - as in if listy[-1] == 10: print("Monty Python") - but the way that came naturally to me was to use the .index method rather than index notation, and it took a very long time for me to figure out why my code was not working(mostly because I'm a beginner). So do what you will, I guess.:)

4 3

Questions about using PEP 637 syntax in PEP 646
by gohanpra＠gmail.com 30 Jan '21

30 Jan '21

Hello, This is a great PEP. It turns out to be applicable in a variety of scenarios. Case in point: Matthew Rahtz and I are working on PEP 646: Variadic Generics (https://www.python.org/dev/peps/pep-0646/; discussion on typing-sig). It is a type system feature that allows specifying an arbitrary tuple of type variables instead of a single type variable. We plan to use your proposed syntax to represent unpacking a tuple type. This would be analogous to `*` for unpacking a tuple value: + `Tensor[int, *Ts, str]` and `Tensor[*Ts1, *Ts2]` + such variadic classes would be declared as `class Tensor(Generic[T, *Ts, T2]):` Questions: 1. Does PEP 637 support unpacking multiple `*` arguments? - e.g., Tensor[int, *Ts, str, *Ts2] 2. Does PEP 637 allow a positional argument after a `*`? - e.g., Generic[T, *Ts, T2] PEP 637 says "Sequence unpacking is allowed inside subscripts", so it looks like these should be allowed (just as in function calls). But I wanted to confirm it explicitly since this is our core use case and there was no example with multiple sequences being unpacked. 3. We also wanted to ask - how's your implementation going? We'll be starting implementation in typing.py soon. Since there's some overlap we wanted to make sure we're not duplicating your work, and that there won't be any merge conflicts later. Do you have a fork we might be able to get early access to? We're also targeting the 3.10 release for our implementation. I'd be happy to provide additional details if needed. Best, Pradeep Kumar Srinivasan Matthew Rahtz

4 4

Parameter documentation
by abed...＠gmail.com 29 Jan '21

29 Jan '21

Currently, python allows variable documentation via PEP 526 <https://www.python.org/dev/peps/pep-0526/>. For most functions with short parameter lists that can fit in a reasonable column limit, I prefer the traditional declaration style with Google-style doc strings: *def connect_to_next_port(self, minimum: int) => int: * """Connects to the next available port. Args: minimum: A port value greater or equal to 1024. Returns: The new minimum port. Raises: ConnectionError: If no available port is found. """ ...code... However, when a signature gets too long, I prefer to list the parameters vertically: *def request(* * method: Method, url: Str,* * params: Dict = None, data: Dict = None, json: Str = None, headers: Dict = None, cookies: Dict = None, files: Dict = None, ...) => Response: """* *Constructs and sends a Request* * Args: ... """ * In which case, it would be nice to in-line some documentation instead of repeating the whole parameter list in the doc string. Something like: *def request(* * method: Method* * #method for the new Request: ``GET``,``POST``, etc.* * , url: Str* * #URL for the request ,* * params: Dict = None* * ...**) => Response:* * """**Constructs and sends a Request* *"""*

2 6

Adding `open_text()` builtin function. (relating to PEP 597)
by Inada Naoki 26 Jan '21

26 Jan '21

Hi, all. I am rewriting PEP 597 to introduce a new EncodingWarning, which subclass of DeprecationWarning and used to warn about future default encoding change. But I don't think we can change the default encoding of `io.TextIOWrapper` and built-in `open()` anytime soon. It is disruptive change. It may take 10 or more years. To ease the pain caused by "default encoding is not UTF-8 (almost) only on Windows" (*), I came up with another idea. This idea is not mutually exclusive with PEP 597, but I want to include it in the PEP because both ideas use EncodingWarning. (*) Imagine that a new Python user writes a text file with notepad.exe (default encoding is UTF-8 without BOM already) or VS Code, and try to read it in Jupyter Notebook. They will see UnicodeDecodeError. They might not know about what encoding yet. ## 1. Add `io.open_text()`, builtin `open_text()`, and `pathlib.Path.open_text()`. All functions are same to `io.open()` or `Path.open()`, except: * Default encoding is "utf-8". * "b" is not allowed in the mode option. These functions have two benefits: * `open_text(filename)` is shorter than `open(filename, encoding="utf-8")`. Its easy to type especially with autocompletion. * Type annotation for returned value is simple than `open`. It is always TextIOWrapper. ## 2. Change the default encoding of `pathlib.Path.read_text()`. For convenience and consistency with `Path.open_text()`, change the default encoding of `Path.read_text()` to "utf-8" with regular deprecation period. * Python 3.10: `Path.read_text()` emits EncodingWarning when the encoding option is omitted. * Python 3.13: `Path.read_text()` change the default encoding to "utf-8". If PEP 597 is accepted, users can pass `encoding="locale"` instead of `encoding=locale.getpreferredencoding(False)` when they need to use locale encoding. We might change more places where the default encoding is used. But it should be done slowly and carefully. --- How do you think about this idea? Is this worth enough to add a new built-in function? Regards, -- Inada Naoki <songofacandy(a)gmail.com>

13 54

Provide UTF-8 version of Python for Windows.
by Inada Naoki 26 Jan '21

26 Jan '21

Sorry for posting multiple threads so quickly. Microsoft provides UTF-8 code page for process. It can be enabled by manifest file. https://docs.microsoft.com/ja-jp/windows/uwp/design/globalizing/use-utf8-co… How about providing Python binaris both of "UTF-8 version" and "ANSI version"? This idea can provide a more smooth transition of the default encoding. 1. Provide UTF-8 version since Python 3.10 2. (Some years later) Recommend UTF-8 version 3. (Some years later) Provide only UTF-8 version 4. (Some years later, maybe) Change the default encoding The upsides of this idea are: * We don't need to emit a warning for `open(filename)`. * We can see the download stats. Especially, the last point is a huge advantage compared to current UTF-8 mode (e.g. PYTHONUTF8=1). We can know how many users need legacy behavior in new Python versions. That is a very important information for us. Of course, there are some downsides: * Windows team needs to maintain more versions. * More divisions for "Python on Windows" environment. Regards, -- Inada Naoki <songofacandy(a)gmail.com>

6 10

dataclass: __init__ kwargs and Optional[type]
by Paul Bryan 25 Jan '21

25 Jan '21

I've created a helper class in my own library that enhances the existing dataclass: a) __init__ accepts keyword-only arguments, b) Optional[...] attribute without a specified default value would default to None in __init__. I think this could be useful in stdlib. I'm thinking a dataclass decorator parameter like "init_kwonly" (default=False to provide backward compatibility) that if True would implement this behavior. Thoughts?

2 3

Changing the default text encoding of pathlib
by Inada Naoki 25 Jan '21

25 Jan '21

My previous thread is hijacked about "auto guessing" idea, so I split this thread for pathlib. Path.open() was added in Python 3.4. Path.read_text() and Path.write_text() was added in Python 3.5. Their history is shorter than built-in open(). Changing its default encoding should be easier than built-in open and TextIOWrapper. New default encodings are: * read_text() default encoding is "utf-8-sig" * write_text() default encoding is "utf-8" * open() default encoding is "utf-8-sig" when mode is "r" or None, "utf-8" otherwise. Of course, we need a regular deprecation period. When encoding is omitted, they emit DeprecationWarning (or EncodingWarning which is a subclass of DeprecationWarning) in three versions (Python 3.10~3.12). How do you think this idea? Should we "change all at once" rather than "step-by-step"? Regards, -- Inada Naoki <songofacandy(a)gmail.com>

3 2

Re: [Python-Dev] reversed enumerate
by Steven D'Aprano 22 Jan '21

22 Jan '21

Hi Ilya, I'm not sure that this mailing list (Python-Dev) is the right place for this discussion, I think that Python-Ideas (CCed) is the correct place. For the benefit of Python-Ideas, I have left your entire post below, to establish context. [Ilya] > I needed reversed(enumerate(x: list)) in my code, and have discovered > that it wound't work. This is disappointing because operation is well > defined. It isn't really well-defined, since enumerate can operate on infinite iterators, and you cannot reverse an infinite stream. Consider: def values(): while True: yield random.random() a, b = reversed(enumerate(values()) What should the first pair of (a, b) be? However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too. My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense. But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard. On the other hand, I think that there is a fairly lightweight work around. Define a helper function: def countdown(n): while True: yield n n -= 1 then call it like this: # reversed(enumerate(seq)) zip(countdown(len(seq)-1), reversed(seq))) So it isn't terribly hard to work around this. But I agree that it would be nice if enumerate encapsulated this for the caller. One potentially serious question: what should `enumerate.__reversed__` do when given a starting value? reversed(enumerate('abc', 1)) Should that yield...? # treat the start value as a start value (1, 'c'), (0, 'b'), (-1, 'a') # treat the start value as an end value (3, 'c'), (2, 'b'), (1, 'a') Something else? My preference would be to treat the starting value as an ending value. Steven On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote: > Hi, > > I needed reversed(enumerate(x: list)) in my code, and have discovered that > it wound't work. This is disappointing because operation is well defined. > It is also well defined for str type, range, and - in principle, but not > yet in practice - on dictionary iterators - keys(), values(), items() as > dictionaries are ordered now. > It would also be well defined on any user type implementing __iter__, > __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors. > > That's plenty of usecases, therefore I guess it would be quite useful to > avoid hacky / inefficient solutions like described here: > https://code.activestate.com/lists/python-list/706205/. > > If deemed useful, I would be interested in implementing this, maybe > together with __reversed__ on dict keys, values, items. > > Best Regards, > -- > Ilya Kamen > > ----------- > p.s. > > *Sketch* of what I am proposing: > > class reversible_enumerate: > > def __init__(self, iterable): > self.iterable = iterable > self.ctr = 0 > > def __iter__(self): > for e in self.iterable: > yield self.ctr, e > self.ctr += 1 > > def __reversed__(self): > try: > ri = reversed(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > try: > l = len(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > indexes = range(l-1, -1, -1) > for i, e in zip(indexes, ri): > yield i, e > > for i, c in reversed(reversible_enumerate("Hello World")): > print(i, c) > > for i, c in reversed(reversible_enumerate([11, 22, 33])): > > print(i, c) > _______________________________________________ > Python-Dev mailing list -- python-dev(a)python.org > To unsubscribe send an email to python-dev-leave(a)python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUD… > Code of Conduct: http://python.org/psf/codeofconduct/

7 14