Mailman 3 January 2021 - Python-ideas

zip(x, y, z, strict=True)
by Ram Rachum Dec. 1, 2022

Dec. 1, 2022

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

Bringing the print statement back
by Guido van Rossum March 2, 2022

March 2, 2022

In Python 3.10 we will no longer be burdened by the old parser (though 3rd party tooling needs to catch up). One thing that the PEG parser makes possible in about 20 lines of code is something not entirely different from the old print statement. I have a prototype: Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020, 16:31:17) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. Cannot read termcap database; using … [View More]dumb terminal settings. >>> print 2+2 4 >>> print "hello world" hello world >>> print "hello", input("Name:") Name:Guido hello Guido >>> print 1, 2, 3, sep=", " 1, 2, 3 >>> But wait, there's more! The same syntax will make it possible to call *any* function: >>> len "abc" 3 >>> Or any method: >>> import sys >>> sys.getrefcount "abc" 24 >>> Really, *any* method: >>> class C: ... def foo(self, arg): print arg ... >>> C().foo 2+2 4 >>> There are downsides too, though. For example, you can't call a method without arguments: >>> print <built-in function print> >>> Worse, the first argument cannot start with a parenthesis or bracket: >>> print (1, 2, 3) 1 2 3 >>> C().foo (1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: C.foo() takes 2 positional arguments but 4 were given >>> print (2+2), 42 4 (None, 42) >>> C().foo [0] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'method' object is not subscriptable >>> No, it's not April 1st. I am seriously proposing this (but I'll withdraw it if the response is a resounding "boo, hiss"). After all, we currently have a bunch of complexity in the parser just to give a helpful error message to people used to Python 2's print statement: >>> print 1, 2, 3 File "<stdin>", line 1 print 1, 2, 3 ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1, 2, 3)? >>> And IIRC there have been a number of aborted attempts at syntactic hacks to allow people to call functions (like print) without parentheses, although (I think) none of them made it into a PEP. The PEG parser makes this much simpler, because it can simply backtrack -- by placing the grammar rule for this syntax (tentatively called "call statement") last in the list of alternatives for "small statement" we ensure that everything that's a valid expression statement (including print() calls) is still an expression statement with exactly the same meaning, while still allowing parameter-less function calls, without lexical hacks. (There is no code in my prototype that checks for a space after 'print' -- it just checks that there's a name, number or string following a name, which is never legal syntax.) One possible extension I didn't pursue (yet -- dare me!) is to allow parameter-less calls inside other expressions. For example, my prototype does not support things like this: >>> a = (len "abc") File "<stdin>", line 1 a = (len "abc") ^ SyntaxError: invalid syntax >>> I think that strikes a reasonable balance between usability and reduced detection of common errors. I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo x' and we should only allow dotted names (sufficient to access functions in imported modules and method calls on variables). Or maybe we should only allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'. Or maybe we should really only bring back print statements. I believe there are some other languages that support a similar grammar (Ruby? R? Raku?) but I haven't investigated. Thoughts? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…> [View Less]

42 66

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 18, 2022

Feb. 18, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, … [View More]

17 32

Allow syntax "func(arg=x if condition)"
by Peter O'Connor Jan. 19, 2022

Jan. 19, 2022

I often find that python lacks a nice way to say "only pass an argument under this condition". (See previous python-list email in "Idea: Deferred Default Arguments?") Example 1: Defining a list with conditional elements include_bd = True current_way = ['a'] + (['b'] if include_bd else [])+['c']+(['d'] if include_bd else []) new_way = ['a', 'b' if include_bd, 'c', 'd' if include_bd] also_new_way = list('a', 'b' if include_bd, 'c', 'd' if include_bd) Example 2: Deferring to … [View More]

6 9

Access to source code when using exec()
by Alex Hall April 3, 2021

April 3, 2021

When using `exec(string)`, the standard Python REPL, or `python -c` in a terminal, the executed source code is not stored anywhere and cannot be retrieved (except in the `-c` case by looking at `sys.argv`). This means that: - The source line doesn't show up in tracebacks, e.g: >>> exec("1/0") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> ZeroDivisionError: division by zero - … [View More]

2 1

Make fnmatch.filter accept a tuple of patterns
by Andre Delfino March 26, 2021

March 26, 2021

Frequently, while globbing, one needs to work with multiple extensions. I’d like to propose for fnmatch.filter to handle a tuple of patterns (while preserving the single str argument functionality, alas str.endswith), as a first step for glob.i?glob to accept multiple patterns as well. Here is the implementation I came up with: https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-mu… If this is deemed reasonable, I’ll write tests and documentation updates. Any opinion?

9 15

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman March 12, 2021

March 12, 2021

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/… [View More]clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman [View Less]

12 35

PEP 9999 (provisional): support for indexing with keyword arguments
by Stefano Borini March 10, 2021

March 10, 2021

PEP for support for indexing with keyword arguments is now submitted as PR. https://github.com/python/peps/pull/1612 Thanks to everybody involved in the development of the PEP and the interesting discussion. All your contributions have been received and often added to the document. If the PEP is approved, I would like to attempt an implementation, but I am not particularly skilled in the python internals. If a core developer is willing to teach me the ropes (especially in the new parser, I … [View More]

20 100

A shortcut to load a JSON file into a dict : json.loadf
by The Nomadic Coder March 10, 2021

March 10, 2021

Hi All, This is the first time I'm posting to this mailing group, so forgive me if I'm making any mistakes. So one of the most common ways to load json, is via a file. This is used extensively in data science and the lines. We often write something like :- with open(filename.json, "r") as f: my_dict = json.load(f) or my_dict = json.load(open("filename.json", "r")) Since this is sooooo common, why doesn't python have something like :- json.loadf("filename.json") Is there an obvious … [View More]

22 67

Conditional with statements
by Jonathan Crall Feb. 16, 2021

Feb. 16, 2021

I'm not sure if this has been asked / suggested before. I'm wondering if there is any interest in conditional or loop-based `with` statements. I think it could be done without a syntax change. ### Napkin proposal Context managers could define `__if__` or `__for__`, and if those dunder methods were defined, then the `with` statement would either behave like a conditional or a loop. If `__if__` was defined then ``` with Ctx(): print('hi') ``` would only print `hi` if `__if__` returned … [View More]True. This doesn't require a syntax change. The `__for__` variant would likely need a minor syntax change. ``` with item in Ctx(): print(item) ``` The `__for__` method is a generator that generates arguments of a loop. The item will be printed as many times as there are items generated by `__for__`. ### Use Cases This would simplify usage of my Timerit and ubelt library. The timerit module defines ``timerit.Timerit``, which is an object that is iterable. It has an ``__iter__`` method that generates ``timerit.TimerTimer`` objects, which are context managers. >>> import math >>> from timerit import Timerit >>> for timer in Timerit(num=200, verbose=2): >>> with timer: >>> math.factorial(10000) The timer context manager measures how much time the body of it takes by "tic"-ing ``__enter__`` and "toc"-ing on ``__exit__``. The underlying object has access to the context manager, so it is able to read its measurement. These measurements are stored and then we compute some statistics on them. Notably the minimum, mean, and standard-deviation of grouped (batched) running times. Unfortunately the syntax is one line and one indent bulker than I would prefer. However, a more consice version of the synax is available. >>> import math >>> from timerit import Timerit >>> for _ in Timerit(num=200, verbose=2): >>> math.factorial(10000) In this case the measurement is made in the `__iter__` method ``Timerit`` object itself, which I believe contains slightly more overhead than the with-statement version. (I should test to determine if this is the case). In the case where it does make a difference, a cool syntax might look like: >>> import math >>> from timerit import Timerit >>> with timer in Timerit(num=200, verbose=2): >>> math.factorial(10000) The other case is that my ``ubelt.Cacher`` library. Currently it requires 4 lines of boilerplate syntax. >>> import ubelt as ub >>> # Defines a cache name and dependencies, note the use of `ub.hash_data`. >>> cacher = ub.Cacher('name', cfgstr=ub.hash_data('dependencies')) # boilerplate:1 >>> # Calling tryload will return your data on a hit and None on a miss >>> data = cacher.tryload() # boilerplate:2 >>> # Check if you need to recompute your data >>> if data is None: # boilerplate:3 >>> # Your code to recompute data goes here (this is not boilerplate). >>> data = 'mydata' >>> # Cache the computation result (pickle is used by default) >>> cacher.save(data) # boilerplate:4 But a conditional ``with`` syntax would reduce boilerplate to 3 lines. >>> import ubelt as ub >>> with ub.Cacher('name', cfgstr=ub.hash_data('dependencies')) as cacher: >>> data = 'mydata' >>> cacher.save(data) >>> data = cacher.data I'm sure there are a lot of viable syntax variations, but does the idea of a conditional or loop aware "with" statement seem like a reasonable language proposal? -- -Dr. Jon Crall (him) [View Less]

13 33

Add extend_const action to argparse
by python＠roganartu.com Feb. 8, 2021

Feb. 8, 2021

Forgive me if this idea has been discussed before, I searched the mailing lists, the CPython repo, and the issue tracker and was unable to find anything. I have found myself a few times in a position where I have a repeated argument that uses the `append` action, along with some convenience arguments that append a specific const to that same dest (eg: `--filter-x` being made equivalent to `--filter x` via `append_const`). This is particularly useful in cli apps that expose some kind of … [View More]

3 2

Make UTF-8 mode more accessible for Windows users.
by Inada Naoki Feb. 6, 2021

Feb. 6, 2021

The fact that Python does not use UTF-8 as the default encoding when opening text files is an obstacle for many Windows users, especially beginners in programming. If you search for UnicodeDecodeError, you will see that many Windows users have encountered the problem. This list is only part of many search results. * https://qiita.com/Yuu94/items/9ffdfcb2c26d6b33792e * https://www.mikan-partners.com/archives/3212 * https://teratail.com/questions/268749 * https://github.com/neovim/pynvim/… [View More]

6 31

List .index() Method
by Francis O'Hara Aidoo Jan. 31, 2021

Jan. 31, 2021

Hello, I'm Francis, a beginner programmer from Ghana, West Africa. I was wondering why the list .index() method doesn't return negative indices as well as well as positive indices. Although ambiguity will make it difficult to implement, in certain cases, it would be useful if it could return the negative index. For instance, if one creates an if statement that checks whether an element is the last item in a list as follows: listy = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] if listy.index(10) == -1: … [View More]

4 3

Questions about using PEP 637 syntax in PEP 646
by gohanpra＠gmail.com Jan. 30, 2021

Jan. 30, 2021

Hello, This is a great PEP. It turns out to be applicable in a variety of scenarios. Case in point: Matthew Rahtz and I are working on PEP 646: Variadic Generics (https://www.python.org/dev/peps/pep-0646/; discussion on typing-sig). It is a type system feature that allows specifying an arbitrary tuple of type variables instead of a single type variable. We plan to use your proposed syntax to represent unpacking a tuple type. This would be analogous to `*` for unpacking a tuple value: + `… [View More]

4 4

Parameter documentation
by abed...＠gmail.com Jan. 29, 2021

Jan. 29, 2021

Currently, python allows variable documentation via PEP 526 <https://www.python.org/dev/peps/pep-0526/>. For most functions with short parameter lists that can fit in a reasonable column limit, I prefer the traditional declaration style with Google-style doc strings: *def connect_to_next_port(self, minimum: int) => int: * """Connects to the next available port. Args: minimum: A port value greater or equal to 1024. Returns: The new minimum port. … [View More]

2 6

Adding `open_text()` builtin function. (relating to PEP 597)
by Inada Naoki Jan. 26, 2021

Jan. 26, 2021

Hi, all. I am rewriting PEP 597 to introduce a new EncodingWarning, which subclass of DeprecationWarning and used to warn about future default encoding change. But I don't think we can change the default encoding of `io.TextIOWrapper` and built-in `open()` anytime soon. It is disruptive change. It may take 10 or more years. To ease the pain caused by "default encoding is not UTF-8 (almost) only on Windows" (*), I came up with another idea. This idea is not mutually exclusive with PEP 597, but … [View More]

13 54

Provide UTF-8 version of Python for Windows.
by Inada Naoki Jan. 26, 2021

Jan. 26, 2021

Sorry for posting multiple threads so quickly. Microsoft provides UTF-8 code page for process. It can be enabled by manifest file. https://docs.microsoft.com/ja-jp/windows/uwp/design/globalizing/use-utf8-co… How about providing Python binaris both of "UTF-8 version" and "ANSI version"? This idea can provide a more smooth transition of the default encoding. 1. Provide UTF-8 version since Python 3.10 2. (Some years later) Recommend UTF-8 version 3. (Some years later) Provide only UTF-8 version … [View More]

6 10

dataclass: __init__ kwargs and Optional[type]
by Paul Bryan Jan. 25, 2021

Jan. 25, 2021

I've created a helper class in my own library that enhances the existing dataclass: a) __init__ accepts keyword-only arguments, b) Optional[...] attribute without a specified default value would default to None in __init__. I think this could be useful in stdlib. I'm thinking a dataclass decorator parameter like "init_kwonly" (default=False to provide backward compatibility) that if True would implement this behavior. Thoughts?

2 3

Changing the default text encoding of pathlib
by Inada Naoki Jan. 25, 2021

Jan. 25, 2021

My previous thread is hijacked about "auto guessing" idea, so I split this thread for pathlib. Path.open() was added in Python 3.4. Path.read_text() and Path.write_text() was added in Python 3.5. Their history is shorter than built-in open(). Changing its default encoding should be easier than built-in open and TextIOWrapper. New default encodings are: * read_text() default encoding is "utf-8-sig" * write_text() default encoding is "utf-8" * open() default encoding is "utf-8-sig" when mode is … [View More]

3 2

Re: [Python-Dev] reversed enumerate
by Steven D'Aprano Jan. 22, 2021

Jan. 22, 2021

Hi Ilya, I'm not sure that this mailing list (Python-Dev) is the right place for this discussion, I think that Python-Ideas (CCed) is the correct place. For the benefit of Python-Ideas, I have left your entire post below, to establish context. [Ilya] > I needed reversed(enumerate(x: list)) in my code, and have discovered > that it wound't work. This is disappointing because operation is well > defined. It isn't really well-defined, since enumerate can operate on infinite … [View More]iterators, and you cannot reverse an infinite stream. Consider: def values(): while True: yield random.random() a, b = reversed(enumerate(values()) What should the first pair of (a, b) be? However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too. My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense. But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard. On the other hand, I think that there is a fairly lightweight work around. Define a helper function: def countdown(n): while True: yield n n -= 1 then call it like this: # reversed(enumerate(seq)) zip(countdown(len(seq)-1), reversed(seq))) So it isn't terribly hard to work around this. But I agree that it would be nice if enumerate encapsulated this for the caller. One potentially serious question: what should `enumerate.__reversed__` do when given a starting value? reversed(enumerate('abc', 1)) Should that yield...? # treat the start value as a start value (1, 'c'), (0, 'b'), (-1, 'a') # treat the start value as an end value (3, 'c'), (2, 'b'), (1, 'a') Something else? My preference would be to treat the starting value as an ending value. Steven On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote: > Hi, > > I needed reversed(enumerate(x: list)) in my code, and have discovered that > it wound't work. This is disappointing because operation is well defined. > It is also well defined for str type, range, and - in principle, but not > yet in practice - on dictionary iterators - keys(), values(), items() as > dictionaries are ordered now. > It would also be well defined on any user type implementing __iter__, > __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors. > > That's plenty of usecases, therefore I guess it would be quite useful to > avoid hacky / inefficient solutions like described here: > https://code.activestate.com/lists/python-list/706205/. > > If deemed useful, I would be interested in implementing this, maybe > together with __reversed__ on dict keys, values, items. > > Best Regards, > -- > Ilya Kamen > > ----------- > p.s. > > *Sketch* of what I am proposing: > > class reversible_enumerate: > > def __init__(self, iterable): > self.iterable = iterable > self.ctr = 0 > > def __iter__(self): > for e in self.iterable: > yield self.ctr, e > self.ctr += 1 > > def __reversed__(self): > try: > ri = reversed(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > try: > l = len(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > indexes = range(l-1, -1, -1) > for i, e in zip(indexes, ri): > yield i, e > > for i, c in reversed(reversible_enumerate("Hello World")): > print(i, c) > > for i, c in reversed(reversible_enumerate([11, 22, 33])): > > print(i, c) > _______________________________________________ > Python-Dev mailing list -- python-dev(a)python.org > To unsubscribe send an email to python-dev-leave(a)python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUD… > Code of Conduct: http://python.org/psf/codeofconduct/ [View Less]

7 14

Making TYPE_CHECKING builtin.
by Inada Naoki Jan. 19, 2021

Jan. 19, 2021

Hi, all. I want to write type hints without worrying about runtime overhead. Current best practice is: ``` from __future__ import annotations import typing if typing.TYPE_CHECKING: import xxx # modules used only in type hints. ``` But it would be nice if I can avoid importing even "typing" module. How about adding TYPE_CHECKING builtin that is False? ``` from __future__ import annotations if TYPE_CHECKING: from typing import Any, Optional # We can use Any, Optional, etc here. ```… [View More]

4 4

Python with braces formal proposal?
by Paul Sokolovsky Jan. 19, 2021

Jan. 19, 2021

Hello, There're tons of projects which introduce alternative braces (i.e. C-like) syntax for Python. Most of them are however not properly documented, and definitely not spec'ed for what they do. I wonder, does anyone here remember more or less formal proposal for braces syntax? A "minimum viable product" criteria would be support for lossless indent -> braces -> indent syntax roundtripping. -- Best regards, Paul mailto:pmiscml@gmail.com

11 34

pathlib.Path.makedirs
by Antonio Cavallo Jan. 18, 2021

Jan. 18, 2021

Hi, I've found myself typing too many time this: pathlib.Path("some-dir-name").mkdir(parent=True, exist_ok=True) Wouldn't be better to have a pathlib.Path.makedirs with parent/exist_ok set to True by default? Thanks

3 2

pathlib enhancements
by Todd Jan. 18, 2021

Jan. 18, 2021

I know enhancements to pathlib gets brought up occasionally, but it doesn't look like anyone has been willing to take the initiative and see things through to completion. I am willing to keep the ball rolling here and even implement these myself. I have some suggestions and I would like to discuss them. I don't think any of them are significant enough to require a pep. These can be split it into independent threads if anyone prefers. 1. copy The big one people keep bringing up that I … [View More]strongly agree on is a "copy" method. This is really the only common file manipulation task that currently isn't possible. You can make files, read them, move them, delete them, create directories, even do less common operations like change owners or create symlinks or hard links. A common objection is that pathlib doesn't work on multiple paths. But that isn't the case. There are a ton of methods that do that, including: * symlink_to * link_to * rename * replace * glob * rglob * iterdir * is_relative_to * relative_to * samefile I think this is really the only common file operation that someone would need to switch to a different module to do, and it seems pretty strange to me to be able to make symbolic or hard links to a file but not straight up copy one. 2. recursive remove This could be a "recursive" option to "rmdir" or a "rmtree" method (I prefer the option). The main reason for this is symmetry. It is possible to create a tree of folders (using "mkdir(parents=True)"), but once you do that you cannot remove it again in a straightforward way. 3. newLine for write_text This is the only relevant option that "Path.open" has but "Path.write_text" doesn't, and is a serious omission when dealing with multiple operating systems. 4. uid and gid You can get the owner and group name of a file (with the "owner" and "group" methods), but there is no easy way to get the corresponding number. 5. Stem with no suffixes The stem property only takes off the last suffix, but even in the example given ('my/library.tar.gz') it isn't really useful because the suffix has two parts ('.tar' and '.gz'). I suggest another property, probably called "rootstem" or "basestem", that takes off all the suffixes, using the same logic as the "suffixes" property. This is another symmetry issue: it is possible to extract all the suffixes, but not remove them. 6. with_suffixes Equivalent to with_suffix, but replacing all suffixes. Again, this is a symmetry issue. It is hard to manipulate all the suffixes right now, as the example show. You can add them or extract them, but not change them without doing several steps. 7. exist_ok for is_* methods Currently all the is_* methods (such as is_file) return False if the file doesn't exist or if it is a broken symlink. This can be dangerous, since it is not trivially easy to tell if you are dealing with the wrong type of file vs. a missing file. And it isn't obvious behavior just from the method name. I suggest adding an "exist_ok" argument to all of these, with the default being "True" for backwards-compatibility. This argument name is already in use elsewhere in pathlib. If this is False and the file is not present, a "FileNotFoundError" is raised. [View Less]

10 21

Consider having numbers.Real provide __complex__?
by Neil Girdhar Jan. 18, 2021

Jan. 18, 2021

I've been following along various issues in mypy regarding challenges with getting the ABCs in numbers (https://docs.python.org/3/library/numbers.html) working. Currently, there's a lot of clever logic in the functions int, float, and complex. Currently, inheriting from Real doesn't give you __complex__. This is because the complex function doesn't need it—it falls back to __float__. I think it would be nice if it did for the purpose of type annotations. Best, Neil

3 3

Best practices of class "reopening" a-la Ruby?
by Paul Sokolovsky Jan. 17, 2021

Jan. 17, 2021

Hello, Ruby has following feature. Suppose the existing class "Cls" is scope (either defined before or imported from some module), then the code like: class Cls def mixin_method(args) ... end end Will "reopen" (Ruby term) that class and will add a new method "mixin_method" to it. Syntactically, that's complete madness - the same syntax is used to both define the class initially and augment it afterwards, and that's expectedly leads to user confusion, e.g. https://… [View More]

5 6

PEP Draft: Build Dependency Specification for Manylinux Wheels
by Chris Antonellis Jan. 16, 2021

Jan. 16, 2021

Hello- This is a draft of a future PEP regarding capturing wheel build dependencies in pyproject.toml. I am looking for any critique or input prior to creating a PR against the PEP repository, Thank you! Previous discussion can be seen here: * https://mail.python.org/archives/list/python-ideas@python.org/message/EQKE5… * https://mail.python.org/archives/list/python-ideas@python.org/message/CGUVG…) PEP: 9999 Title: Build Dependency Specification for Manylinux Wheels Author: Christopher … [View More]Antonellis <cantonellis(a)wayfair.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 22-Dec-2020 Post-History: 21-Dec-2020 Abstract ======== This document specifies how Python software packages should specify what actions are required to modify a standard manylinux environment to correctly build and bundle the package into a manylinux wheel. Motivation ========== Python wheels with compiled extensions may link to system libraries, requiring that the system libraries be available on the host systems (the system the wheel is installed on) to operate correctly. The manylinux project solves this problem through the auditwheel package, which identifies system libraries at compile time and bundles required libraries with the python wheel. This allows the python library, its compiled extensions, and any required system libraries to be installed on a host system without having to install the system library directly. In an ideal world, all package authors would make use of the manylinux project and all python packages that require system libraries would provide compiled, bundled distributions on PyPI. However, this is not the case, and many packages do not. There are valid instances where an author may not provide a manylinux wheel by choice: for example when a required system library cannot be bundled due to licensing. **However, there are packages on PyPI that do not provide a wheel when one could be provided.** This means that these libraries require local compilation prior to use, resulting in multiple negative side effects for the end user: - Required system libraries are not easily determined. They must be gleaned from project-specific documentation with no standardized format. - Compiling extensions can take a long time, adding additional expense to rebuilding environments. - Compiling a wheel that requires system libraries is non-trivial; it is easy to mismatch system library and python library version and be presented with cryptic error messages. Some authors do provide manylinux wheels on PyPI by making use of the manylinux project. However, the manylinux project does not provide a standardized way to capture environment setup data. This results in package authors keeping this data in project documentation or sometimes not recording it at all. Rationale ========= This PEP proposes a common format for the data required to correctly setup manylinux environments to compile a wheel with required system libraries. This concept is borrowed from package managers such as RPM which make use of a .SPEC file to capture this data. This data can be used in a manylinux container to set up the environment prior to compiling, resulting in a valid manylinux wheel. This data can be standardized to allow for automated building of manylinux wheels. Standardization of this data will allow package consumers to more easily contribute to building manylinux wheels when an existing distribution is lacking or not available. Specification ============= The data will be located in the `pyproject.toml` file of a python project, in a main table titled `manylinux_build_specification`. The data will be grouped in sub tables titled per the manylinux version they are targeting, ex `manylinux2014`. **pyproject.toml** .. code-block:: toml [manylinux_build_specification.manylinux2014] extra_base_system_repositories = ["http://foo.com/packages/"] system_dependencies = ["foo-1.0.0", "bar-1.0.0"] python_dependencies = ["foo==1.0.0", "bar==1.0.0"] environment_variables = ["FOO=BAR"] steps = [ "./scripts/build_and_upload.sh --my_option" ] All actions will be performed within a manylinux image. Given that the manylinux project uses CentOS as the base linux flavor, we can assume the following: * Use of `yum` for system package management * Python versions available in `/opt/python/` `extra_base_system_repositories` -------------------------------- Repositories to add to `yum` prior to installing system dependencies Additional repositories from which to download system dependencies. This allows access to builds of system libraries with the most up to date patches etc. `system_dependencies` --------------------- System dependencies to install with `yum` prior to building. Entries are expected to be in `yum` `name-version` format. `environment_variables` ----------------------- Environment variables to set prior to building. `python_dependencies` --------------------- Python libraries to install with `pip` prior to building. Will be installed for each version of python available in `/opt/python/`. `steps` ------- Steps to be executed sequentially with bash. The entire build process can be captured here, or this can be a call to a separate script. How to Teach This ================= This will be taught through examples and documentation provided in a reference implementation. Reference Implementation ======================== A reference implementation is currently in development. This will include the following: 1. The data spec will be defined in an example python package. 2. A python package will be created that consumes the data spec and sets up a manylinux container appropriately. 3. A manylinux docker image will be created that runs the python package that consumes the data spec prior to building a wheel. References ========== - `PEP 508 -- Dependency specification for Python Software Packages <https://www.python.org/dev/peps/pep-0508/>`_ - `PEP 518 -- Specifying Minimum Build System Requirements for Python Projects <https://www.python.org/dev/peps/pep-0518/>`_ - `PEP 571 -- The manylinux2010 Platform Tag <https://www.python.org/dev/peps/pep-0571/>`_ - `PEP 599 -- The manylinux2014 Platform Tag <https://www.python.org/dev/peps/pep-0599/>`_ - `PEP 600 -- Future ‘manylinux’ Platform Tags for Portable Linux Built Distributions <https://www.python.org/dev/peps/pep-0600/>`_ - `PEP 631 -- Dependency specification in pyproject.toml based on PEP 508 <https://www.python.org/dev/peps/pep-0631/>`_ - `RPM Packaging Guide: What is a SPEC File? <https://rpm-packaging-guide.github.io/#what-is-a-spec-file>`_ Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. [View Less]

2 2

Off-topic: What costs NaN pounds for a null amount?
by Jonathan Fine Jan. 14, 2021

Jan. 14, 2021

Hi There's interest here in arithmetic operations on NaN . I've just seen a product listed as costing NaN pounds to buy a null amount. That was written as £NaN/null. The bargain item is Glade Shake & Vacuum Citrus, and you can see it at https://www.tesco.com/groceries/en-GB/products/253732570 Searching on this site for 'NaN' brings up many entries for https://en.wikipedia.org/wiki/Naan bread. with best regards Jonathan

2 1

Re: Additional LRU cache introspection facilities
by Raymond Hettinger Jan. 14, 2021

Jan. 14, 2021

> I propose a method: > ... > returns a dictionary {arg: value} representing the cache. > It wouldn't be the cache itself, just a shallow copy > of the cache data I recommend against going down this path. It exposes (and potentially locks in) implementation details such as how we distinguish positional arguments, keyword arguments, and type information (something that has changed more than once). Also, a shallow copy still leaves plenty of room for meddling with the contents … [View More]

2 1

Renaming variable 'mod' to 'readline' in pymain_import_readline()
by Jero Bado Jan. 13, 2021

Jan. 13, 2021

Hello, In Modules/main.c, wherever PyImport_ImportModule() is use, the variables (runpy and sys) are named as follows: runpy = PyImport_ImportModule("runpy"); // line 265 sys = PyImport_ImportModule("sys"); // line 444 Can we also rename 'mod' to 'readline' for consistency in line 215? PyObject *mod = PyImport_ImportModule("readline"); // line 215 if (mod == NULL) { PyErr_Clear(); } else { Py_DECREF(mod); } Suggested changes: PyObject *readline = … [View More]

1 0

Additional LRU cache introspection facilities
by Steven D'Aprano Jan. 12, 2021

Jan. 12, 2021

Is anyone else interested in additional introspection facilities for the functools.lru_cache? You can view the cache hit and miss statistics using the cache_info() method, but you can't see what values actually are in the cache. That would be useful to me. I propose a method: @functools.lru_cache() def function(arg): ... function.cache_export() that returns a dictionary {arg: value} representing the cache. It wouldn't be the cache itself, just a shallow copy of the … [View More]

5 9

timeit improvement idea: add an option to measure a script execution time
by Alex Prengère Jan. 10, 2021

Jan. 10, 2021

Hello, Today I had a quite simple need, I am unsure about the best way to do it, and saw a possible improvement for the *timeit *module. I have about 30 Python scripts and I want to measure precisely their execution times, without measuring the interpreter startup time, because for most of them it is quite short (<1ms). I see several ways to do this: 0. Just measure the execution time with "time python script.py" > Not great for me, because for the fastest scripts, it is very … [View More]

4 5

Off-topic: Happy Birthday, Don Knuth (and Python coroutines)
by Jonathan Fine Jan. 10, 2021

Jan. 10, 2021

Hi Don Knuth, author of The Art Of Computer Programming and creator of the TeX typesetting system is 83 today. Happy Birthday Don. Thank you for TeX and everything else. You're warmly invited to join me online to celebrate Don's life and work. The virtual party is on Thursday 14 January, 6:30 to 7:30pm UK time. You'll find the zoom details at (and a goto tribute) at: https://jfine2358.github.io/post/2021/01/10/happy-birthday-don-knuth/ This link also contains a discussion of how merging … [View More]

1 0

Support reversed(itertools.chain(x, y, z))
by Ram Rachum Jan. 9, 2021

Jan. 9, 2021

Today I had a need: I had a tuple of dynamic sequence-like objects. I wanted to iterate on them reversed, starting with items of the last one and slowly making my way towards the first one. In short, I want `reversed(itertools.chain(x, y, z))` that behaves like `itertools.chain(map(reversed, (z, y, x)))`. What do you think?

7 10

Add command-line option to unittest for enabling post-mortem debugging
by Dominik Vilsmeier Jan. 9, 2021

Jan. 9, 2021

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-… [View More]

1 1

Add a "closed for" loop (was "PEP 533 Redux?")
by Brendan Moloney Jan. 7, 2021

Jan. 7, 2021

I mentioned this off-hand in the other thread, but the more I think about it the more I think it is a reasonable compromise between the two schools of thought on PEP 533. Wouldn't the new PEG parser allow a "closed for" statement without breaking code that uses "closed" as a variable (it would be a context sensitive keyword)? The users who work in the space that cares about such things can start writing: closed for document in read_newline_separated_json(path): ... Instead of: with … [View More]

2 1

An option to force the path separator for the "os.path.join()" method.
by Mikhail V Jan. 6, 2021

Jan. 6, 2021

I'd like to have an option to force the path separator for the "os.path.join()" method. E.g. if I run the script on Windows, but I generate, say, an URL, I'd find it convenient to use the same method, but with an explicit flag to "join" with the forward slash (because URLs use it). Currently I simply use an f-string to combine a path, like e.g.: path = f"{root}/{dir}" but it will insert extra "/" if there if "root" already has "/" on the end. But the "join" method will not, which is a … [View More]

7 7

Python JIT Compilation Thoughts
by redradist＠gmail.com Jan. 5, 2021

Jan. 5, 2021

Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!

9 15

Add venv activate command to the venv modules
by Abdur-Rahmaan Janhangeer Jan. 4, 2021

Jan. 4, 2021

Greetings list, put simply, be able to use $ python -m venv venv_name activate To activate an env instead of having each platform have a way of handling it Kind Regards, Abdur-Rahmaan Janhangeer about <https://compileralchemy.github.io/> | blog <https://www.pythonkitchen.com> github <https://github.com/Abdur-RahmaanJ> Mauritius

7 13

asyncio.to_process function, similar to asyncio.to_thread function
by Abdulla Al Kathiri Jan. 3, 2021

Jan. 3, 2021

Hello all, I suppose asyncio.to_thread uses concurrent.futures.ThreadPoolExecutor under the hood, so it saves us from getting the running event loop and calling the loop.run_in_executor function . Why don’t we have a similar function for the ProcessPoolExecutor, e.g. asyncio.to_process? Abdulla

2 1

A PEP to encourage people to mind docstrings in stub files
by Alexey Volobuev Jan. 2, 2021

Jan. 2, 2021

Hi! Sometimes you have to put docstrings in stub files e.g. when using code generation. In that case, stubs are the only place you can locate "objects" to add docstrings to. Thus it is frustrating to find that docstring-related software generally tends to ignore the presence of docstrings in stubs or to work incorrectly with those. I think we need a PEP to clarify the interaction between stubs and docstrings. This PEP might also encourage developers of docstring-related software to mind … [View More]

8 9

Standard tool for iterating over recursive data structures?
by Steven D'Aprano Jan. 2, 2021

Jan. 2, 2021

This isn't so much an idea for Python, as a request for ideas to solve a problem in Python. Back in the early days of Python, printing recursive lists could crash the interpreter: a = [1, 2, 3] a.append(a) print(a) If you try that today, you get a nice display: [1, 2, 3, [...]] It's not just lists that we can have this. Dicts and nearly any object can too: d = {} d[None] = d class Demo: pass x = Demo() x.attr = x With a bit of jiggery-pokery,… [View More]

13 24

Right way to dataclass -> dict conversion.
by Anton Abrosimov Jan. 1, 2021

Jan. 1, 2021

Example task: ``` from dataclasses import dataclass, asdict, fields from typing import List @dataclass class Point: x: int y: int @dataclass class Axes: title: str points: List[Point] def plot(title: str, points: List[Point]): print(title) for point in points: print(f'x={point.x}', f'y={point.y}') axes = Axes('example', [Point(i, i) for i in range(3)]) # Now I need to plot my axes. ``` Way 0: `plot(axes.title, axes.points)` Yes, it works great, but the … [View More]

3 7

Re: Standard tool for iterating over recursive data structures?
by Peter Ludemann Jan. 1, 2021

Jan. 1, 2021

A variant of the problem is the "occurs check", used in theorem proving: https://en.wikipedia.org/wiki/Occurs_check Removing the occurs check reduces the cost of unifying two terms (t1, t2) from *O(size(t1) + size(t2))* to *O(min(size(t1), size(t2)))*, which is why Prolog implementations don't do the occurs check by default (but provide unify_with_occurs_check and acyclic_term predicates for when the check is needed). One use for cyclic terms is in representing grammars. Of course, the loops … [View More]

1 0

Standard tool for iterating over recursive data structures?
by Jonathan Fine Jan. 1, 2021

Jan. 1, 2021

Hi Jeff You wrote: Detecting cycles will involve a lot of bikeshedding. (Sorry, couldn't > resist.) > Indeed. That really brought a smile to my face. Thank you. As I recall, bikeshedding is endless discussion (cycles again) of the COLOUR to paint the bikeshed. And in mathematics there's the topic of https://en.wikipedia.org/wiki/Graph_coloring. By the way, according to wikipedia, the convention of using colors originates from the coloring of the countries of a map, as in the famous … [View More]

1 0

Option to not raise if file does not exists for os.remove()?
by Marco Sulla Jan. 1, 2021

Jan. 1, 2021

What about a parameter, false by default, that suppresses the FileNotFoundError exception if true for os.remove()? Something similar to exist_ok for os.makedirs().

3 3

Re: Add user defined __eq__ to itertools.cycle class
by Andres Torres Jan. 1, 2021

Jan. 1, 2021

Yep, thanks for the explanation! After some more research into this it appears this exact situation is accounted for within the `SQLAlchemy` library (reference: https://github.com/sqlalchemy/sqlalchemy/issues/5810)! Additionally, due to the complications brought up in this thread it seems like it's probably best to leave `__eq__` as the default implementation. Thanks for expanding my knowledge and Happy New Year! 🎊 Andres On Thu, Dec 31, 2020 at 4:40 PM Rob Cliffe <rob.cliffe(a)… [View More]

1 1

built in to clear terminal
by Christopher Barker Jan. 1, 2021

Jan. 1, 2021

Sorry -- I can't find the thread, but there was a discussion on this list about whether there should be a built in for clearing the terminal screen. I'm was on the fence about it -- but one more data point: One of my beginning Python students just submitted code with this in it: def clear_screen(): ''' Function: clear_screen() (Windows/Unix) Input: None Process: Clear Screen Output: None ''' if (opSys == 'Windows'): os.system('cls') else: … [View More]

13 37

Unpack operator "**" and Mapping
by Anton Abrosimov Jan. 1, 2021

Jan. 1, 2021

I am trying to release comfortable dataclass unpacking using `**` operator. Now I have 5 different ways to do it. But not a single good one. Confused by the implementation of the unpacking operator. So when I try to unpack any custom class, I get the error: `type object argument after ** must be a mapping, not MyClass` Ok, nothing special. I need to use `collections.abc.Mapping` right? Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem. But additionally I get: `keys`… [View More]

11 58

Add asyncSetUpClass and asyncTearDown to IsolatedAsyncioTestCase
by Andres Torres Jan. 1, 2021

Jan. 1, 2021

Title says it all. This might have been an oversight while making this class. If not please let me know why it was not included! Thank you for considering my request! Andres 🙃

2 2

Add user defined __eq__ to itertools.cycle class
by Andres Torres Jan. 1, 2021

Jan. 1, 2021

As the title suggests, I am requesting an user defined __eq__ method for the itertools.cycle class. A suggestion for how equivalency might work is that __eq__ would return True if two cyclers have the same iterable and are at the same location in their cycle and False in any other condition. Thank you for considering my suggestion, Andres

2 2