Mailman 3 September 2020 - Python-ideas

zip(x, y, z, strict=True)
by Ram Rachum Dec. 1, 2022

Dec. 1, 2022

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

Bringing the print statement back
by Guido van Rossum March 2, 2022

March 2, 2022

In Python 3.10 we will no longer be burdened by the old parser (though 3rd party tooling needs to catch up). One thing that the PEG parser makes possible in about 20 lines of code is something not entirely different from the old print statement. I have a prototype: Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020, 16:31:17) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. Cannot read termcap database; using … [View More]dumb terminal settings. >>> print 2+2 4 >>> print "hello world" hello world >>> print "hello", input("Name:") Name:Guido hello Guido >>> print 1, 2, 3, sep=", " 1, 2, 3 >>> But wait, there's more! The same syntax will make it possible to call *any* function: >>> len "abc" 3 >>> Or any method: >>> import sys >>> sys.getrefcount "abc" 24 >>> Really, *any* method: >>> class C: ... def foo(self, arg): print arg ... >>> C().foo 2+2 4 >>> There are downsides too, though. For example, you can't call a method without arguments: >>> print <built-in function print> >>> Worse, the first argument cannot start with a parenthesis or bracket: >>> print (1, 2, 3) 1 2 3 >>> C().foo (1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: C.foo() takes 2 positional arguments but 4 were given >>> print (2+2), 42 4 (None, 42) >>> C().foo [0] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'method' object is not subscriptable >>> No, it's not April 1st. I am seriously proposing this (but I'll withdraw it if the response is a resounding "boo, hiss"). After all, we currently have a bunch of complexity in the parser just to give a helpful error message to people used to Python 2's print statement: >>> print 1, 2, 3 File "<stdin>", line 1 print 1, 2, 3 ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1, 2, 3)? >>> And IIRC there have been a number of aborted attempts at syntactic hacks to allow people to call functions (like print) without parentheses, although (I think) none of them made it into a PEP. The PEG parser makes this much simpler, because it can simply backtrack -- by placing the grammar rule for this syntax (tentatively called "call statement") last in the list of alternatives for "small statement" we ensure that everything that's a valid expression statement (including print() calls) is still an expression statement with exactly the same meaning, while still allowing parameter-less function calls, without lexical hacks. (There is no code in my prototype that checks for a space after 'print' -- it just checks that there's a name, number or string following a name, which is never legal syntax.) One possible extension I didn't pursue (yet -- dare me!) is to allow parameter-less calls inside other expressions. For example, my prototype does not support things like this: >>> a = (len "abc") File "<stdin>", line 1 a = (len "abc") ^ SyntaxError: invalid syntax >>> I think that strikes a reasonable balance between usability and reduced detection of common errors. I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo x' and we should only allow dotted names (sufficient to access functions in imported modules and method calls on variables). Or maybe we should only allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'. Or maybe we should really only bring back print statements. I believe there are some other languages that support a similar grammar (Ruby? R? Raku?) but I haven't investigated. Thoughts? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…> [View Less]

42 66

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 18, 2022

Feb. 18, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, … [View More]

17 32

Make fnmatch.filter accept a tuple of patterns
by Andre Delfino March 26, 2021

March 26, 2021

Frequently, while globbing, one needs to work with multiple extensions. I’d like to propose for fnmatch.filter to handle a tuple of patterns (while preserving the single str argument functionality, alas str.endswith), as a first step for glob.i?glob to accept multiple patterns as well. Here is the implementation I came up with: https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-mu… If this is deemed reasonable, I’ll write tests and documentation updates. Any opinion?

9 15

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman March 12, 2021

March 12, 2021

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/… [View More]clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman [View Less]

12 35

PEP 9999 (provisional): support for indexing with keyword arguments
by Stefano Borini March 10, 2021

March 10, 2021

PEP for support for indexing with keyword arguments is now submitted as PR. https://github.com/python/peps/pull/1612 Thanks to everybody involved in the development of the PEP and the interesting discussion. All your contributions have been received and often added to the document. If the PEP is approved, I would like to attempt an implementation, but I am not particularly skilled in the python internals. If a core developer is willing to teach me the ropes (especially in the new parser, I … [View More]

20 100

A shortcut to load a JSON file into a dict : json.loadf
by The Nomadic Coder March 10, 2021

March 10, 2021

Hi All, This is the first time I'm posting to this mailing group, so forgive me if I'm making any mistakes. So one of the most common ways to load json, is via a file. This is used extensively in data science and the lines. We often write something like :- with open(filename.json, "r") as f: my_dict = json.load(f) or my_dict = json.load(open("filename.json", "r")) Since this is sooooo common, why doesn't python have something like :- json.loadf("filename.json") Is there an obvious … [View More]

22 67

Add extend_const action to argparse
by python＠roganartu.com Feb. 8, 2021

Feb. 8, 2021

Forgive me if this idea has been discussed before, I searched the mailing lists, the CPython repo, and the issue tracker and was unable to find anything. I have found myself a few times in a position where I have a repeated argument that uses the `append` action, along with some convenience arguments that append a specific const to that same dest (eg: `--filter-x` being made equivalent to `--filter x` via `append_const`). This is particularly useful in cli apps that expose some kind of … [View More]

3 2

Re: [Python-Dev] reversed enumerate
by Steven D'Aprano Jan. 22, 2021

Jan. 22, 2021

Hi Ilya, I'm not sure that this mailing list (Python-Dev) is the right place for this discussion, I think that Python-Ideas (CCed) is the correct place. For the benefit of Python-Ideas, I have left your entire post below, to establish context. [Ilya] > I needed reversed(enumerate(x: list)) in my code, and have discovered > that it wound't work. This is disappointing because operation is well > defined. It isn't really well-defined, since enumerate can operate on infinite … [View More]iterators, and you cannot reverse an infinite stream. Consider: def values(): while True: yield random.random() a, b = reversed(enumerate(values()) What should the first pair of (a, b) be? However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too. My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense. But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard. On the other hand, I think that there is a fairly lightweight work around. Define a helper function: def countdown(n): while True: yield n n -= 1 then call it like this: # reversed(enumerate(seq)) zip(countdown(len(seq)-1), reversed(seq))) So it isn't terribly hard to work around this. But I agree that it would be nice if enumerate encapsulated this for the caller. One potentially serious question: what should `enumerate.__reversed__` do when given a starting value? reversed(enumerate('abc', 1)) Should that yield...? # treat the start value as a start value (1, 'c'), (0, 'b'), (-1, 'a') # treat the start value as an end value (3, 'c'), (2, 'b'), (1, 'a') Something else? My preference would be to treat the starting value as an ending value. Steven On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote: > Hi, > > I needed reversed(enumerate(x: list)) in my code, and have discovered that > it wound't work. This is disappointing because operation is well defined. > It is also well defined for str type, range, and - in principle, but not > yet in practice - on dictionary iterators - keys(), values(), items() as > dictionaries are ordered now. > It would also be well defined on any user type implementing __iter__, > __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors. > > That's plenty of usecases, therefore I guess it would be quite useful to > avoid hacky / inefficient solutions like described here: > https://code.activestate.com/lists/python-list/706205/. > > If deemed useful, I would be interested in implementing this, maybe > together with __reversed__ on dict keys, values, items. > > Best Regards, > -- > Ilya Kamen > > ----------- > p.s. > > *Sketch* of what I am proposing: > > class reversible_enumerate: > > def __init__(self, iterable): > self.iterable = iterable > self.ctr = 0 > > def __iter__(self): > for e in self.iterable: > yield self.ctr, e > self.ctr += 1 > > def __reversed__(self): > try: > ri = reversed(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > try: > l = len(self.iterable) > except Exception as e: > raise Exception( > "enumerate can only be reversed if iterable to > enumerate can be reversed and has defined length." > ) from e > > indexes = range(l-1, -1, -1) > for i, e in zip(indexes, ri): > yield i, e > > for i, c in reversed(reversible_enumerate("Hello World")): > print(i, c) > > for i, c in reversed(reversible_enumerate([11, 22, 33])): > > print(i, c) > _______________________________________________ > Python-Dev mailing list -- python-dev(a)python.org > To unsubscribe send an email to python-dev-leave(a)python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUD… > Code of Conduct: http://python.org/psf/codeofconduct/ [View Less]

7 14

Add command-line option to unittest for enabling post-mortem debugging
by Dominik Vilsmeier Jan. 9, 2021

Jan. 9, 2021

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-… [View More]

1 1

Python JIT Compilation Thoughts
by redradist＠gmail.com Jan. 5, 2021

Jan. 5, 2021

Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!

9 15

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Add the brotli & zstandard compression algorithms as modules
by Omer Katz Dec. 14, 2020

Dec. 14, 2020

Hello everyone, We have many compression algorithms as standard modules, but we lack two important new algorithms: brotli and zstandard. Brotli is an important compression algorithm both for HTTP2 and for compressing/decompressing HTTP responses. ZStandard is another algorithm for compression that can easily beat gzip with a better compression ratio and less compression time. Both are now stable enough and are past the 1.0 mark. Let's include them in the standard library and, of course, make … [View More]

12 21

'Infinity' constant in Python
by Cade Brown Nov. 23, 2020

Nov. 23, 2020

I am positing that Python should contain a constant (similar to True, False, None), called Infinity. It would be equivalent to `float('inf')`, i.e. a floating point value representing a non-fininte value. It would be the positive constant; negative infinity could retrieved via `-Infinity` Or, to keep float representation the same, the name `inf` could be used, but that does not fit Python's normal choice for such identifiers (but indeed, this is what C uses which is the desired behavior of … [View More]

28 161

f-strings as assignment targets
by Dennis Sweeney Oct. 23, 2020

Oct. 23, 2020

TL;DR: I propose the following behavior: >>> s = "She turned me into a newt." >>> f"She turned me into a {animal}." = s >>> animal 'newt' >>> f"A {animal}?" = s Traceback (most recent call last): File "<pyshell#2>", line 1, in <module> f"A {animal}?" = s ValueError: f-string assignment target does not match 'She turned me into a newt.' >>> f"{hh:d}:{mm:d}:{ss:d}" = "11:59:59" >>… [View More]> hh, mm, ss (11, 59, 59) === Rationale === Part of the reason I like f-strings so much is that they reduce the cognitive overhead of reading code: they allow you to see *what* is being inserted into a string in a way that also effortlessly shows *where* in the string the value is being inserted. There is no need to "paint-by-numbers" and remember which variable is {0} and which is {1} in an unnecessary extra layer of indirection. F-strings allow string formatting that is not only intelligible, but *locally* intelligible. What I propose is the inverse feature, where you can assign a string to an f-string, and the interpreter will maintain an invariant kept in many other cases: >>> a[n] = 17 >>> a[n] == 17 True >>> obj.x = "foo" >>> obj.x == "foo" True # Proposed: >>> f"It is {hh}:{mm} {am_or_pm}" = "It is 11:45 PM" >>> f"It is {hh}:{mm} {am_or_pm}" == "It is 11:45 PM" True >>> hh '11' This could be thought of as analogous to the c language's scanf function, something I've always felt was just slightly lacking in Python. I think such a feature would more clearly allow readers of Python code to answer the question "What kinds of strings are allowed here?". It would add certainty to programs that accept strings, confirming early that the data you have is the data you want. The code reads like a specification that beginners can understand in a blink. === Existing way of achieving this === As of now, you could achieve the behavior with regular expressions: >>> import re >>> pattern = re.compile(r'It is (.+):(.+) (.+)') >>> match = pattern.fullmatch("It is 11:45 PM") >>> hh, mm, am_or_pm = match.groups() >>> hh '11' But this suffers from the same paint-by-numbers, extra-indirection issue that old-style string formatting runs into, an issue that f-strings improve upon. You could also do a strange mishmash of built-in str operations, like >>> s = "It is 11:45 PM" >>> empty, rest = s.split("It is ") >>> assert empty == "" >>> hh, rest = rest.split(":") >>> mm, am_or_pm = s.split(" ") >>> hh '11' But this is 5 different lines to express one simple idea. How many different times have you written a micro-parser like this? === Specification (open to bikeshedding) === In general, the goal would be to pursue the assignment-becomes-equal invariant above. By default, assignment targets within f-strings would be matched as strings. However, adding in a format specifier would allow the matches to be evaluated as different data types, e.g. f'{foo:d}' = "1" would make foo become the integer 1. If a more complex format specifier was added that did not match anything that the f-string could produce as an expression, then we'd still raise a ValueError: >>> f"{x:.02f}" = "0.12345" Traceback (most recent call last): File "<pyshell#2>", line 1, in <module> f"{x:.02f}" = "0.12345" ValueError: f-string assignment target does not match '0.12345' If we're feeling adventurous, one could turn the !r repr flag in a match into an eval() of the matched string. The f-string would match with the same eager semantics as regular expressions, backtracking when a match is not made on the first attempt. Let me know what you think! [View Less]

22 110

Fwd: Re: Experimenting with dict performance, and an immutable dict
by Marco Sulla Oct. 15, 2020

Oct. 15, 2020

On Wed, 22 Jul 2020 at 15:35, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > The deltas also look small for a micro-benchmark. I certainly don't > think this is a sufficient reason to add a new datatype to Python. > I think some of the optimizations can be experimented with dict itself. Deltas seem to be small, because I divide them by the number of timeit runs. I think that a 30-45% gain in constructor speed is not something small. The benchmarks in question have the Name … [View More]that starts with "constructor". PS: sorry for double mail, I sent this message to only you by mistake. On Wed, 22 Jul 2020 at 18:26, Guido van Rossum <guido(a)python.org> wrote: > Did you study PEP 416 (frozendict) and PEP 603 (frozenmap)? > Yes. About frozenmap, at the very start I added to the benchmarks also immutables.Map, but I removed it when I realized that it was slower than frozendict in every bench. Maybe this is because immutables.Map is a C extension and not a builtin type. About PEP 416, yes, I tried to follow it. Indeed hash is calculated using the strategy described in the PEP. I also take a look to frozenset and tuple code. Frankly I must admit that the rejection of PEP 416 was a good idea. Indeed I started to implement an immutable dict only for a symmetry reason, and because I am somewhat fascinated by functional programming and immutables, without a rational reason. I suppose I would have given up and admitted that a frozendict is quite useless in Python, if I did not see that the constructor speed can be faster. I'm not sure 100% of the result. This is why I'm suggesting to try to implement the speed optimizations to the constructor of dict first: 1. the optimizations could be not safe. Indeed recently I got a segfault that I have to investigate 2. maybe dict can be really optimized further. After that, the difference between dict and frozendict performance could be minimal That said, maybe there are four good use cases for a frozendict: 1. they could be used for __annotations__ of function objects, and similar cases 2. they could be used to implement "immutable" modules or classes 3. frozendict can be easily cached, like tuples. 4. as I said, as an alternative to MappingProxyType, since it's very slow and it's used also in the C code of CPython and in the Python stdlib. This maybe is not useful because: a. the speed of MappingProxyType can be improved b. MappingProxyType can't be replaced because you *really* want and need a proxy [View Less]

6 20

A new suggestion for Python
by Jonatan Oct. 3, 2020

Oct. 3, 2020

Hi, My name is Jonatan and i am programming in Python for about 4 years, I have a great idea, there are iX` methods, such as __ior__, __iadd__, __iand__ etc.., which implements the |=, +=, &= behavior, it would be nice if you could implement also __igetattr__ or something, which means: instead of con = "some text here" con = con.replace("here", "there") we could do con = "some text here" con .= replace("here", "there") Please let me know what do you think about it, Jonatan

12 16

Trouble with "decorator mixins" and ABCs
by Ben Avrahami Oct. 1, 2020

Oct. 1, 2020

Hey all, I recently ran into some trouble and that I think deserves some attention. Consider the following case: class A(ABC): @abstractmethod def __lt__(self, other): pass @dataclass(order=True) class B(A): x: int = 0 class C(B): pass Although B technically implements A's abstract methods, it is still considered an abstract class, and calling B() will fail. However, C is considered a non-abstract class and calling C() will succeed.… [View More]

5 16

Package kwkey v0.0.2 provides items-keys duality to support PEP 637
by Jonathan Fine Sept. 30, 2020

Sept. 30, 2020

I'm pleased to announce v0.0.2 of my kwkeys package. https://pypi.org/project/kwkey/ The main new feature is items-key duality. Based on that, it emulates the semantics proposed by D'Aprano and van Rossum. It also emulates the semantics proposed by myself. What does this mean? Here, by duality I mean treating items[key] key[items] as roughly equivalent. This allows us to move the keyword arguments outside the square brackets. In other words, the following items[1, 2, a=3, b=4] = … [View More]

1 0

Regular Expression | re - Support String List Input Type List[str]
by Giang Le Sept. 29, 2020

Sept. 29, 2020

Hi Everyone, I would like to propose an idea for the regular expression module re.search(Pattern, Input) to support List[str] input type. So it will return a matched object list if the input is a string list. Otherwise, it will return a normal matched object, if the input is a normal string Best Regards Giang

5 5

Re: Regular Expression | re - Support String List Input Type List[str]
by Giang Hoang Le Sept. 29, 2020

Sept. 29, 2020

Hi, I would like to have the same function re.search(pattern, input) to keep main code structure to be the same. The input into the re.search function will be depended on other previous functions. It is also more dynamic for users. Thank you.

1 0

Improved Error Message for "Unhashable Type"
by Samuel Freilich Sept. 28, 2020

Sept. 28, 2020

The error message for using a mutable sequence as a set item or map key seems to frequently confuse Python beginners. The current wording is: >>> {[1, 2, 3]: [4, 5, 6]} TypeError: unhashable type: 'list' The first thing a Google search finds for "unhashable type" is ~4k Stack Overflow results like: https://stackoverflow.com/questions/13264511/typeerror-unhashable-type-dict The message does not include: * The type on which the operation is performed * Which input to the operation is … [View More]

3 3

Adding mixed positional and keyword to PEP 637
by David Mertz Sept. 28, 2020

Sept. 28, 2020

None of the examples posted in use cases show combinations of position and keyword indexing. This feels like an important example, and moreover without it, the discussion of what sentinel to use is less motivated. In discussions, I have mentioned two examples, e.g. distances[4:7, 100:120, 30:40, unit="metres"] Which could make comparisons of arrays with different internal units easier. disributed_array[4:7, 100:120, 30:40, source="worker1"] Which works be a natural way to address … [View More]

2 1

Re: Suggestion: annotated exceptions
by Sascha Schlemmer Sept. 27, 2020

Sept. 27, 2020

I think a clear distinction has to be made between errors that belong to the api surface of a function e.g. some `ValueError` or `ZeroDivisionError` and *exceptional* errors e,g.`KeyboardInterrupt` or `DatabaseExplodedError`. For the latter case exceptions work perfectly well as they are and it is both unhelpful and infeasible to annotate/document every exception - just let the exception bubble up and someone will handle it (might even be a process supervisor like supervisord) For the first … [View More]case I think a better way to model a function, that has some *expected* failure mode is to try and write a *total* function that doesn’t use exceptions at all. Let me show what I mean: ``` def div(a: int, b: int) -> float: return a / b ``` This function tries to divide two integers, trouble is that `div` is not defined for every input we may pass it, i.e. passing `b=0` will lead to `ZeroDivisionError` being raised. In other words `div` is a *partial* function because it is not defined for every member of its domain (type of its arguments). There are two ways to amend this issue and make `div` a *total* function that is honest about its domain and co-domain (types of input arguments and return type): 1. Extending the type of the functions co-domain (return type) and making the error case explicit (see https://returns.readthedocs.io/en/latest/index.html <https://returns.readthedocs.io/en/latest/index.html> for a library that implements this style of error handling) ``` def div(a: int, b: int) -> Maybe[float]: try: return Some(a / b) except ZeroDivisionError: return Nothing() ``` or ``` def div(a: int, b: int) -> Result[float, ZeroDivisionError]: try: return Success(a / b) except ZeroDivisionError as error: return Failure(error) ``` Now `div` does return a valid instance of `Maybe` (or `Result` if more a more detailed failure case is wanted) which is either something like `Some(3.1415) ` or `Nothing` (analogous to `None` ). The caller of the function then has to deal with this and mypy will correctly warn if the user fails to do so properly e.g. ´´´ div(1,1) + 1 # mypy will give a type error ´´´ 2. Restricting the functions domain to values with defined behavior ``` def div(a: int, b: NonZeroInteger) -> float: return a / b ``` In this case the burden is put on the caller to supply valid inputs to the function and this effectively pushes the error handling/ input validation out of `div` A language that does all this really well is F# (which like python is a multi-paradigm language that offers both object-oriented and functional-programming-oriented constructs). The trouble in python at the moment is that using something like `Maybe` and `Result` is not as nice to use as exceptions due to lack of a nice syntax i.e. for exceptions we have `try`/`except` and for functional error handling we’d need pattern matching. [View Less]

5 6

Trash bin
by Marco Sulla Sept. 26, 2020

Sept. 26, 2020

That I hope it's not the place where this proposal will be sent. My idea is apparently simple: what if, anytime we create an object, instead of deleting it, we send it in a trash bin? If the object is, for some reason, recreated, we can take it from the trash bin. If there is no more memory, the trash bin will be empty. The problem is that, probably, args and kwargs used by object creation must be stored. Maybe also copied? This could slow down the object creation instead of speed it up? … [View More]

4 3

Suggestion: annotated exceptions
by Sergio Fenoll Sept. 26, 2020

Sept. 26, 2020

Hi, I was wondering if the following idea would be a useful addition to the Python language and if it could use a new PEP. I personally find myself often looking into the documentation & implementation of libraries I use to try and figure out what exceptions a function may raise. In the same vein as adding type annotations to code, I think it'd be very useful to have exception "raises" annotations, i.e. a way to annotate what exceptions a function raises. Again, like type annotations, it … [View More]

15 57

Re: PEP 9999 (provisional): support for indexing with keyword arguments
by Eric Wieser Sept. 26, 2020

Sept. 26, 2020

Thanks for the PEP, and for pinging the numpy list about it. Some comments: Sequence unpacking remains a syntax error inside subscripts: Reason: unpacking items would result it being immediately repacked into a tuple A simple counter-example is [:, *args, :], which could be treated as [(slice(None), *args, slice(None))]. When there are index objects to the left or right of *args, it enables : syntax in places that would otherwise be forbidden. I’d argue this should be folded into the scope of … [View More]

3 2

Re: Suggestion: annotated exceptions
by Paul Moore Sept. 25, 2020

Sept. 25, 2020

On Fri, 25 Sep 2020 at 18:28, Samuel Colvin <S(a)muelcolvin.com> wrote: >> There's also the problem that you've explicitly acknowledged, that >> exception hints are *always* going to be inaccurate, in stark contrast >> to type hints which are expected to be correct when used. > > We should draw a distinction between two kinds of "inaccurate": > * inaccuracies because data about what errors could be raised is > missing some exceptions > * errors that … [View More]could happen at almost any time: > MemoryError, OverflowError, SystemError, SystemExit > > The first set of errors is not in stark contrast to type hints, > in fact they're very similar to errors in type hints which come up > from time to time. I get what you are saying, but I think you're being optimistic in assuming there's a clean distinction. It may be that you;'re thinking mostly about application code, where you are in control of what types exist in the system. But in library code, you simply cannot assume that. Consider a library that manipulates strings. There have been many attempts to write libraries that represent filesystem paths using a subclass of the string type. Suppose your library was passed one of those, and concatenated some text onto it. Possible exceptions that could in theory be raised include: * UnicodeError (if the path class checks that the value can be encoded in the filesystem encoding) * OSError (if the class enforces maximum path length or allowed character restrictions) * ValueError (if the programmer was lazy and used these rather than one of the above) as well as pretty much anything else, if the code that implemented __add__ for the path class had a bug. And of course, if libraries don't declare "expected" exceptions, application code using those libraries don't have anything to work with. Maybe you could get away with just declaring use of exception types that "you" created. But that's going to be very messy to get right. > Granted the "exceptions raised" database would be much more > immature than the type hints database to start with. But that's > not a reason not to start the error. Agreed. What *is* a reason to not start is if we can't even agree what we're building. > The second set of exceptions are a different matter, I think > in general they should not be included in the > "exceptions raised" database. If that makes exceptions > raised data invalid, then virtually all python logic in the world > that catches some exceptions but doesn't catch those > exceptions is also invalid! Precisely. So it's critical to clearly define exactly what exceptions can happen "at any time", and given that operations like indexing, basic operators, and even attribute access can call arbitrary Python code, essentially any exception can happen at any time. Unless you start mandating exception behaviour on basic operations, and then you preclude creative but unusual designs such as the path class I described above. Maybe that's not unreasonable (over-clever path classes never really became that popular) but Python has a tradition of *not* arbitrarily constraining how features can be used. > In other words: right now when we write code and catch > exceptions we generally (but not always) catch > exceptions that are likely to happen in the general > course of calling the code in the try, except block. But > not all the strange things that could conceivably happen. > > I think we should maintain that convention but provide a > way to remind developers when they forget to catch an exception. How do you distinguish between a developer "forgetting" to catch an exception and deliberately letting it bubble up. I'm a strong -1 on any proposal that makes people explicitly say "I meant to let this through" (it's essentially leading to checked exceptions, as well as violating the principle I stated above about not constraining how people use features - in this case the feature of exceptions bubbling up). And it's obviously impossible to make people declare when they forgot to catch an exception. > I guess overall, the question is: would it be useful to have a > way to check what exceptions a block of code might raise? > > Yes, massively. You state that without any justification, and it's the key point. My view is that it's (barely) possible that it might be occasionally convenient, but no-one has really demonstrated a benefit that I care about at this point. And to be honest, claims like "tools might be able to warn about certain types of error" are too vague to carry much weight. I've been developing Python for many years, and work on pip, which is a major codebase that's full of legacy weirdness - and I don't think I've *ever* hit an issue where I've felt that having a checker warn me that I haven't caught a type of exception would be useful. (Many, many times I've had to fix issues where uncaught exceptions were raised, but they have basically always been logic errors where fixing the underlying issue is the correct action, not catching the exception. And that raises another point - often, uncaught exceptions are *better* than carefully catching everything. Better for the developer, not for the end user, but I'll come to that. If an uncaught exception happens, there's a full traceback that is often invaluable in debugging what went wrong. Certainly, a traceback is the worst sort of thing for the user to see, but I've debugged way too many systems where some bright spark has added an exception handler that converts exceptions to "user friendly" messages, and in doing so completely discards all of the context that I need to fix the issue. One potential risk with this proposal is that it will encourage a culture of thoughtlessly catching exceptions and "handling" them, often in a way that makes maintenance and support orders of magnitude harder. Consider the following: def some_function() -> None[raises AppError]: try: do() some() complicated() piece() of() work() except Exception as e: log.debug("Caught exception " + str(e)) raise AppError("Unexpected problem with some_function()") That looks like it's doing a good job of exception handling, but it would be a nightmare to debug. It hides the underlying error in a debug message (that the user might well not have enabled). It doesn't retain the call stack. It puts a huge chunk of code in a try...except block. Clearly it can be improved, and ideally someone with experience would point out the issues in code review. But in the real world, a junior developer, armed with a style guide that says "functions must declare the exceptions they could raise, and must log any unexpected problems" could have this in production before you could say "real life's never as ideal as you'd hope" :-) > Would it be a lot of work? yes, massive. > > Would the work be worth the reward? Hard to say right now. Unless someone defines the reward more explicitly, it's easy - no. Give me £100 and I might give you something in return. That's the proposition here at the moment... Sorry - this comes across as rather negative. Your arguments are well-stated and I appreciate the time you're taking to address the points I'm making. It's just that I remain unconvinced, I'm afraid. Paul [View Less]

1 0

Suggest mentioning venv --upgrade in installer output & release notes
by Steve Barnes Sept. 25, 2020

Sept. 25, 2020

If you are upgrading a maintenance version (following the major.minor.maintenance naming) of python, e.g. 3.8.5 -> 3.8.6 it is also a good idea to update any 3.8 venvs that you may be using. The venv –upgrade flag is reasonably new and I think that relatively few people will have heard about it. I was wondering if it might not be a good idea to issue a reminder, at the end of installation &/or mention reasonably prominently in the release notes, that it is a good idea to upgrade any … [View More]

1 0

Naming Accepted PEPs as PAPs
by Abdur-Rahmaan Janhangeer Sept. 22, 2020

Sept. 22, 2020

Greeting lists, I am thinking of proposing to name accepted PEPs as PAPs namely: Python Accepted Proposals. Hence if we say PAP8 we know it's an accepted proposal. Backstory: I quoted V. Stinner in this <https://dev.to/abdurrahmaanj/the-zen-of-python-as-related-by-masters-1p9i> for PEP 608 <https://www.python.org/dev/peps/pep-0608/>, he told me by the way it was not accepted. This got me thinking that to know accepted peps on reading or hearing of it without seeing the status, … [View More]

13 36

f-string code formatting
by Joseph Perez Sept. 20, 2020

Sept. 20, 2020

Note: this suggestion should echo https://mail.python.org/archives/list/python-ideas@python.org/thread/UUFFAI… but has a bigger scope and completely differs in the way to proceed. Currently, Python lacks a way to integrate code parts into string; when I say code, I don't talk only about identifiers (on which is focusing `nameof` suggestion) but also about expression. Actually, this feature exists in Python but is hidden in a 3.8 [f-string feature](https://docs.python.org/3.8/whatsnew/3.8.html#… [View More]

4 9

Allow arbitrary expressions in decorator syntax
by Paolo Lammens Sept. 19, 2020

Sept. 19, 2020

The current grammar for decorators is decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE This restricts the decorator syntax to dotted names with an optional call with an argument list: @spam.eggsdef f(): ... @ham('foo, 'bar')def g(): ... I love the amount of generality and abstraction in Python, but I think this is a place where there could be more of that: I think arbitrary expressions should be allowed; the value of that expression is what gets used as the … [View More]

6 10

Magic attribute for attribute names retrieving
by Joseph Perez Sept. 18, 2020

Sept. 18, 2020

A lot of libraries use string for attribute names to do some "dynamic" things. A typical example are SQLAlchemy [validators](https://docs.sqlalchemy.org/en/13/orm/mapped_attributes.html#si…: ```python from sqlalchemy.orm import validates class EmailAddress(Base): __tablename__ = 'address' id = Column(Integer, primary_key=True) email = Column(String) @validates('email') # Here def validate_email(self, key, address): assert '@' in address return address ```… [View More] In this example of SQLAlchemy documentation, email validator use `"email"` string in order to associate the validator to the `email` column. However this dynamic things don't play well with all static tools like linters, but especially IDE (for usages finding, navigation, refactoring, etc.), and of course type checkers. This issue could be solved with a "magic attribute" `__attrs__` (name can be discussed), used the following way: ```python @dataclass class Foo: bar: int foo = Foo(0) assert getattr(foo, Foo.__attrs__.bar) == 0 assert getattr(foo, foo.__attrs__.bar) == 0 ``` To make it usable in class declaration, the `__attrs__` symbol should be added to class declaration namespace: ```python class Foo: bar: int @validator(__attrs__.bar) def validate(self): ... ``` No check would be done by `__attrs__`, they are let to linters which would integrate this language feature and check for the presence of the attribute in the class/instance. And that becomes more interesting because type checkers could use this feature to check dynamic attribute retrieving. A special type `Attribute[Owner, T]` could be defined to be used by type checkers such as `getattr` signature become: ``` # default parameter omitted for concision def getattr(obj: Owner, attr: Attribute[Owner, T], /) -> T: ... ``` (of course, `getattr` can still be used with strings, as the relation between `Attribute` and `str` is explained later) It could allow to add typing to function like the following: ```python Key = TypeVar("Key", bound=Hashable) def dict_by_attr(elts: Collection[T], key_attr: Attribute[T, Key]) -> Mapping[Key, T]: return {getattr(elt, key_attr): elt for elt in elts} ``` Concerning the implementation of this feature would be very straightforward, `__attrs__` being defined as a instance of a class: ```python class Attrs: __slots__ = [] def __getattribute__(self, name): return name ``` Thus, `Foo.__attrs__.bar` would be simply equal to `"bar"`; `Attribute` would be a special type, but backed by a `str`. hence there is no need to modify `getattr` implementation or existing code using it. `Attribute` type should then be a kind of `_SpecialForm`, compatible with string by the "relation" `str` <=> `Attribute[Any, Any]` The only modifications in the langage would be to add the `Attrs` class, an `__attrs__` field to `type` and in class definition namespace when it is evaluated. The rest of the work (checking the attribute presence, type checking with `Attribute[Owner, T]`) should be done by external tools (Pycharm, Mypy, etc.) to handle the feature. To sum up, `__attrs__` would be a pseudo-static wrapper to attribute name retrieving in order to benefit of static tools (refactoring, usages finding, navigation, type checking), with a straightforward and backward compatible implementation. And I don't see that as a "niche" feature, because a lot of libraries could actually benefit from it (and SQLAlchemy, Pydantic, etc. are not a small libraries). Joseph [View Less]

3 5

__typing__: new top level special variable to indicate typing enabled
by rea.aft＠gmail.com Sept. 18, 2020

Sept. 18, 2020

In short, the idea is simple to add a special variable, available in the global scope, so you can do this: if __typing__: import typing import another_heavy_typing_lib It has similar semantics as typing.TYPE_CHECKING, but it would be available in any context without the need to import anything, just like __name__, __package__ or __loader__. By default, it would be false (like False ou None). When running static checkers and related tools it would be a true value. Additionally, It … [View More]

1 0

Allow required parameters after optional in dataclasses
by Laurie Opperman Sept. 18, 2020

Sept. 18, 2020

I would like to do evil and nasty things with my dataclasses: inherit and extend them. In particular, I want to allow non-defaulted fields (ie required parameters to `__init__`) to be specified after defaulted fields (ie optional parameters), specifically in sub-classes (both parent and children are usable dataclasses). This is simply to reduce re-definition of fields between similar types (I wasn't able to come up with an elegant design using composition for my use-case). The technical … [View More]

1 0

Re: f-strings as assignment targets
by Alexis Masson Sept. 18, 2020

Sept. 18, 2020

> TL;DR: I propose the following behavior: > > >>> s = "She turned me into a newt." > >>> f"She turned me into a {animal}." = s > >>> animal > 'newt' > > >>> f"A {animal}?" = s > Traceback (most recent call last): > File "<pyshell#2>", line 1, in <module> > f"A {animal}?" = s > ValueError: f-string assignment target does not match 'She turned me into a newt.' … [View More]

4 6

Re: Python-ideas Digest, Vol 166, Issue 76
by Alexis Masson Sept. 17, 2020

Sept. 17, 2020

> On Fri, Sep 18, 2020 at 2:13 AM Alexis Masson<a.masson555(a)ntymail.com> wrote: >> This, in addition with locals().update(_), feels much better to me. Furthermore, it would allow other string-like classes, such as bytes or bytearray, to use that feature. >> > But locals().update() isn't a supported operation, except in the > situation where locals() is globals(). So what you're suggesting would > work fine in the REPL but not in any production usage. > > … [View More]

4 6

Re: A shortcut to load a JSON file into a dict : json.loadf
by William Pickard Sept. 16, 2020

Sept. 16, 2020

os.fspath exists for a reason.

1 0

Decorators for class non function properties
by redradist＠gmail.com Sept. 14, 2020

Sept. 14, 2020

Hi all, Seems like this topic was previously raised, but what if we add possibility to decorate non function properties in class: ```python class Neuron: @linear_activation activation ```

17 71

Access (ordered) dict by index; insert slice
by Hans Ginzel Sept. 14, 2020

Sept. 14, 2020

Date: Fri, 26 Jun 2020 18:47:44 +0200 From: Hans Ginzel <hans(a)matfyz.cz> To: Hans Ginzel <hans(a)artax.karlin.mff.cuni.cz> Subject: Access (ordered) dict by index; insert slice Hello, thank you for making dict ordered. Is it planned to access key,value pair(s) by index? See https://stackoverflow.com/a/44687752/2556118 for example. Both for reading and (re)writing? Is it planned to insert pair(s) on exact index? Or generally to slice? See splice() in Perl, https://perldoc.perl.… [View More]

32 182

Curious : Why staticmethod if classmethods can do everything a static method can?
by The Nomadic Coder Sept. 14, 2020

Sept. 14, 2020

This question is to further my understanding of the internals. It seems to me that a classmethod can do everything a staticmethod can, and additionally is not limited by inheritance. Why does python have them as two different constructs?

12 15

argparse.py in version 3.8.2 Action=Custom ActionClass needs an option to have methods
by rangapv08＠gmail.com Sept. 13, 2020

Sept. 13, 2020

Hi, Got the argparse() in version 3.8.2 and above the action=CusotmClass can have a user defined custom subclass and override the functionality in the def __init__() method. Along with this can we also have a built in custom method/methods that I can define under this custom Class, so that instead of calling the Action itself I have the option of calling the Action.Method() during the add_argument defintition. Regards Ranga

4 4

Compound with .. else statement
by Jimmy Thrasibule Sept. 9, 2020

Sept. 9, 2020

Hi, In Python, there are multiple [compound statements](https://docs.python.org/3/reference/compound_stmts.html) with the `else` keyword. For example: ``` for x in iterable: if x == sentinel: break else: print("Sentinel not found.") ``` or: ``` try: do_something_sensitive() except MyError: print("Oops!") else: print("We're all safe.") ``` In my situation, I would like to mix the `with` statement with `else`. In this case I would like that if no exception is … [View More]

6 10

wrapping up 4 years of open source python > 12 pep ideas
by Sylvain MARIE Sept. 7, 2020

Sept. 7, 2020

Dear python enthusiasts, --(you can skip this part if you're in a hurry)-- 4 years ago I was discovering this great language. Already an experienced software developer and data scientist at that time, accustomed with both low-level (C++), Object-Oriented (Java), Query (T-SQL), and Script (MATLAB) languages, I still have to admit that I was blown away by the capabilities and expressiveness of python ! Our team leveraged python during this period to develop our second cloud-based Analytics as … [View More]

1 0

Python logging with a custom clock
by N. Benes Sept. 4, 2020

Sept. 4, 2020

Dear list, log records in the Python logging library always use timestamps provided by `time.time()`, i.e. the usual system clock (UTC, CLOCK_REALTIME). This time is used as absolute timestamp in log records and for timestamps relative to the load of the library (Lib/logging/__init__.py: `_startTime`). I would like to receive feedback to and propose the attached (and possibly incomplete) patch that allows the programmer to provide a custom callable to get the time instead of `time.time()`. … [View More]For example, this could be: clock = lambda: time.clock_gettime(time.CLOCK_TAI) or def clock(): return time.clock_gettime(time.CLOCK_TAI) and the callable could be provided during initial logging setup logging.basicConfig(clock=clock, ...) There is a similar approach in log4j to specify a custom clock [0]. This change enables the use of non-UTC clocks, e.g. `CLOCK_TAI` or `CLOCK_MONOTONIC`, which are unaffected by leap seconds and count SI seconds. (In fact, logging's use of differences of UTC timestamps could make users believe that the obtained duration reflects SI seconds, which it doesn't in all cases.) Combining a custom absolute clock such as `CLOCK_TAI` with custom log formatters allows users to /store or transfer/ log records with TAI timestamps, and /display/ them with UTC timestamps (e.g. properly converted from TAI to UTC with a "60" second during active leap second). This resolves the ambiguity when analysing and correlating logs from different machines also during leap seconds. Attached is a simple example showing the different timestamps based on UTC and TAI (assuming the current offset of +37 seconds [1] is properly configured on the host, e.g. through PTP or `adjtimex()` with `ADJ_TAI`). $ export TZ=GMT $ date --iso-8601=seconds && python3 example.py 2020-09-02T14:34:14+00:00 2020-09-02T14:34:51+0000 INFO message According to the documentation `time.CLOCK_TAI` was introduced in Python 3.9 [2], but already today the system constant can be used (e.g. on Debian Buster, Linux 4.19.0, Python 3.7.3 it is 11). The two patches provided are for Python 3.7.3 (Debian Buster) and Python 3.8.5 (python.org). In the latter case, it may need to be considered how changing the Python logging clock works: it probably should fail if handlers are already configured, unless `force` is also provided and handlers are reset. Kind regards, -- nicolas benes [0] `log4j.Clock` in https://logging.apache.org/log4j/log4j-2.8/manual/configuration.html [1] https://www.timeanddate.com/worldclock/other/tai [2] https://docs.python.org/dev/library/time.html#time.CLOCK_TAI -------------------------------------------------------------- Nicolas Benes nbenes(a)eso.org + Software Engineer +E S+ European Southern Observatory https://www.eso.org O Karl-Schwarzschild-Strasse 2 + D-85748 Garching b. Muenchen Germany -------------------------------------------------------------- [View Less]

1 0

Pickle improvement: global qualnames for local classes
by haael Sept. 4, 2020

Sept. 4, 2020

Only objects of globally defined classes are picklable: class Global: pass picklable = Global() def f(): class Local: pass return Local Local_here = f() unpicklable = Local_here() However, instances become picklable if we assign the local class to some globally reachable identifier. Local_here.__qualname__ = 'Local_here' now_picklable = Local_here() Why not make it a feature? Let's define module level global … [View More]

3 3

Package kwargs to support Guido's favoured semantics
by Jonathan Fine Sept. 3, 2020

Sept. 3, 2020

The next release of kwkey will provide a class A with the following property. It is that >>> A(1, 2, a=3, b=4)[d] = 'val' will result in >>> d.__setitem__((1, 2), 'val', a=3, b=4) This is precisely the same call that would result from >>> d[1, 2, a=3, b=4] = 'val' using the semantics that Guido currently favours. Further, to better support exploration, the call instead will be >>> d._A_setitem__((1, 2), 'val', a=3, b=4) when d._A_setitem__ … [View More]

1 1

Ensuring that a package is available.
by Charles Machalow Sept. 3, 2020

Sept. 3, 2020

I may be in the minority, but sometimes it seems like having to spin up a venv, etc. to run a small script could be a little bit much. What if we have something like this POC I wrote: https://pypi.org/project/pyensure/ We could have a switch or python interpreter argument to have packages get auto-installed (to a temp directory) in the event that we don't have it installed in the current environment. One of the beauties of this is that it doesn't pollute the current environment, since … [View More]

3 2

Changing item dunder method signatures to utilize positional arguments (open thread)
by Ricky Teachey Sept. 3, 2020

Sept. 3, 2020

The following comment is from the thread about adding kwd arg support to the square bracket operator (eg, `Vec = Dict[i=float, j=float]`). On Tue, Aug 4, 2020, 2:57 AM Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: On 4/08/20 1:16 pm, Steven D'Aprano wrote: > Why would we want to even consider a new approach to handling keyword > arguments which applies only to three dunder methods, `__getitem__`, > `__setitem__` and `__delitem__`, instead of handling keyword arguments > … [View More]in the same way that every other method handles them? These methods are already kind of screwy in that they don't handle *positional* arguments in the usual way -- packing them into a tuple instead of passing them as individual arguments. I think this is messing up everyone's intuition on how indexing should be extended to incorporate keyword args, or even whether this should be done at all. -- Greg So here is the main question of this thread: Is there really not a backwards compatible, creative way a transition to positional args from a tuple in the item dunders could not be accomplished? It's not my intention to champion any specific idea here, just creating a specific space for ideas to be suggested and mulled over. There could be several reasons for changing the item dunder signatures. The immediate reason is making the intuition around how to add kwd arg support to square brackets more obvious and sane. A second reason is it might be more intuitive for users who have to learn and remember that multiple arguments to [ ] get packed into a tuple, but this doesn't happen anywhere else. Another reason: it could make writing code for specialized libraries that tend to abuse (for the good of us all!) item dunders, like pandas, much easier. Right now such libraries have to rely on their own efforts to break up a key: def __getitem__(self, key): try: k1, k2 = key except TypeError: raise TypeError("two tuple key required") But for regular function calls (as opposed to item getting) we get to write our signature however we want and rely on the language to handle all of this for us: def f(k1, k2): # no worries about parsing out the arguments ----------- One idea: change the "real" names of the dunders. Give `type` default versions of the new dunders that direct the call to the old dunder names. The new get and del dunders would have behavior and signatures like (I am including **__kwargs since that could be an option in the future) : def __getx__(self, /, *__key, **__kwargs): return self.__getitem__(__key, **__kwargs) def __delx__(self, /,, *__key, **__kwargs): del self.__delitem__(__key, **__kwargs) However the set dunder signature would be a problem, because to mirror the current behavior we end up writing what is now a syntax error: def __setx__(self, /, *__key, __value, **__kwargs): self.__setitem__(__key, __value, **__kwargs) The intended meaning above would be that the last positional argument gets assigned to __value. Maybe someone could suggest a way to fix this. Item getting, setting, deleting would call these new dunders instead of the old ones. I haven't thought through how to handle inheritance-- I'm sort of hoping someone smarter than me could come up with a way to solve that...: class My: def __getx__(self, my_arg, *args, my_kwarg, **kwargs): # the way I have written things this super call will cause a recursion error v = super().__getitem__(*args, **kwargs) return combine(my_arg, my_kwarg, v) By the way I'm not bike shedding on what these new dunder names should be, though we could probably do worse then getx, setx, and delx (nice and short!). ----------- Ok now this is the part where I get to wait for everyone smarter than me to show the errors of my ways/how naive I am. :) [View Less]

16 71

Decimal - official !!!
by dust dimka Sept. 2, 2020

Sept. 2, 2020

Rus: "Также любителям бухгалтерии тоже Decimal просто необходим. Я вот думаю Гвидо(создателю питона) надо ввести обозначение для децималов например 'D'.А то вот float есть x = 0.012f, а для децималов было бы удобно x = 0.012D или x = 0.012d...Причем если в выражении есть децимал то все float int double в программе подтягиваются до Decimal то есть x = 0.012f + 0.3D ... float 0.012f становится decimal, и х = сумме 2х децималов. " DeepL translate: "Also lovers of accounting, too, Decimal just need.… [View More]

1 0

Proposed new syntax for subscripting (was PEP 472)
by Steven D'Aprano Sept. 2, 2020

Sept. 2, 2020

This is slightly revised version of something I sent to Stefano two weeks ago. I hope he is planning to use this, or something similar, in the PEP, but for what it's worth here it is for discussion. This is, as far as I can tell, the minimum language change needed to support keywords in subscripts, and it will support all the desired use-cases. * * * (1) An empty subscript is still illegal, regardless of context. obj[] # SyntaxError (2) A single subscript remains a single … [View More]argument: obj[index] # calls type(obj).__getitem__(index) obj[index] = value # calls type(obj).__setitem__(index, value) del obj[index] # calls type(obj).__delitem__(index) (This remains the case even if the index is followed by keywords; see point 5 below.) (3) Comma-seperated arguments are still parsed as a tuple and passed as a single positional argument: obj[spam, eggs] # calls type(obj).__getitem__((spam, eggs)) obj[spam, eggs] = value # calls type(obj).__setitem__((spam, eggs), value) del obj[spam, eggs] # calls type(obj).__delitem__((spam, eggs)) Points (1) to (3) mean that classes which do not want to support keyword arguments in subscripts need do nothing at all. (Completely backwards compatible.) (4) Keyword arguments, if any, must follow positional arguments. obj[1, 2, spam=None, 3) # SyntaxError This is like function calls, where intermixing positional and keyword arguments give a SyntaxError. (5) Keyword subscripts, if any, will be handled like they are in function calls. Examples: # Single index with keywords: obj[index, spam=1, eggs=2] # calls type(obj).__getitem__(index, spam=1, eggs=2) obj[index, spam=1, eggs=2] = value # calls type(obj).__setitem__(index, value, spam=1, eggs=2) del obj[index, spam=1, eggs=2] # calls type(obj).__delitem__(index, spam=1, eggs=2) # Comma-separated indices with keywords: obj[foo, bar, spam=1, eggs=2] # calls type(obj).__getitem__((foo, bar), spam=1, eggs=2) and *mutatis mutandis* for the set and del cases. (6) The same rules apply with respect to keyword subscripts as for keywords in function calls: - the interpeter matches up each keyword subscript to a named parameter in the appropriate method; - if a named parameter is used twice, that is an error; - if there are any named parameters left over (without a value) when the keywords are all used, they are assigned their default value (if any); - if any such parameter doesn't have a default, that is an error; - if there are any keyword subscripts remaining after all the named parameters are filled, and the method has a `**kwargs` parameter, they are bound to the `**kwargs` parameter as a dict; - but if no `**kwargs` parameter is defined, it is an error. (7) Sequence unpacking remains a syntax error inside subscripts: obj[*items] Reason: unpacking items would result it being immediately repacked into a tuple. Anyone using sequence unpacking in the subscript is probably confused as to what is happening, and it is best if they receive an immediate syntax error with an informative error message. (8) Dict unpacking is permitted: items = {'spam': 1, 'eggs': 2} obj[index, **items] # equivalent to obj[index, spam=1, eggs=2] (9) Keyword-only subscripts are permitted: obj[spam=1, eggs=2] # calls type(obj).__getitem__(spam=1, eggs=2) del obj[spam=1, eggs=2] # calls type(obj).__delitem__(spam=1, eggs=2) but note that the setter is awkward since the signature requires the first parameter: obj[spam=1, eggs=2] = value # wants to call type(obj).__setitem__(???, value, spam=1, eggs=2) Proposed solution: this is a runtime error unless the setitem method gives the first parameter a default, e.g.: def __setitem__(self, index=None, value=None, **kwargs) Note that the second parameter will always be present, nevertheless, to satisfy the interpreter, it too will require a default value. (Editorial comment: this is undoubtably an awkward and ugly corner case, but I am reluctant to prohibit keyword-only assignment.) Comments -------- (a) Non-keyword subscripts are treated the same as the status quo, giving full backwards compatibility. (b) Technically, if a class defines their getter like this: def __getitem__(self, index): then the caller could call that using keyword syntax: obj[index=1] but this should be harmless with no behavioural difference. But classes that wish to avoid this can define their parameters as positional-only: def __getitem__(self, index, /): (c) If the method is declared with no positional arguments (aside from self), only keyword subscripts can be given: def __getitem__(self, *, index) # requires obj[index=1] not obj[1] Although this is unavoidably awkward for setters: # Intent is for the object to only support keyword subscripts. def __setitem__(self, i=None, value=None, /, *, index) if i is not None: raise TypeError('only keyword arguments permitted') Gotchas ------- If the subscript dunders are declared to use positional-or-keyword parameters, there may be some surprising cases when arguments are passed to the method. Given the signature: def __getitem__(self, index, direction='north') if the caller uses this: obj[0, 'south'] they will probably be surprised by the method call: # expected type(obj).__getitem__(0, direction='south') # but actually get: obj.__getitem__((0, 'south'), direction='north') Solution: best practice suggests that keyword subscripts should be flagged as keyword-only when possible: def __getitem__(self, index, *, direction='north') The interpreter need not enforce this rule, as there could be scenarios where this is the desired behaviour. But linters may choose to warn about subscript methods which don't use the keyword-only flag. -- Steve [View Less]

8 12

PEP-0472 revisited - draft
by Stefano Borini Sept. 2, 2020

Sept. 2, 2020

For those interested in the new draft of ex PEP-0472 (which has joined the choir invisible), I opened a draft PR https://github.com/python/peps/pull/1579 This first draft contains only the revisited motivation and background. I'll add the technical decisions etc in the upcoming nights. -- Kind regards, Stefano Borini

5 5