Mailman 3 November 2021 - Python-ideas

Should bare logical comparisons raise a warning?
by ehmatthes＠gmail.com Nov. 26, 2021

Nov. 26, 2021

Hi everyone, I've been really impressed by the recent improvements to error messages in Python 3.10. The specific suggestions in some error messages are going to save countless hours of people's time. The benefits for beginners are fairly obvious. But even for experienced programmers, the difference between seeing a missing colon in an error message and just adding it where it should have been vs spending 30 seconds or a minute figuring out what's missing at the start of a loop adds up. One common error that I haven't seen discussed is bare logical comparisons. They're syntactically legal so they don't raise errors, but I have never seen a real-world use case for one. Here's the simplest example I can come up with: name = 'eric' print(name) name == 'ever' print(name) We want to assign a new value to an existing variable, but we accidentally type a second equals sign. The output here makes the mistake fairly obvious, but in longer programs it can take a while to spot this issue. I know there are some editors and linters that flag this. Is there any reason not to raise a warning or even an error at the language level? Is there any reason to use a bare logical comparison in real-world code?

6 6

Re: extending ast.parse with some lib2to3.pytree features
by Neil Girdhar Nov. 22, 2021

Nov. 22, 2021

I wish this had gotten more attention! :)

3 2

Python Developer’s Guide chinese version
by zhouwenbonwpu＠mail.nwpu.edu.cn Nov. 22, 2021

Nov. 22, 2021

hi all, I want to know does Python Developer’s Guide have chinese version, If not, i want to do some translation, may i tell to PSF(or other people) to grant authorization? May you also want to do this translation, please contact me. Best, bobozi

4 6

Runtime-accessible attribute docstrings
by tmkehrenberg＠gmail.com Nov. 20, 2021

Nov. 20, 2021

Hi all, I have seen discussion of docstrings for class attributes before on this list, but not with this exact proposal. My motivation is that I have a dataclass for which I want to write docstrings that can be accessed at runtime. In my specific case, the docstrings would be used to create a help message. Sphinx supports a syntax to document class attributes. It looks like this: @dataclass class A: """Docstring for class A.""" x: int """Docstring for x""" y: bool = True "Docstring for y" It is a bit awkward that the docstring is *below* the definition of the attribute, but it can't be put above because it could be confused for the class docstring. My proposal would be to just enshrine this syntax as the syntax for attribute docstring, and to make them available at runtime. They would be stored in a dictionary like the type annotations. For example like this: A.__attrdoc__ == {"x": "Docstring for x", "y": "Docstring for y"} Best, Thomas

11 16

Recursive submit in concurrent.future.ProcessPoolExecutor
by Evan Greenup Nov. 19, 2021

Nov. 19, 2021

Hi, Currrently, is it allowed for process worker to submit new task to its parent executor? Considering the following Python script: ```python3 import concurrent.futures with concurrent.futures.ProcessPoolExecutor(max_workers=12) as ppe: def hello(n: int) -> int: if n == 0: return 0 return n+ ppe.submit(hello, n-1).result() future = ppe.submit(hello, 1) print(future.result()) ``` It will never stuck and never ended. Sent with [ProtonMail](https://protonmail.com/) Secure Email.

1 0

Adding pep8-casing-compliant aliases for the entire stdlib
by Matt del Valle Nov. 18, 2021

Nov. 18, 2021

So I was reading the docs for the `threading` module and I stumbled upon this little note: Note: In the Python 2.x series, this module contained camelCase names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower. And it got me thinking. Given that there is some precedent, would it be feasible to make a concerted effort to add aliases across the board for all public-facing stdlib types and functions that don't follow pep8-recommended casing? I realize that large chunks of the stdlib predates pep8 and therefore use various non-uniform conventions. For example, the logging module is fully camelCased, and many core types like `str` and `list` don't use PascalCase as pep8 recommends. The `collections` module is a veritable mosaic of casing conventions, with some types like `deque` and `namedtuple` being fully lowercased while others like `Counter` and `ChainMap` are PascalCased. My motivation for this twofold: 1) I'll confess that this has just always been a wart that has bothered me way more than it has any right to. I just *hate* it. Somewhere deep inside my lizard-brain it makes me unhappy to have disparate naming conventions in my code. I realize this isn't a good reason in and of itself but I wonder if this might not be the case for others as well. While I've come to accept it because that's just how it is, maybe it doesn't have to be this way? 2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not. So the scope of my suggestion is as follows: - lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`) - lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`) - pep8 and the python docs are updated to state that the pep8-compliant forms of stdlib names should be strongly preferred over the legacy names, and that IDEs and linters should include (configurable?) weak warnings to discourage the use of legacy-cased stdlib names - `help()` would be special-cased for builtin types to no longer display any current non-pep8-compliant names, and the python docs would also no longer show them, instead only making a note at the top of the page as with the `threading` module. Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road. If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal. So yeah, thoughts?

21 30

Make the global/nonlocal keyword a prefix
by Jonathan Sh. Nov. 17, 2021

Nov. 17, 2021

Usually when we want to make a variable/def/class etc global, we can just declare `global v` at the beginning, but I feel like there could be a better way to do that. So my idea is to allow doing stuff like `global a = 3` or `nonlocal def x(): ...` or even `for global i in []: ...` in addition to the current `global` statement to allow backwards compatibility What do you think?

5 6

Generic Type Parameters
by Abdulla Al Kathiri Nov. 16, 2021

Nov. 16, 2021

Hello, Instead of writing 'class A(Generic[T]): ...', can we do something like rust lang, i.e., 'class A[T]: ...'? Rust uses <T> but in python we use [T]. This reflects the type annotation for the class nicely. For example, a: A[T]. It’s weird seeing the word Generic inside the class inheritance parentheses which should be reserved for abstract base classes (abc) or explicit protocol classes or superclasses in general (even with their generic type parameters. For example, class A[T](Sequence[T]). Rust also defines the generic type parameter for functions. “fn func<T>(a: A[T]) -> T {function body}”. In Python, we can do the following: def func[T](a: A[T]) -> T: … With the new proposed type hint for Callable, The type annotation for the above function will be .. func: [T](A[T]) -> T And this as well reflects nicely with the function signature. Now, suddenly, we don’t need to write 'T=TypeVar(“T”, bound=…,)' if the generic type is enclosed for the class and function scopes (local not global) as in rust. "class A[T]: ...” can be thought of as a syntactic sugar for "class A(Generic[TypeVar(“T”)]): … “ . If the generic type implements a protocol or inherits from a base class, then we can do something like the following: @dataclass class A[T, U: Container[T]]: attr1: T attr2: U def check_all(self, x: T, container: U) -> bool: return (self.attr1 in self.attr2) and (x in container) here the T has no bounds nor constraints. It can be anything, but the U has to implement the Container protocol of type T. Lets create a function and see how we are going to use the class A above.. def test_function[T, U: Container[T]](a: A[T, U], x: T, container: U) -> bool: return a.check_all(x, container) a1 = A(10, [1, 2, 10]) # Valid creation since [1, 2, 10] is a Container[int] a2 = A(10, [“aa", “bb", “cc"]) # Not Valid, attr1: int, but attr2 is not Container[int] a3 = A(10.0, {1.0, 2.0, 10.0}) # Valid since {1.0, 2.0, 10.0} is a Container[float] test_function(a1, 3, [1, 5, 3]) #Valid test_function(a1, 3, {1, 5, 3}) # Not Valid The last one will be valid if the function signature were like the following: def test_function[T, U: Container[T]](a: A[T, U], x: T, container: Container[T]) -> bool: … because the the container argument (set[int]) does not need to be the same as U in a1 (list[int]). In Rust, they use the keyword “where” if the generic type parameters are long. I don’t know how we can do something similar in Python. Maybe like this? def test_function[T, U, V]( a: A[T, U], x: T, container: U, y: V ) -> bool: where ( U: Container[T], V: Sized + RandomProtocol, #suppose RandomProtocol has “random" method ) print(f”variable y size is {len(y)} and the random numbers are {y.random()}") return a.check(x, container) #### The + sign is to bound V further. That’s how rust does it. Think of intersection (Sized & RandomProtocol) as opposed to union. It could support multiple inheritance. To completely mimic TypeVar, we need to think of the constraints *args and the covariant and contravariant parameters. Maybe something like the following format: T: Bound(s) = constraint1 | constraint2 ... # T is invariant, e.g., "AnyStr = str | byte" has no bounds/abc/protocols, it has only constraints T<: Bound(s) = constraint1 | constraint2 … # T is covariant T>: Bound(s) = constraint1 | constraint2 … # T is contravariant U: Bound(s) = default_generic_type For the default type, this example illustrates it: class AddProtocol[T=Self, U=Self](Protocol): def __add__(self, other: T) -> U: … class GreaterProtocol[T=Self](Protocol): def __gt__(self, other: T) -> bool: ... @dataclass class Test[U]: where U: AddProtocol + GreaterProtocol = int #here we didn’t need to write AddProtocol[V, W] or GreaterProtocol[V] because they take default parameter Self (which is U in this case) for the generic types attr: U Also the below function does not need to define the generic parameter U if int is wanted def func_generic[U: AddProtocol + CompareProtocol](test1: Test[U], test2: Test[U]) -> tuple[U, bool]: addition = test1.attr + test2.attr comparison = test1.attr > test2.attr return addition, comparison def func_int(test1: Test, test2: Test) -> tuple[int, bool]: … test_int_1 = Test(1) test_int_2 = Test(2) test_str_1 = Test(“hello ”) test_str_2 = Test(“world”) func_generic(test_int_1, test_int_2) # Valid func_int(test_int_1, test_int_2) # Valid func_generic(test_str_1, test_str_2) # Valid func_int(test_str_1, test_str_2) # Invalid

1 0

Unpacking in tuple/list/set/dict comprehensions
by Erik Demaine Nov. 14, 2021

Nov. 14, 2021

Extended unpacking notation (* and **) from PEP 448 gives us great ways to concatenate a few iterables or dicts: ``` (*it1, *it2, *it3) # tuple with the concatenation of three iterables [*it1, *it2, *it3] # list with the concatenation of three iterables {*it1, *it2, *it3} # set with the union of three iterables {**dict1, **dict2, **dict3} # dict with the combination of three dicts # roughly equivalent to dict1 | dict2 | dict3 thanks to PEP 584 ``` I propose (not for the first time) that similarly concatenating an unknown number of iterables or dicts should be possible via comprehensions: ``` (*it for it in its) # tuple with the concatenation of iterables in 'its' [*it for it in its] # list with the concatenation of iterables in 'its' {*it for it in its} # set with the union of iterables in 'its' {**d for d in dicts} # dict with the combination of dicts in 'dicts' ``` The above is my attempt to argue that the proposed notation is natural: `[*it for it in its]` is exactly analogous to `[*its[0], *its[1], ..., *its[len(its)-1]]`. There are other ways to do this, of course: ``` [x for it in its for x in it] itertools.chain(*its) sum(it for it in its, []) functools.reduce(operator.concat, its, []) ``` But none are as concise and (to me, and hopefully others who understand * notation) as intuitive. For example, I recently wanted to write a recursion like so, which accumulated a set of results from within a tree structure: ``` def recurse(node): # base case omitted return {*recurse(child) for child in node.children} ``` In fact, I am teaching a class and just asked a question on a written exam for which several students wrote this exact code in their solution (which inspired writing this message). So I do think it's quite intuitive, even to those relatively new to Python. Now, on to previous proposals. I found this thread from 2016 (!); please let me know if there are others. https://mail.python.org/archives/list/python-ideas@python.org/thread/SBM3LY… There are several arguments for and against this feature in that thread. I'll try to summarize: Arguments for: * Natural extension to PEP 448 (it's mentioned as a variant within PEP 448) * Easy to implement: all that's needed in CPython is to *remove* some code blocking this. Arguments against: * Counterintuitive (to some) * Hard to teach * `[...x... for x in y]` is no longer morally equivalent to `answer = []; for x in y: answer.append(...x...)` (unless `list1.append(a, b)` were equivalent to `list1.extend([a, b])`) Above I've tried to counter the first two "against" arguments. Some counters to the third "against" argument: 1. `[*...x... for x in y]` is equivalent to `answer = []; for x in y: answer.extend(...x...)` (about as easy to teach, I'd say) 2. Maybe `list1.append(a, b)` should be equivalent to `list1.extend([a, b])`? It is in JavaScript (`Array.push`). And I don't see why one would expect it to append a tuple `(a, b)`; that's what `list1.append((a, b))` is for. I think the main argument against this is to avoid programming errors, which is fine, but I don't see why it should hold back an advanced feature involving both unpacking and comprehensions. Erik -- Erik Demaine | edemaine(a)mit.edu | http://erikdemaine.org/

15 30

[Python-Dev] Adding pep-8-casing-compliant aliases for unittest and logging
by Ethan Furman Nov. 13, 2021

Nov. 13, 2021

On 11/11/21 11:53 AM, Matt del Valle wrote: > Okay, so from the replies so far it looks like this is very quickly going into the 'never gonna happen' > dumpster, so in the interests of salvaging *something* out of it: [...] > I just dislike having to settle for 'it's what we've got'. With these two modules in particular, a lot > of the arguments that have been made so far either don't apply or are not as strong (such as the > confusion of having builtins.list, builtins.List and typing.List). Additionally, these are significantly > more ancillary portions of the stdlib than the builtins, much less likely to cause as severe of a > disruption (I personally don't believe a backward-compatible change like this which only adds aliases > would be as disruptive as many people claim, but concede that that's subjective), and much less likely > to have the implementation change so drastically as to want to change out types for factory functions > or vice-versa. > > So perhaps we could narrow the scope of this down to just adding snake_case aliases to the logging and > unittest modules [...] +1 I would happily change all my unittest and logging usage to snake case if those aliases existed. -- ~Ethan~

6 9