Mailman 3 January 2021 - Python-ideas

A PEP to encourage people to mind docstrings in stub files
by Alexey Volobuev 02 Jan '21

02 Jan '21

Hi! Sometimes you have to put docstrings in stub files e.g. when using code generation. In that case, stubs are the only place you can locate "objects" to add docstrings to. Thus it is frustrating to find that docstring-related software generally tends to ignore the presence of docstrings in stubs or to work incorrectly with those. I think we need a PEP to clarify the interaction between stubs and docstrings. This PEP might also encourage developers of docstring-related software to mind docstrings provided with stub files. -- Yours sincerely, Alexey Volobuev

8 9

Standard tool for iterating over recursive data structures?
by Steven D'Aprano 02 Jan '21

02 Jan '21

This isn't so much an idea for Python, as a request for ideas to solve a problem in Python. Back in the early days of Python, printing recursive lists could crash the interpreter: a = [1, 2, 3] a.append(a) print(a) If you try that today, you get a nice display: [1, 2, 3, [...]] It's not just lists that we can have this. Dicts and nearly any object can too: d = {} d[None] = d class Demo: pass x = Demo() x.attr = x With a bit of jiggery-pokery, even tuples can be recursive: t = (1, 2, 3, []) t[3].append(t) The built-ins handle these cases okay. Likewise the copy.deepcopy function takes care to avoid getting stuck in a loop with recursive data structures. On the other hand, when I write code to process nested data structures, I always worry about such recursive patterns, but actually doing something about it is too hard. So I just ignore the risk and feel guilty about it. I'm sure I'm not the only one who feels guilty about ignoring this problem. To say nothing of those who don't even know there's a problem to worry about. Is there something we can do to make it easier for people to deal with recursive data structures, like the list repr and deepcopy code does? Is there a pattern we can abstract out and provide as a tool or recipe for people to use in their own code? Relevant: https://bugs.python.org/issue42801 -- Steve

13 24

Right way to dataclass -> dict conversion.
by Anton Abrosimov 01 Jan '21

01 Jan '21

Example task: ``` from dataclasses import dataclass, asdict, fields from typing import List @dataclass class Point: x: int y: int @dataclass class Axes: title: str points: List[Point] def plot(title: str, points: List[Point]): print(title) for point in points: print(f'x={point.x}', f'y={point.y}') axes = Axes('example', [Point(i, i) for i in range(3)]) # Now I need to plot my axes. ``` Way 0: `plot(axes.title, axes.points)` Yes, it works great, but the topic about `dataclass2dict` conversion. Way 1: `plot(**asdict(axes))` Using `asdict` is what comes to mind first. But it is recursive, so it won't work. `asdict`, `astuple`... is an "bad naming" example, I think. Unexpected name for a recursive function. People don't read the documentation, and even when they do, they forget. As an example: https://stackoverflow.com/q/52229521/6146442 Viewed 7k times. Way 2: `plot(**vars(axes))` Everything seems to be good. But pseudo-fields which are `ClassVar` or `InitVar` will be returned. `asdict` use: `return tuple(f for f in fields.values() if f._field_type is _FIELD)` Way 3: Using undocumented `__dataclass_fields__` is look like a hack. Way 4: `plot(**{f.name: getattr(axes, f.name) for f in fields(axes)})` This will actually work. But I'm not sure about the elegance. My suggestions: 1. Add default `True` `recursive` flag to `asdict` and implement Way 4 internally. At the same time, IDE will remind people from stackoverflow that `asdict` is recursive. 2. Or add new not-recursive function to `dataclasses` module. Leave `asdict` frozen. 3. Do nothing. Way 4 is handsome enough. Describe the way in the documentation so that people don't get confused and don't use `vars` or `asdict`. I am ready to offer my assistance in implementation. Thank you for your attention and happy holidays!

3 7

Re: Standard tool for iterating over recursive data structures?
by Peter Ludemann 01 Jan '21

01 Jan '21

A variant of the problem is the "occurs check", used in theorem proving: https://en.wikipedia.org/wiki/Occurs_check Removing the occurs check reduces the cost of unifying two terms (t1, t2) from *O(size(t1) + size(t2))* to *O(min(size(t1), size(t2)))*, which is why Prolog implementations don't do the occurs check by default (but provide unify_with_occurs_check and acyclic_term predicates for when the check is needed). One use for cyclic terms is in representing grammars. Of course, the loops can be avoided by changing the representation from a graph to a BNF-style set of clauses - that is, by naming the nodes.

1 0

Standard tool for iterating over recursive data structures?
by Jonathan Fine 01 Jan '21

01 Jan '21

Hi Jeff You wrote: Detecting cycles will involve a lot of bikeshedding. (Sorry, couldn't > resist.) > Indeed. That really brought a smile to my face. Thank you. As I recall, bikeshedding is endless discussion (cycles again) of the COLOUR to paint the bikeshed. And in mathematics there's the topic of https://en.wikipedia.org/wiki/Graph_coloring. By the way, according to wikipedia, the convention of using colors originates from the coloring of the countries of a map, as in the famous https://en.wikipedia.org/wiki/Four_color_theorem. Once again, thank you for your careful attention and sense of humour. -- Jonathan

1 0

Option to not raise if file does not exists for os.remove()?
by Marco Sulla 01 Jan '21

01 Jan '21

What about a parameter, false by default, that suppresses the FileNotFoundError exception if true for os.remove()? Something similar to exist_ok for os.makedirs().

3 3

Re: Add user defined __eq__ to itertools.cycle class
by Andres Torres 01 Jan '21

01 Jan '21

Yep, thanks for the explanation! After some more research into this it appears this exact situation is accounted for within the `SQLAlchemy` library (reference: https://github.com/sqlalchemy/sqlalchemy/issues/5810)! Additionally, due to the complications brought up in this thread it seems like it's probably best to leave `__eq__` as the default implementation. Thanks for expanding my knowledge and Happy New Year! 🎊 Andres On Thu, Dec 31, 2020 at 4:40 PM Rob Cliffe <rob.cliffe(a)btinternet.com> wrote: > > > You bring up a good point regarding how to determine iterable equality. > For example, with your last example, I had no idea that the same iterable > behavior could produce radically different results between cyclers. To be > quite honest, I don't really understand what causing this quirky behavior. > Assuming that you refer to Steven D'Aprano's example: > > I didn't understand it either until I played with it a bit. If I > remember correctly it started something like this: > > it = iter([1,2,3]) > a = cycle(it) > b = cycle(it) > > Now cycle() reads elements from the iterator until it is exhausted, > meanwhile storing them. Then it cycles through the stored copies. The > problem is that a and b point to the *same* iterator and interfere with > one another. So: > > next(a) # gets 1 from 'it', stores 1, and returns 1 > next(b) # gets 2 from 'it', stores 2, and returns 2 > next(a) # gets 3 from 'it', stores 3, and returns 3 > > Now the iterator is exhausted. So subsequent calls of next(a) will > cycle through a's stored elements, viz. 1 and 3. Subsequent calls of > next (b) will repeat b's only stored element, viz 2. > > If instead we wrote: > > a = cycle(iter([1,2,3])) > b = cycle(iter([1,2,3])) > > then although a and b *don't* refer to the *same iterator*, they *will* > behave identically. > > It seems to me that this makes trying to define "equality" of cycles > problematic. > > Happy New Year > Rob Cliffe > >

1 1

built in to clear terminal
by Christopher Barker 01 Jan '21

01 Jan '21

Sorry -- I can't find the thread, but there was a discussion on this list about whether there should be a built in for clearing the terminal screen. I'm was on the fence about it -- but one more data point: One of my beginning Python students just submitted code with this in it: def clear_screen(): ''' Function: clear_screen() (Windows/Unix) Input: None Process: Clear Screen Output: None ''' if (opSys == 'Windows'): os.system('cls') else: os.system('clear') return None Which makes me think -- yes, it would be nice to have that available in the standard lib somewhere. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

13 37

Unpack operator "**" and Mapping
by Anton Abrosimov 01 Jan '21

01 Jan '21

I am trying to release comfortable dataclass unpacking using `**` operator. Now I have 5 different ways to do it. But not a single good one. Confused by the implementation of the unpacking operator. So when I try to unpack any custom class, I get the error: `type object argument after ** must be a mapping, not MyClass` Ok, nothing special. I need to use `collections.abc.Mapping` right? Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem. But additionally I get: `keys`, `items`, `values`. Hey, I don't need them. I don't need the full mapping functionality. I only need the double asterisk to work. Right, we have a duck typing! We throw out `abc.Mapping`. What do we need to implement? It's `__getitem__` and `keys`. Wtf `keys`? I am looking at Python Data model: https://docs.python.org/3/reference/datamodel.html There many operators, and they depend on special double underscore methods. Hmm, I don't see unpack operators there, it's strange. But why it's `keys`? Because the historical is `dict`? I think a dependency on `__iter__` is more preferable and expectable over a userspace named `keys`. Actually, `items()` is more predictable. But this is not the end. The `__getitem__` overload is often used for additional checking. I think `__iter__` and `keys` should only return a valid keys. Therefore, we don't need to further check them when unpacking. At the very least, we must control this. And in the end. `Mapping` keys can be `Any` type. `Unpack` keys must be `str` type. Some `Mapping` can be unpackable and some `Unpack` can be mappable. My suggestion: * Add new `collections.abc.Unpack` abstract layout for `**` unpack. * Add new special method like: def __unpack__(self): if issubclass(self, collections.abc.Mapping): # Really overload this method in `Mapping` and `dict`. keys = self.keys() # or return self.items()? else: keys = iter(self) return ((k, self[k]) for k in keys) * Update the implementation of the unpack operator to use the `__unpack__` function. As a result: * We can make the class unpackable without the advanced `Mapping` functionality. * We can control the unpacking process separately. * We throw away userspace named dependencies. * I think we are making behavior more predictable. What do you think about it?

11 58

Add asyncSetUpClass and asyncTearDown to IsolatedAsyncioTestCase
by Andres Torres 01 Jan '21

01 Jan '21

Title says it all. This might have been an oversight while making this class. If not please let me know why it was not included! Thank you for considering my request! Andres 🙃

2 2