Python-ideas December 2020

python-ideas@python.org

81 participants
59 discussions

by Todd

I know enhancements to pathlib gets brought up occasionally, but it doesn't look like anyone has been willing to take the initiative and see things through to completion. I am willing to keep the ball rolling here and even implement these myself. I have some suggestions and I would like to discuss them. I don't think any of them are significant enough to require a pep. These can be split it into independent threads if anyone prefers. 1. copy The big one people keep bringing up that I strongly agree on is a "copy" method. This is really the only common file manipulation task that currently isn't possible. You can make files, read them, move them, delete them, create directories, even do less common operations like change owners or create symlinks or hard links. A common objection is that pathlib doesn't work on multiple paths. But that isn't the case. There are a ton of methods that do that, including: * symlink_to * link_to * rename * replace * glob * rglob * iterdir * is_relative_to * relative_to * samefile I think this is really the only common file operation that someone would need to switch to a different module to do, and it seems pretty strange to me to be able to make symbolic or hard links to a file but not straight up copy one. 2. recursive remove This could be a "recursive" option to "rmdir" or a "rmtree" method (I prefer the option). The main reason for this is symmetry. It is possible to create a tree of folders (using "mkdir(parents=True)"), but once you do that you cannot remove it again in a straightforward way. 3. newLine for write_text This is the only relevant option that "Path.open" has but "Path.write_text" doesn't, and is a serious omission when dealing with multiple operating systems. 4. uid and gid You can get the owner and group name of a file (with the "owner" and "group" methods), but there is no easy way to get the corresponding number. 5. Stem with no suffixes The stem property only takes off the last suffix, but even in the example given ('my/library.tar.gz') it isn't really useful because the suffix has two parts ('.tar' and '.gz'). I suggest another property, probably called "rootstem" or "basestem", that takes off all the suffixes, using the same logic as the "suffixes" property. This is another symmetry issue: it is possible to extract all the suffixes, but not remove them. 6. with_suffixes Equivalent to with_suffix, but replacing all suffixes. Again, this is a symmetry issue. It is hard to manipulate all the suffixes right now, as the example show. You can add them or extract them, but not change them without doing several steps. 7. exist_ok for is_* methods Currently all the is_* methods (such as is_file) return False if the file doesn't exist or if it is a broken symlink. This can be dangerous, since it is not trivially easy to tell if you are dealing with the wrong type of file vs. a missing file. And it isn't obvious behavior just from the method name. I suggest adding an "exist_ok" argument to all of these, with the default being "True" for backwards-compatibility. This argument name is already in use elsewhere in pathlib. If this is False and the file is not present, a "FileNotFoundError" is raised.

1 day, 5 hours

Add command-line option to unittest for enabling post-mortem debugging

by Dominik Vilsmeier

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-mortem debugging. `--failfast` doesn't help, the exception is still swallowed. Since I am not aware of a solution that enables post-mortem debugging in such a case (without modifying the test scripts, please correct me if one exists), I propose adding a command-line option to `unittest` for [running test cases in debug mode](https://docs.python.org/3/library/unittest.html#unittest.TestCase.deb… so that post-mortem debugging can be used. P.S.: There is also [this SO question](https://stackoverflow.com/q/4398967/3767239) on a similar topic.

1 week, 3 days

Python JIT Compilation Thoughts

by redradist＠gmail.com

Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!

2 weeks

A PEP to encourage people to mind docstrings in stub files

by Alexey Volobuev

Hi! Sometimes you have to put docstrings in stub files e.g. when using code generation. In that case, stubs are the only place you can locate "objects" to add docstrings to. Thus it is frustrating to find that docstring-related software generally tends to ignore the presence of docstrings in stubs or to work incorrectly with those. I think we need a PEP to clarify the interaction between stubs and docstrings. This PEP might also encourage developers of docstring-related software to mind docstrings provided with stub files. -- Yours sincerely, Alexey Volobuev

2 weeks, 3 days

Right way to dataclass -> dict conversion.

by Anton Abrosimov

Example task: ``` from dataclasses import dataclass, asdict, fields from typing import List @dataclass class Point: x: int y: int @dataclass class Axes: title: str points: List[Point] def plot(title: str, points: List[Point]): print(title) for point in points: print(f'x={point.x}', f'y={point.y}') axes = Axes('example', [Point(i, i) for i in range(3)]) # Now I need to plot my axes. ``` Way 0: `plot(axes.title, axes.points)` Yes, it works great, but the topic about `dataclass2dict` conversion. Way 1: `plot(**asdict(axes))` Using `asdict` is what comes to mind first. But it is recursive, so it won't work. `asdict`, `astuple`... is an "bad naming" example, I think. Unexpected name for a recursive function. People don't read the documentation, and even when they do, they forget. As an example: https://stackoverflow.com/q/52229521/6146442 Viewed 7k times. Way 2: `plot(**vars(axes))` Everything seems to be good. But pseudo-fields which are `ClassVar` or `InitVar` will be returned. `asdict` use: `return tuple(f for f in fields.values() if f._field_type is _FIELD)` Way 3: Using undocumented `__dataclass_fields__` is look like a hack. Way 4: `plot(**{f.name: getattr(axes, f.name) for f in fields(axes)})` This will actually work. But I'm not sure about the elegance. My suggestions: 1. Add default `True` `recursive` flag to `asdict` and implement Way 4 internally. At the same time, IDE will remind people from stackoverflow that `asdict` is recursive. 2. Or add new not-recursive function to `dataclasses` module. Leave `asdict` frozen. 3. Do nothing. Way 4 is handsome enough. Describe the way in the documentation so that people don't get confused and don't use `vars` or `asdict`. I am ready to offer my assistance in implementation. Thank you for your attention and happy holidays!

2 weeks, 4 days

Option to not raise if file does not exists for os.remove()?

by Marco Sulla

What about a parameter, false by default, that suppresses the FileNotFoundError exception if true for os.remove()? Something similar to exist_ok for os.makedirs().

2 weeks, 4 days

built in to clear terminal

by Christopher Barker

Sorry -- I can't find the thread, but there was a discussion on this list about whether there should be a built in for clearing the terminal screen. I'm was on the fence about it -- but one more data point: One of my beginning Python students just submitted code with this in it: def clear_screen(): ''' Function: clear_screen() (Windows/Unix) Input: None Process: Clear Screen Output: None ''' if (opSys == 'Windows'): os.system('cls') else: os.system('clear') return None Which makes me think -- yes, it would be nice to have that available in the standard lib somewhere. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

2 weeks, 5 days

Unpack operator "**" and Mapping

by Anton Abrosimov

I am trying to release comfortable dataclass unpacking using `**` operator. Now I have 5 different ways to do it. But not a single good one. Confused by the implementation of the unpacking operator. So when I try to unpack any custom class, I get the error: `type object argument after ** must be a mapping, not MyClass` Ok, nothing special. I need to use `collections.abc.Mapping` right? Now I need to implement: `__getitem__`, `__iter__`, `__len__`. Not a problem. But additionally I get: `keys`, `items`, `values`. Hey, I don't need them. I don't need the full mapping functionality. I only need the double asterisk to work. Right, we have a duck typing! We throw out `abc.Mapping`. What do we need to implement? It's `__getitem__` and `keys`. Wtf `keys`? I am looking at Python Data model: https://docs.python.org/3/reference/datamodel.html There many operators, and they depend on special double underscore methods. Hmm, I don't see unpack operators there, it's strange. But why it's `keys`? Because the historical is `dict`? I think a dependency on `__iter__` is more preferable and expectable over a userspace named `keys`. Actually, `items()` is more predictable. But this is not the end. The `__getitem__` overload is often used for additional checking. I think `__iter__` and `keys` should only return a valid keys. Therefore, we don't need to further check them when unpacking. At the very least, we must control this. And in the end. `Mapping` keys can be `Any` type. `Unpack` keys must be `str` type. Some `Mapping` can be unpackable and some `Unpack` can be mappable. My suggestion: * Add new `collections.abc.Unpack` abstract layout for `**` unpack. * Add new special method like: def __unpack__(self): if issubclass(self, collections.abc.Mapping): # Really overload this method in `Mapping` and `dict`. keys = self.keys() # or return self.items()? else: keys = iter(self) return ((k, self[k]) for k in keys) * Update the implementation of the unpack operator to use the `__unpack__` function. As a result: * We can make the class unpackable without the advanced `Mapping` functionality. * We can control the unpacking process separately. * We throw away userspace named dependencies. * I think we are making behavior more predictable. What do you think about it?

2 weeks, 5 days

Add asyncSetUpClass and asyncTearDown to IsolatedAsyncioTestCase

by Andres Torres

Title says it all. This might have been an oversight while making this class. If not please let me know why it was not included! Thank you for considering my request! Andres 🙃

2 weeks, 5 days

Add user defined __eq__ to itertools.cycle class

by Andres Torres

As the title suggests, I am requesting an user defined __eq__ method for the itertools.cycle class. A suggestion for how equivalency might work is that __eq__ would return True if two cyclers have the same iterable and are at the same location in their cycle and False in any other condition. Thank you for considering my suggestion, Andres

2 weeks, 5 days

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas December 2020