Python-ideas April 2021

python-ideas@python.org

56 participants
27 discussions

: str() vs format(): trivia question
by Stephen J. Turnbull 22 Apr '21

22 Apr '21

@serhiy Moving my speculative discussion to python-ideas. CC as courtesy, comment from you is welcome but not necessarily expected. Serhiy Storchaka writes: > format() without format specifier and str() should return the same > value in general, otherwise it will confuse users. I think this is a good rule of thumb because it's a very plausible default for format(), and easy to explain. But then what is the purpose of format()? Why not just give str() the functionality of format(), and deprecate format() for future use? Or vice versa, format() might be the better name, but the backward compatibility implications given the pervasive use of str() would be awesome, and not in a good way. Given how pervasive both str and format are, and the fact that str also has both .format() and .__format__ methods, I guess I'm asking for a lot of trouble. But this plethora of approaches to providing a string presentation of an object seems designed to confuse users, and it's not clear to me that trying to maintain str(thing) == format(thing) necessary helps dissolve that confusion. Steve

3 2

allow initial comma
by roland.puntaier＠chello.at 16 Apr '21

16 Apr '21

I had posted this as https://github.com/python/peps/issues/1867 The discussion so far is below. Please make some arguments. The major point to me is, that the symmetry is broken, which leads to extra editing actions, like removing the comma in the first line. I guess, this was the reason to allow the comma after the last line/entry: `[1,2,]`. ``[,1,2]`` should also be allowed, too. The best argument is one that says: less or more effort in this or that situation. For example, with `[1,2,]`, in line-wise formatting, one can do without special action at the last line (removing the comma there). All code from previous versions of Python would still work after a `[,1,2]` syntax allowance were introduced. ================================================================================= rpuntaie wrote: ================================================================================= Allow initial comma =================== Final comma works: t = ( 1, 2, ) x = [ 1, 2, ] y = { 1, 2, } z = { 1:11, 2:22, } def fun( a, b, ): pass Initial comma does not work: t = ( , 1 , 2 ) x = [ , 1 , 2 ] y = { , 1 , 2 } z = { , 1:11 , 2:22 } def fun( , a , b ): pass To make the syntax symmetric in this regard\ gives more freedom to format the code. I occasionally found the restriction an unnecessary nuisance. Before writing a PEP, I would like to discuss, - whether something like that has been proposed already? - what counter-arguments there could be? ================================================================================= pxeger wrote: ================================================================================= This is not the appropriate place to propose language changes. Try the [python-ideas](https://mail.python.org/mailman3/lists/python-ideas.python.or… mailing list. However, I don't think you'll get anyone to agree. What kind of code style are you using where you want to put commas at the start of the line? That is totally non-standard (see [PEP 8](https://www.python.org/dev/peps/pep-0008)), ugly, and confusing. Arbitrary symmetry is not a good reason for changing the language. We don't have a `tnirp` function just for the sake of symmetry with `print` because it would be pointless and add extra complication ================================================================================= rpuntaie wrote: ================================================================================= I surely agree, that not ignoring the sequence is essential. Else one would loose identifier space and thus information. I would never have the idea to make all permutations of `p.r.i.n.t` point to the same function. Therefore you just made a bad example. But the comma is just a separator. Why did they allow to have the comma before a closing bracket/parenthesis/brace? Because of symmetry between lines, is my guess. Occasionally one sees many spaces just the have the final comma aligned vertically. That could be avoided by placing the comma at the beginning. I personally also have a macro in the editor that evaluates a line in the parameter list, but drops an initial comma before doing that. Therefore this is my preferred formatting. I don't think that [PEP](https://www.python.org/dev/peps/pep-0008) is wrong. I just don't want to be restricted by unnecessary rules. Rules need to have a reason beyond someone dictating them. If that is the case, I follow them, because I see the reason, but not because someone dictates them. I'll go to [Python Ide as](https://mail.python.org/mailman3/lists/python-ideas.python.org/), then. Thanks.

19 27

Iterable scalar values returning itself ones?
by Hans Ginzel 16 Apr '21

16 Apr '21

Are there any reasons not to make scalar types iterable returning the value ones? Should each function check if it has got one value or a list/tuple before iteration over the argument? What is wrong on scalars for iteration, please? There is even legal to iterate over an empty set – an empty cycle. Python 3.8.5 (default, Jan 27 2021, 15:41:15) >>> def chain(*iterables): ... for iterable in iterables: ... yield from iterable ... >>> a = 5 >>> b = range(3) >>> for i in chain(a, b): ... print(i) ... Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in chain TypeError: 'int' object is not iterable HG

8 10

Top Level Await in Python like in Deno
by redradist＠gmail.com 15 Apr '21

15 Apr '21

Hi all, Deno is JavaScript runtime that has very nice feature like Top Level Await, I think it would be also nice to have such feature in Python, it will make using async/await more convenient What do you think ? Share your ideas lets discuss ...

5 8

Why not deprecate Py_XDECREF in a future python version? Maybe a DeprecationWarning for now?
by Shreyan Avigyan 15 Apr '21

15 Apr '21

After going through the https://github.com/python/cpython/blob/master/Include/object.h it seems that Py_XDECREF is just calling Py_DECREF if the PyObject pointer is not NULL. This if statement could be directly implemented in Py_DECREF right? Therefore is it really required to have Py_XDECREF? Why not use a DeprecationWarning for now and plan to deprecate Py_XDECREF in the near future? How about implementing an if statement in Py_DECREF to make it work like Py_XDECREF? Thanking you, With Regards

3 3

Revive: PEP 396 -- Module Version Numbers ?
by Christopher Barker 14 Apr '21

14 Apr '21

Over the years, I've seen __version__ used very broadly but not *quite* in all packages. I've always known it was a convention, not a requirement. But it turns out it's not even a "official" convention. But it really would be nice if there was a consistent way that I could count on to get the version of a package at run time. Turns out this was suggested in PEP 396 -- and deferred almost 13 years ago! https://www.python.org/dev/peps/pep-0396/ In the status note, it says: """ Further exploration of the concepts covered in this PEP has been deferred for lack of a current champion interested in promoting the goals of the PEP and collecting and incorporating feedback, and with sufficient available time to do so effectively. """ Well, I may be willing to be that champion, if a core dev is willing to sponsor, and I see some interest from this group. And, well, after 13 years, we've seen __version__ be very broadly, though certainly not universally used. Honestly, I haven't looked to see to what extent setuptools supports it, but will, of course, do so if folks think this is worth pursuing. Or maybe this is a settled issue in setuptools, and we just need to change the status of the PEP. For my part, I FAR prefer the version info to be embedded in the code of the package in a simple way, rather than hiding among the setuptools/egg/pip metadata, and requiring setuptools.pkg_resources to get a version at runtime. I note that PEP8 uses __version__ as an example of a "module level dunder" -- but only suggests where it should be put, not how it be used :-) Of course, there will be a need to update the PEP to match current practice (and setuptools and pip) Or should this be brought up on The Distutils-SIG list? -CHB ,-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

4 8

Add an export keyword to better manage __all__
by Theia Vogel 13 Apr '21

13 Apr '21

Hi, I was refactoring some code today and ran into an issue that always bugs me with Python modules. It bugged me enough this time that I spent an hour banging out this potential proposal to add a new contextual keyword. Let me know what you think! Theia -------------------------------------------------------------------------------- A typical pattern for a python module is to have an __init__.py that looks something like: from .foo import ( A, B, C, ) from .bar import ( D, E, ) def baz(): pass __all__ = [ "A", "B", "C", "D", "E", "baz", ] This is annoying for a few reasons: 1. It requires name duplication a. It's easy for the top-level imports to get out of sync with __all__, meaning that __all__, instead of being useful for documentation, is actively misleading b. This encourages people to do `from .bar import *`, which screws up many linting tools like flake8, since they can't introspect the names, and also potentially allows definitions that have been deleted to accidentally persist in __all__. 2. Many symbol-renaming tools won't pick up on the names in __all__, as they're strings. Prior art: ================================================================================ # Rust Rust distinguishes between "use", which is a private import, "pub use", which is a globally public import, and "pub(crate) use", which is a library-internal import ("crate" is Rust's word for library) # Javascript In Javascript modules, there's an "export" keyword: export function foo() { ... } And there's a pattern called the "barrel export" that looks similar to a Python import, but additionally exports the imported names: export * from "./foo"; // re-exports all of foo's definitions Additionally, a module can be gathered and exported by name, but not in one line: import * as foo from "./foo"; export { foo }; # Python decorators People have written utility Python decorators that allow exporting a single function, such as this SO answer: https://stackoverflow.com/a/35710527/1159735 import sys def export(fn): mod = sys.modules[fn.__module__] if hasattr(mod, '__all__'): mod.__all__.append(fn.__name__) else: mod.__all__ = [fn.__name__] return fn , which allows you to write: @export def foo(): pass # __all__ == ["foo"] , but this doesn't allow re-exporting imported values. # Python implicit behavior Python already has a rule that, if __all__ isn't declared, all non-underscore-prefixed names are automatically exported. This is /ok/, but it's not very explicit (Zen) -- it's easy to accidentally "import sys" instead of "import sys as _sys" -- it makes doing the wrong thing the default state. Proposal: ================================================================================ Add a contextual keyword "export" that has meaning in three places: 1. Preceding an "import" statement, which directs all names imported by that statement to be added to __all__: import sys export import .foo export import ( A, B, C, D ) from .bar # __all__ == ["foo", "A", "B", "C", "D"] 2. Preceding a "def", "async def", or "class" keyword, directing that function or class's name to be added to __all__: def private(): pass export def foo(): pass export async def async_foo(): pass export class Foo: pass # __all__ == ["foo", "async_foo", "Foo"] 3. Preceding a bare name at top-level, directing that name to be added to __all__: x = 1 y = 2 export y # __all__ == ["y"] # Big Caveat For this scheme to work, __all__ needs to not be auto-populated with names. While the behavior is possibly suprising, I think the best way to handle this is to have __all__ not auto-populate if an "export" keyword appears in the file. While this is somewhat-implicit behavior, it seems reasonable to me to expect that if a user uses "export", they are opting in to the new way of managing __all__. Likewise, I think manually assigning __all__ when using "export" should raise an error, as it would overwrite all previous exports and be very confusing.

15 28

Python Idea - extension of 'with'
by anthony.flury 10 Apr '21

10 Apr '21

We are all familiar with the `with` statement for context managers, so that the resources are correctly finalized/closed in case of an exception either in terms of the resources being allocated of the processing. It is very common though to wrap the `with block` in an outer `try` block, since although we know that the resource has been closed, at an 'application' level it is still neccessary to deal with the exception - an example might be : try: with open('config.cfg', 'r') as cfg: # Process the open file config = load_config(cfg) except FileNotFound: logging.info('Config file not found - using default configuration') except PermissionError: logging.warning('Cannot open config .cfg - using default configuration') config = default_config() else: logging.info('Using config from config.cfg') It struck me that this is probably quite a common idiom, and we have the main processing (of the opened resources) indented twice just to accommodate the outer try block. I was wondering whether a worthwhile extension might be to allow the `with` statement to have an `except` and `else` clauses which would have the same semantics as wrapping the `with` block with a try - for example the above would now look like: with open('config.cfg', 'r') as cfg: # Process the open file config = load_config(cfg) except FileNotFound: logging.info('Config file not found - using default configuration') except PermissionError: logging.warning('Cannot open config .cfg - using default configuration') config = default_config() else: logging.info('Using config from config.cfg') Treating the 'with' as an implied `try` would reduce the march to the right - now the key processing of the resource is now indented only one level - and the association of the exception from the `with` block is syntactically clear. I am not good enough with core development to put together a working prototype, but I can imagine that this simple extension would be worth while, but I would be more than happy to author a PEP for this if it gets some initial positive feedback. Open questions - that I have - should we allow `except`, `else` and `finally` clauses (or just `except` and `else` - do we need `finally` here). -- Anthony Flury *Moble*: +44 07743 282707 *Home*: +44 (0)1206 391294 *email*: anthony.flury(a)btinternet.com <mailto:anthony.flury@btinternet.com>

11 18

Adding syntax for the empty set
by ucodery＠gmail.com 10 Apr '21

10 Apr '21

Hello, I would like to propose adding literal syntax to allow creation of an empty set without the need to call the type constructor. I believe the best choice for such a literal, and one that has been proposed before, is `{,}`. I know this idea has surfaced a few times before, but I cannot find any serious consideration since the 3K ideas chain with the closest to a pronouncement here https://mail.python.org/pipermail/python-3000/2006-May/001599.html. In that thread the idea of a new syntax for the empty set did not seem to be rejected outright, but more that the idea of set literals at all was still in question and as empty sets were not implemented for 2.7 they were ultimately still left out of 3k, intentionally or unintentionally. Within cpython itself `set()` is used just over 200 times to assign a variable the value of an empty set. As a comparison `dict()` is used only 15 times, and all but one of those are within the test module, mostly for testing dict. On the other hand, `{}` is used for assignment a little over 1300 times; I think there is a clear preference for using literal over type construction when possible. I have a working implementation of this new syntax at https://github.com/ucodery/cpython/tree/empty-set which includes a few changes. Python from this branch allows `{,} == set()` but also `[,] == list()` and `(,) == tuple`. Partly this was because the grammar changes were easier to do all three but also because I personally liked keeping some symmetry between these collection literals. The main point of starting this thread is to gather opinions on adding `{,}` to the language, but I would like to discuss the other two as well. As a secondary point, I do think that this new tuple syntax would make teaching python easier. Those new to python often believe that parentheses are what make a tuple and only later, if ever, realize that it is only value(s) followed by a comma that construct a tuple and that parentheses are places around it mainly for readability. This misunderstanding often leads to a bug like `iter((4))`, assuming the programmer meant `iter((4,))`. I believe this confusion about whether it is the parentheses or the commas that construct a tuple is deepened because two parentheses surrounding nothing _does_ specifically create the empty tuple but `empty = ,` is a SyntaxError. With this literal tuple change, those new to the language can be told that parentheses containing at least one comma always creates a tuple (at least anywhere is is grammatically valid to have a tuple). Jeremy

14 20

Re: Non-blocking concurrent.futures ThreadPoolExecutor
by Brunoais 10 Apr '21

10 Apr '21

(I know this has some time but I think it makes sense in context) That is much easier said than done in real life when the whole project is doing by a team and you are stuck with whatever you all do in the group. Even then, I prefer making my own programs with a mix of daemon and non-daemon threads to do the word. Daemon threads do work that can be interrupted at any time for any reason. Non-daemon threads are more well behaved and focus on doing their short job with hard IO. In my case, the ThreadPoolExecutor does most of the work for me for these cases. Unfortunately, I think I'll have to implement my own for daemon threads.

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas April 2021