Python-ideas June 2021

python-ideas@python.org

75 participants
32 discussions

Subject=Re: Re: concurrent.futures.ProcessPoolExecutor(restart_workers=True)
by Ram Rachum 26 Jun '21

26 Jun '21

Thank you, that's helpful. Though I really like the future interface, I've used it many times while I never used `multiprocessing.pool.Pool` or its `AsyncResult`. I worry that some of the expectations I've grown to have from `Future` objects would be broken here. It would be nice if this feature existed in `concurrent.futures`. Thanks, Ram.

1 0

Adding a `wait_in_order` function to concurrent.futures
by santiago.basulto＠gmail.com 26 Jun '21

26 Jun '21

Hello list, it's my first email here, so let me know if this is in any way out of order. I've been working a lot with concurrent.futures lately. I find both `wait` and `as_completed` very useful, but something I'm usually lacking is a `wait` version where the order of the futures is preserved. I quickly hacked a proof of concept to discuss with you and see if anybody else would find it useful: https://gist.github.com/santiagobasulto/10b689ba5fcadf307ffc5cd4f4ae00ec I also realized that the implementation of `ThreadPoolExecutor.map` could be greatly simplified with this function as part of the stdlib. What do you think? Thanks!

2 3

Extension methods in Python
by Johan Vergeer 25 Jun '21

25 Jun '21

After working with Kotlin for a while I really started to like the idea of extension methods. I delivers a great way to extend the language without having to add features to the language itself. As a practical example I would like to take a first item from a list my_list = [] first = my_list[0] # This will obviously throw an error In this case it would be great to create an extension method like def list.first_or_none(self): if len(self): return self[0] return None Or ... def list.first_or_else(self, fallback_value): if len(self): return self[0] return fallback_value Then, we could retrieve a value from the list: from my_list_extensions import first_or_none, first_or_else first = my_list.first_or_none() Or ... first = my_list.first_or_else(0) Some of the main advantages (not an exhaustive list): - It also provides a way to add extension methods before adding methods to the standard library, like in PEP 616 - Classes can be extended without using inheritance - Easier for IDE's to know what methods are available for a type, therefore delivering auto-completion. - It would also become easier to extend classes from other libraries next to the standard library - It will be much easier to create methods on a class that can be chained, which leads to less intermediate variables and (in my opinion) better readability. Sometimes these are called fluent builders or fluent interfaces. my_list.filter(lambda x: x >= 0).map(lambda x: str(x)) instead of [str(x) for x in my_list if x >= 0]. (ok, in hindsight this looks better with the list comprehension, but there are much more complicated examples that would benefit) A more comprehensive example would be an html builder. Let's say I created a library that builds html from python code, with a nice builder interface: class Html: def __init__(self): self.__items = [] def add_p(self, text): self.__items.append(text) return self def add_h1(self, text): self.__items.append(text) return self def add_div(self, items): self.__items.append(items) return self To add methods (which is very likely in a class like this), I would have to create a subclass, or create a pull request with the author (if the author approves). In all languages that I know of that have extension methods the most given argument is better code readability.

17 77

Fill in missing contextvars/asyncio task support
by Mark Gordon 23 Jun '21

23 Jun '21

With normal synchronous code you can use `contextvars.Context.run()` to change what context code is executing within. However, there is no analagous concept for asyncio code. I'm proposing something similar, for example: coro = foo() my_context = convextvars.Context() await asyncio.run_in_context(coro) Currently the workaround is to run the coroutine on a separate task. coro = foo() my_context = convextvars.Context() await my_context.run(asycnio.create_task, coro) However this is not exactlly the same as the task will inherit a copy of my_context rather than running directly on my_context. Additionally (obviously) it will also be running in a separate task. Similarly it would be nice if create_task and the Task constructor could take an optional context kwarg to use as the task context rather than the default of copying the calling context. Pull request with sample implementation (although I think missing the change to create_task): https://github.com/python/cpython/pull/26664

4 9

Off topic: Rational discursive thought and helping others
by Jonathan Fine 22 Jun '21

22 Jun '21

Off-topic Someone on this list wrote: Mu. > > https://en.wikipedia.org/wiki/Mu_(negative)#%22Unasking%22_the_question This is a koan, a device in Zen Buddhism used by a teacher to help the student liberate themselves from being imprisoned by rational discursive thought. Here is another koan. A Zen student told Ummon: "Brilliancy of Buddha illuminates the whole universe." Before he finished the phrase Ummon asked: "You are reciting another's poem, are you not?" "Yes," answered the student. "You are sidetracked," said Ummon. Here's my commentary. The teacher neither agrees or disagrees with the student's statement. This is "Mu" if you wish. Instead the teacher directly observes the student's mental processes, confirms the observation, and provides helpful advice. The teacher is not trapped by rational discursive thought. Here the teacher uses only 12 words. Sometimes a teacher uses a single word, or no words at all. You'll find this koan and further commentary at https://www.sacred-texts.com/bud/glg/glg39.htm. -- Jonathan

1 0

Deprecate sum of lists
by Oliver Margetts 22 Jun '21

22 Jun '21

Hi all, I came across the following performance issue with the sum function while summing lists: https://bugs.python.org/issue18305 It's been discussed previously in this list and other issues. The rationale in that ticket makes the case that performance won't be fixed because sum() should not be treated as a generic way to concatenate sequences (and there is documentation saying this). Having non-linear complexity is not a suitable way to discourage this behaviour though. Non-linearity is hard to detect in testing and development, so is more likely to lead to performance bugs in libraries and production, than to discourage use from the outset. Moreover, it is confusing to end users to have a feature while discouraging its use through a somewhat passive-aggressive runtime. New users, and experienced users who do not pour through the documentation will use sum because it supports this API, and one would assume that's intentional. On the other hand, the behaviour for strings is a far superior user-experience. When attempting to use sum() on strings, users are presented with a TypeError and a handy tip (use "".join(seq) instead). It is clear to users that they are attempting an anti-pattern and how to correct it. So I propose moving (eventually) to the same system with lists as for strings. Or else removing the performance trip-hazard from the sum function with lists. Best regards, Oliver

21 63

Allow modifying message of TypeError from NotImplemented
by wyz23x2＠163.com 19 Jun '21

19 Jun '21

Python raises TypeError when NotImplemented is returned from __add__, __invert__ etc. and __radd__ etc. aren't available. However, this disallows the customization of error messages. For example, in Python 3.8, __float__ etc. were removed from complex to allow methods like __rfloat__. But this makes abs(z) undiscoverable for many users. The original error message is very helpful. I suggest to make a way for this usage. Maybe NotImplementedType can accept *args, and NotImplemented can be callable, equal to __init__: >>> class A: def __add__(self, other): return NotImplemented # Just like before def __neg__(self, other): return NotImplemented(f'bad operand type for unary -: {type(self).__name__!r}; use ~ to invert') # <-- def __invert__(self): return ~5 >>> a = A() >>> a + 2 TypeError: unsupported operand type(s) for +: 'A' and 'int' >>> -a TypeError: bad operand type for unary -: 'A'; use ~ to invert >>> ~a -6

4 3

channel or message queue in asyncio
by zhongquan789＠gmail.com 19 Jun '21

19 Jun '21

Functionalities are similar to Go channel but with topics or consumers like a message queue. Possible syntax: ```python async def main(): ch = Channel() await ch.put(1) # put await ch.get() # get await ch.close() # close c1 = ch.consumer() c2 = ch.consumer() async for data in c1: # two consumer are indpendent ... async for data in c2: # async for syntax ... ``` Related project: https://github.com/Drakkar-Software/Async-Channel https://github.com/tbug/aiochannel https://github.com/ebb29c/asyncio-channel https://github.com/navytux/pygolang Though it can be implemented by using asyncio.Queue，If this can be added into the asyncio library?

2 1

"except;" - semicolon after except, to get rid of indentation when doing error recovery
by Soni L. 17 Jun '21

17 Jun '21

Sometimes it would be useful to be able to write: def foo(): try: return thing() except ValueError; try: return otherthing() except ValueError; try: return yetotherthing() except ValueError; if shouldraise(): raise But currently this needs to be written like so: def foo(): try: return thing() except ValueError: try: return otherthing() except ValueError: try: return yetotherthing() except ValueError: if shouldraise(): raise Look at all that unnecessary indentation! Would be nice to get rid of it.

5 8

Introduce constant variables in Python
by Shreyan Avigyan 17 Jun '21

17 Jun '21

Many times a programmer would want a variable that will be a constant. The programmer could have been careless and suddenly changed a very important variable. Proposed syntax, constant variable = 10 variable = 20 # Error Now constant means making a variable value immutable. It means now we can neither change the variable to point to another value nor the value itself anymore. For example, if we have a list we cannot change it. If we try to then "Error". So for lists, constant variable = ["List"] variable.append("Some") # Error variable = ["Another"] # Error Usually if we have a constant we don't create another reference to it. So do not support two references. Why not? Many reasons, especially deallocation problems. Thanking you, With Regards

10 16

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas June 2021