Python-ideas October 2020

python-ideas@python.org

94 participants
50 discussions

I frequently find myself in the following situations
by 71f13rvj＠powerencry.com 20 Oct '20

20 Oct '20

Hi Ideas, I frequently find myself in the following situations: 1) I wish to do something if no exception is thrown, for instance: try: logger.debug('Fiber starting: %s', self) try: self._result = self._routine() finally: logger.debug('Fiber finished: %s', self) except: raise else: raise FiberExit(self) finally: self._finished.set() unregister_fiber(self) Here it's sufficient that that if an exception is already present, I don't need to raise another. The except clause is clearly pointless. 2) I'm experimenting with catching various exceptions and remove the last except clause. I need to put a finally: pass; to avoid having to restructure all my code, since this is currently the only way to maintain the try-except-else-finally statement without catching dummy exceptions. I propose that the except clause be optional. Cheers, Matt https://www.assortlist.com

2 1

New feature
by ankith abhayan 18 Oct '20

18 Oct '20

Hi, I would like to request a new feature that allows you to clear the console screen. Like in c++, the CLS function Thanks

22 56

option to keep TemporaryDirectory
by David Kolovratník 16 Oct '20

16 Oct '20

Hi Everyone, When adding debug option to a program I find useful to keep temporary stuff. tempfile.NamedTemporaryFile has option delete defaulting to True. What about adding such option to tempfile.TemporaryDirectory as well? It would be just a minor modification in Lib/tempfile.py. What do you think? Kind regards, David Kolovratník

1 0

Weakrefs for lru_cache?
by Ram Rachum 16 Oct '20

16 Oct '20

Hi everyone, For many years, I've used a `cache` decorator that I built <https://github.com/cool-RR/python_toolbox/blob/master/python_toolbox/cachin…> for caching Python functions. Then `functools.lru_cache` was implemented, which is much more standard. However, as far as I know, it holds normal references to its keys, rather than weak references. This means that it can cause memory leaks when it's storing items that don't have any references elsewhere. This often makes me reluctant to use it. What do you think about supporting weakrefs in for keys lru_cache? If I remember correctly, the main difficulty was that not all keys are of a type that can be weakreffed. If I remember correctly again, I've solved this by including logic that attempts a weakref when possible, and degrades to a strong ref. What do you think about that? Thanks, Ram.

9 20

except-try block
by haael 16 Oct '20

16 Oct '20

Very often I use the following construction (especially in operators): try: something except SomeError: try: something_else except AnotherError: try: something_completely_different except Whatever: return NotImplemented I propose a construction to simplify the staircase hierarchy of exception handlers: try: something except SomeError try: something_else except AnotherError try: something_completely_different except Whatever: return NotImplemented This would be somewhat analogous to `else if` blocks. This pattern is very common in operator implementations. And this proposal could simplify it a bit. haael

8 16

Fwd: Re: Experimenting with dict performance, and an immutable dict
by Marco Sulla 15 Oct '20

15 Oct '20

On Wed, 22 Jul 2020 at 15:35, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > The deltas also look small for a micro-benchmark. I certainly don't > think this is a sufficient reason to add a new datatype to Python. > I think some of the optimizations can be experimented with dict itself. Deltas seem to be small, because I divide them by the number of timeit runs. I think that a 30-45% gain in constructor speed is not something small. The benchmarks in question have the Name that starts with "constructor". PS: sorry for double mail, I sent this message to only you by mistake. On Wed, 22 Jul 2020 at 18:26, Guido van Rossum <guido(a)python.org> wrote: > Did you study PEP 416 (frozendict) and PEP 603 (frozenmap)? > Yes. About frozenmap, at the very start I added to the benchmarks also immutables.Map, but I removed it when I realized that it was slower than frozendict in every bench. Maybe this is because immutables.Map is a C extension and not a builtin type. About PEP 416, yes, I tried to follow it. Indeed hash is calculated using the strategy described in the PEP. I also take a look to frozenset and tuple code. Frankly I must admit that the rejection of PEP 416 was a good idea. Indeed I started to implement an immutable dict only for a symmetry reason, and because I am somewhat fascinated by functional programming and immutables, without a rational reason. I suppose I would have given up and admitted that a frozendict is quite useless in Python, if I did not see that the constructor speed can be faster. I'm not sure 100% of the result. This is why I'm suggesting to try to implement the speed optimizations to the constructor of dict first: 1. the optimizations could be not safe. Indeed recently I got a segfault that I have to investigate 2. maybe dict can be really optimized further. After that, the difference between dict and frozendict performance could be minimal That said, maybe there are four good use cases for a frozendict: 1. they could be used for __annotations__ of function objects, and similar cases 2. they could be used to implement "immutable" modules or classes 3. frozendict can be easily cached, like tuples. 4. as I said, as an alternative to MappingProxyType, since it's very slow and it's used also in the C code of CPython and in the Python stdlib. This maybe is not useful because: a. the speed of MappingProxyType can be improved b. MappingProxyType can't be replaced because you *really* want and need a proxy

6 20

Exact decimal multiplication and division operations
by Random832 15 Oct '20

15 Oct '20

I was making a "convert Fraction to Decimal, exactly if possible" function and ran into a wall: it's not possible to do some of the necessary operations with exact precision in decimal: - multiplication - division where the result can be represented exactly [the divisor is an integer whose prime factors are only two and five, or a rational number whose numerator qualifies] I assume there's some way around it that I haven't spent enough time to figure out [create a temporary context with sufficint digits for multiplication, and work out the reciprocal power of 10 by hand to use this multiplication to implement division], but I feel like these exact operations should be supported in the standard library.

9 27

Enhancing Python custom memory allocators
by Julien Danjou 15 Oct '20

15 Oct '20

Hi there, Since Python 3.4, once can replace easily the Python memory allocators via the C API with its own copy (https://docs.python.org/3/c-api/memory.html#customize-memory-allocators). This is a great feature for implementing performance optimizations or profiling. This has given the ability to build powerful tools such as tracemalloc! However, there's one cruelly missing feature. The API only gives you access to the number of bytes to allocate and later to the pointer to free. It would be really useful to have more context about the allocation itself. For example, one piece of information that would be really useful is to have the type that is being allocated. This would allow profilers, such as tracemalloc, to get more information about the allocated type, and not only a number of bytes in order to help debugging or building statistics. I doubt this is currently doable without changing or adding a new C API unfortunately. Thoughts? -- Julien Danjou // Free Software hacker // https://julien.danjou.info

1 0

Exceptions as function calls to kill boilerplate
by jmward01＠gmail.com 14 Oct '20

14 Oct '20

I generally find exception objects are really just boilerplate heavy objects masking what I really want them to be, function calls: class MySpecialException(Exception): def __init__(self, some, variables, i, am, tracking): self.some = some ... ... try: if some_test_that_fails(variables): raise MySpecialException(a, b, c, d, e, f) except MySpecialException as e: logger.warning(f"BAD THING {e.a} HAPPENED!") if not handle_it(e.a, e.b, e.c, e.f): raise ... Instead of needing a whole new class definition, wouldn't it be nice to just have something like: .... #notice there isn't a boilerplate custom class created! try: if some_test_that_fails(variables): #I still have a base exception to fall back on for handlers that don't know my special exception raise Exception.my_special_exception(a, b, c, d, e, f) except Exception.my_special_excpetion(a:int, b:str, d, e, f): logger.warning(f"BAD THING {a} HAPPENED!") if not handle_it(a, b, c, f): raise The core idea here is that we are just passing control, so why make exceptions an exception to how control is normally passed, via functions. Just a thought.

15 32

Add _KB, _MB, _GB to numeric literals
by malincns＠163.com 14 Oct '20

14 Oct '20

PEP 515 added underscore to numeric literals, it brings better readability. PEP 515 -- Underscores in Numeric Literals https://www.python.org/dev/peps/pep-0515/ Is it possible to add _KB, _MB, _GB to numeric literals, for example: 200_KB (200*1024) 150_MB (150*1024*1024) 2_GB (2*1024*1024*1024) Do you think it's a good code style?

9 10

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas October 2020