Python-ideas October 2020

python-ideas@python.org

94 participants
49 discussions

by ankith abhayan

Hi, I would like to request a new feature that allows you to clear the console screen. Like in c++, the CLS function Thanks

11 months

option to keep TemporaryDirectory

by David Kolovratník

Hi Everyone, When adding debug option to a program I find useful to keep temporary stuff. tempfile.NamedTemporaryFile has option delete defaulting to True. What about adding such option to tempfile.TemporaryDirectory as well? It would be just a minor modification in Lib/tempfile.py. What do you think? Kind regards, David Kolovratník

11 months

Weakrefs for lru_cache?

by Ram Rachum

Hi everyone, For many years, I've used a `cache` decorator that I built <https://github.com/cool-RR/python_toolbox/blob/master/python_toolbox/cachin…> for caching Python functions. Then `functools.lru_cache` was implemented, which is much more standard. However, as far as I know, it holds normal references to its keys, rather than weak references. This means that it can cause memory leaks when it's storing items that don't have any references elsewhere. This often makes me reluctant to use it. What do you think about supporting weakrefs in for keys lru_cache? If I remember correctly, the main difficulty was that not all keys are of a type that can be weakreffed. If I remember correctly again, I've solved this by including logic that attempts a weakref when possible, and degrades to a strong ref. What do you think about that? Thanks, Ram.

11 months

except-try block

by haael

Very often I use the following construction (especially in operators): try: something except SomeError: try: something_else except AnotherError: try: something_completely_different except Whatever: return NotImplemented I propose a construction to simplify the staircase hierarchy of exception handlers: try: something except SomeError try: something_else except AnotherError try: something_completely_different except Whatever: return NotImplemented This would be somewhat analogous to `else if` blocks. This pattern is very common in operator implementations. And this proposal could simplify it a bit. haael

11 months

Fwd: Re: Experimenting with dict performance, and an immutable dict

by Marco Sulla

On Wed, 22 Jul 2020 at 15:35, Antoine Pitrou <solipsis(a)pitrou.net> wrote: > The deltas also look small for a micro-benchmark. I certainly don't > think this is a sufficient reason to add a new datatype to Python. > I think some of the optimizations can be experimented with dict itself. Deltas seem to be small, because I divide them by the number of timeit runs. I think that a 30-45% gain in constructor speed is not something small. The benchmarks in question have the Name that starts with "constructor". PS: sorry for double mail, I sent this message to only you by mistake. On Wed, 22 Jul 2020 at 18:26, Guido van Rossum <guido(a)python.org> wrote: > Did you study PEP 416 (frozendict) and PEP 603 (frozenmap)? > Yes. About frozenmap, at the very start I added to the benchmarks also immutables.Map, but I removed it when I realized that it was slower than frozendict in every bench. Maybe this is because immutables.Map is a C extension and not a builtin type. About PEP 416, yes, I tried to follow it. Indeed hash is calculated using the strategy described in the PEP. I also take a look to frozenset and tuple code. Frankly I must admit that the rejection of PEP 416 was a good idea. Indeed I started to implement an immutable dict only for a symmetry reason, and because I am somewhat fascinated by functional programming and immutables, without a rational reason. I suppose I would have given up and admitted that a frozendict is quite useless in Python, if I did not see that the constructor speed can be faster. I'm not sure 100% of the result. This is why I'm suggesting to try to implement the speed optimizations to the constructor of dict first: 1. the optimizations could be not safe. Indeed recently I got a segfault that I have to investigate 2. maybe dict can be really optimized further. After that, the difference between dict and frozendict performance could be minimal That said, maybe there are four good use cases for a frozendict: 1. they could be used for __annotations__ of function objects, and similar cases 2. they could be used to implement "immutable" modules or classes 3. frozendict can be easily cached, like tuples. 4. as I said, as an alternative to MappingProxyType, since it's very slow and it's used also in the C code of CPython and in the Python stdlib. This maybe is not useful because: a. the speed of MappingProxyType can be improved b. MappingProxyType can't be replaced because you *really* want and need a proxy

11 months

Exact decimal multiplication and division operations

by Random832

I was making a "convert Fraction to Decimal, exactly if possible" function and ran into a wall: it's not possible to do some of the necessary operations with exact precision in decimal: - multiplication - division where the result can be represented exactly [the divisor is an integer whose prime factors are only two and five, or a rational number whose numerator qualifies] I assume there's some way around it that I haven't spent enough time to figure out [create a temporary context with sufficint digits for multiplication, and work out the reciprocal power of 10 by hand to use this multiplication to implement division], but I feel like these exact operations should be supported in the standard library.

11 months

Enhancing Python custom memory allocators

by Julien Danjou

Hi there, Since Python 3.4, once can replace easily the Python memory allocators via the C API with its own copy (https://docs.python.org/3/c-api/memory.html#customize-memory-allocators). This is a great feature for implementing performance optimizations or profiling. This has given the ability to build powerful tools such as tracemalloc! However, there's one cruelly missing feature. The API only gives you access to the number of bytes to allocate and later to the pointer to free. It would be really useful to have more context about the allocation itself. For example, one piece of information that would be really useful is to have the type that is being allocated. This would allow profilers, such as tracemalloc, to get more information about the allocated type, and not only a number of bytes in order to help debugging or building statistics. I doubt this is currently doable without changing or adding a new C API unfortunately. Thoughts? -- Julien Danjou // Free Software hacker // https://julien.danjou.info

11 months

Exceptions as function calls to kill boilerplate

by jmward01＠gmail.com

I generally find exception objects are really just boilerplate heavy objects masking what I really want them to be, function calls: class MySpecialException(Exception): def __init__(self, some, variables, i, am, tracking): self.some = some ... ... try: if some_test_that_fails(variables): raise MySpecialException(a, b, c, d, e, f) except MySpecialException as e: logger.warning(f"BAD THING {e.a} HAPPENED!") if not handle_it(e.a, e.b, e.c, e.f): raise ... Instead of needing a whole new class definition, wouldn't it be nice to just have something like: .... #notice there isn't a boilerplate custom class created! try: if some_test_that_fails(variables): #I still have a base exception to fall back on for handlers that don't know my special exception raise Exception.my_special_exception(a, b, c, d, e, f) except Exception.my_special_excpetion(a:int, b:str, d, e, f): logger.warning(f"BAD THING {a} HAPPENED!") if not handle_it(a, b, c, f): raise The core idea here is that we are just passing control, so why make exceptions an exception to how control is normally passed, via functions. Just a thought.

11 months, 1 week

Add _KB, _MB, _GB to numeric literals

by malincns＠163.com

PEP 515 added underscore to numeric literals, it brings better readability. PEP 515 -- Underscores in Numeric Literals https://www.python.org/dev/peps/pep-0515/ Is it possible to add _KB, _MB, _GB to numeric literals, for example: 200_KB (200*1024) 150_MB (150*1024*1024) 2_GB (2*1024*1024*1024) Do you think it's a good code style?

11 months, 1 week

I'm no longer a co-author of PEP 637 - keyword args in item access

by Jonathan Fine

Hi The following is copied and pasted from https://github.com/python/peps/pull/1650 Stefano Borini =========== Commit message: PEP 637 I've asked Jonathan Fine to resign as PEP author, and he's agreed to do so. Details in the pull request Pull request comment: Hi Jonathan Thank you for accepting my invitation to be a co-author of PEP 637. I regret that because of disagreeing direction for the PEP I think it best for the community that I ask you to resign as co-author. Thank you for your help === Jonathan Fine =========== Hi Stefano I'm sorry things didn't work out, the way either of us wanted. Yes, there were disagreements! Particularly on python-ideas, involving also PEP sponsor Steve D'Aprano. I'd like the PEP to be improved before it is submitted. Jonathan === with best regards Jonathan

11 months, 1 week

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas October 2020