Mailman 3 January 2021 - Python-ideas

Additional LRU cache introspection facilities
by Steven D'Aprano 12 Jan '21

12 Jan '21

Is anyone else interested in additional introspection facilities for the functools.lru_cache? You can view the cache hit and miss statistics using the cache_info() method, but you can't see what values actually are in the cache. That would be useful to me. I propose a method: @functools.lru_cache() def function(arg): ... function.cache_export() that returns a dictionary {arg: value} representing the cache. It wouldn't be the cache itself, just a shallow copy of the cache data. Any interest? -- Steve

5 9

timeit improvement idea: add an option to measure a script execution time
by Alex Prengère 10 Jan '21

10 Jan '21

Hello, Today I had a quite simple need, I am unsure about the best way to do it, and saw a possible improvement for the *timeit *module. I have about 30 Python scripts and I want to measure precisely their execution times, without measuring the interpreter startup time, because for most of them it is quite short (<1ms). I see several ways to do this: 0. Just measure the execution time with "time python script.py" > Not great for me, because for the fastest scripts, it is very imprecise. 1. Modify the scripts to add some "import time; start = time.perf_counter()" at the beginning of each script and "print(time.perf_counter() - start)" at the end of each script. > I would prefer to not change the scripts source code, but I agree that this works otherwise. 2. Have a way to dynamically inject those "time.perf_counter()" as an command line option. I searched and do not think this is currently possible, but 2 options looking like that would work : $ python script.py --before 'import time; start = time.perf_counter()' --after 'print(time.perf_counter() - start)' 3. Use timeit. The scripts have no side effects so repeating their execution the way timeit does, works for me. The only issue is that, as far as I know, timeit only allows statements as input parameters, not the whole script, like for example: $ python -m timeit --script script.py 4. Write custom code to import each script. Unless I am mistaken, I feel like solution #3 with a new option of timeit is the most appropriate. I may have missed something more obvious though. What do you think?

4 5

Off-topic: Happy Birthday, Don Knuth (and Python coroutines)
by Jonathan Fine 10 Jan '21

10 Jan '21

Hi Don Knuth, author of The Art Of Computer Programming and creator of the TeX typesetting system is 83 today. Happy Birthday Don. Thank you for TeX and everything else. You're warmly invited to join me online to celebrate Don's life and work. The virtual party is on Thursday 14 January, 6:30 to 7:30pm UK time. You'll find the zoom details at (and a goto tribute) at: https://jfine2358.github.io/post/2021/01/10/happy-birthday-don-knuth/ This link also contains a discussion of how merging two binary trees provides a good example of how coroutines can be useful, as in Python's async and wait. with best wishes Jonathan

1 0

Support reversed(itertools.chain(x, y, z))
by Ram Rachum 09 Jan '21

09 Jan '21

Today I had a need: I had a tuple of dynamic sequence-like objects. I wanted to iterate on them reversed, starting with items of the last one and slowly making my way towards the first one. In short, I want `reversed(itertools.chain(x, y, z))` that behaves like `itertools.chain(map(reversed, (z, y, x)))`. What do you think?

7 10

Add command-line option to unittest for enabling post-mortem debugging
by Dominik Vilsmeier 09 Jan '21

09 Jan '21

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-mortem debugging. `--failfast` doesn't help, the exception is still swallowed. Since I am not aware of a solution that enables post-mortem debugging in such a case (without modifying the test scripts, please correct me if one exists), I propose adding a command-line option to `unittest` for [running test cases in debug mode](https://docs.python.org/3/library/unittest.html#unittest.TestCase.deb… so that post-mortem debugging can be used. P.S.: There is also [this SO question](https://stackoverflow.com/q/4398967/3767239) on a similar topic.

1 1

Add a "closed for" loop (was "PEP 533 Redux?")
by Brendan Moloney 06 Jan '21

06 Jan '21

I mentioned this off-hand in the other thread, but the more I think about it the more I think it is a reasonable compromise between the two schools of thought on PEP 533. Wouldn't the new PEG parser allow a "closed for" statement without breaking code that uses "closed" as a variable (it would be a context sensitive keyword)? The users who work in the space that cares about such things can start writing: closed for document in read_newline_separated_json(path): ... Instead of: with closing(read_newline_separated_json(path)) as genobj: for document in genobj: ... It is still on the API consumer to know when they need to use a "closed for" loop, but at least they don't have to restructure their code, and there would be no backwards incompatibility. If we want to get really crazy we can also allow an "open for" loop and kick off a very long deprecation process where eventually an unadorned for loop would become "closed" by default. Thanks, Brendan

2 1

An option to force the path separator for the "os.path.join()" method.
by Mikhail V 06 Jan '21

06 Jan '21

I'd like to have an option to force the path separator for the "os.path.join()" method. E.g. if I run the script on Windows, but I generate, say, an URL, I'd find it convenient to use the same method, but with an explicit flag to "join" with the forward slash (because URLs use it). Currently I simply use an f-string to combine a path, like e.g.: path = f"{root}/{dir}" but it will insert extra "/" if there if "root" already has "/" on the end. But the "join" method will not, which is a pro for the method vs "manual" string construct. I know there is the "pathlib" module with all conversion methods, but it's overkill for many tasks. So I'd rather like to have an option to write for example: path = os.path.join (root, dir, sep = "posix") So that it joins with the forward slash even if run on Windows. Also it seems that e.g. "os.path.split()" already supports both separators, so I think this addition won't bring inconsistency into the module and its usage.

7 7

Python JIT Compilation Thoughts
by redradist＠gmail.com 05 Jan '21

05 Jan '21

Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!

9 15

Add venv activate command to the venv modules
by Abdur-Rahmaan Janhangeer 04 Jan '21

04 Jan '21

Greetings list, put simply, be able to use $ python -m venv venv_name activate To activate an env instead of having each platform have a way of handling it Kind Regards, Abdur-Rahmaan Janhangeer about <https://compileralchemy.github.io/> | blog <https://www.pythonkitchen.com> github <https://github.com/Abdur-RahmaanJ> Mauritius

7 13

asyncio.to_process function, similar to asyncio.to_thread function
by Abdulla Al Kathiri 03 Jan '21

03 Jan '21

Hello all, I suppose asyncio.to_thread uses concurrent.futures.ThreadPoolExecutor under the hood, so it saves us from getting the running event loop and calling the loop.run_in_executor function . Why don’t we have a similar function for the ProcessPoolExecutor, e.g. asyncio.to_process? Abdulla

2 1