Mailman 3 February 2022 - Python-ideas

CLI convenience?
by Eliot Lear Feb. 21, 2022

Feb. 21, 2022

Hi everyone, I'm sure there's a clear answer, probably written some place obvious, but I wonder if there is a reason why certain functions are not linked to console_scripts. In particular I would think this would be good for json.tool (though I might call it something else, like pyjson_pp). Same with venv. I'm sure I'm missing something obvious. Thanks, Eliot

4 4

A memory map based data persistence and startup speedup approach
by 严懿宸(文极) Feb. 21, 2022

Feb. 21, 2022

Hi folks, as illustrated in faster-cpython#150 [1], we have implemented a mechanism that supports data persistence of a subset of python date types with mmap, therefore can reduce package import time by caching code object. This could be seen as a more eager pyc format, as they are for the same purpose, but our approach try to avoid [de]serialization. Therefore, we get a speedup in overall python startup by ~15%. Currently, we’ve made it a third-party library and have been working on open-sourcing. Our implementation (whose non-official name is “pycds”) mainly contains two parts: importlib hooks, this implements the mechanism to dump code objects to an archive and a `Finder` that supports loading code object from mapped memory. Dumping and loading (subset of) python types with mmap. In this part, we deal with 1) ASLR by patching `ob_type` fields; 2) hash seed randomization by supporting only basic types who don’t have hash-based layout (i.e. dict is not supported); 3) interned string by re-interning strings while loading mmap archive and so on. After pycds has been installed, complete workflow of our approach includes three parts: Record name of imported packages to heap.lst, `PYCDSMODE=TRACE PYCDSLIST=heap.lst python run.py` Dump memory archive of code objects of imported packages, this step does not involve the python script, `PYCDSMODE=DUMP PYCDSLIST=heap.lst PYCDSARCHIVE=heap.img python` Run other python processes with created archive, `PYCDSMODE=SHARE PYCDSARCHIVE=heap.img python run.py` We could even make use of immortal objects if PEP 683 [2] was accepted, that could gives CDS more performance improvements. Currently, any archived object is virtually immortal, we add rc by 1 to who has been copied to the archive to avoid being deallocated. However, without changes to CPython, rc fields of archived object will still be updated, therefore have extra footprint due to CoW. More background and detail implementation could be found at [1]. We think it could be an effective way to improve python’s startup performance, and could even do more like sharing large data between python instances. We’re welcome for suggestions and questions. Best, Yichen Yan Alibaba Compiler Group [1] “Faster startup -- Share code objects from memory-mapped file”, https://github.com/faster-cpython/ideas/discussions/150 [2] PEP 683: "Immortal Objects, Using a Fixed Refcount" (draft), https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDC…

4 3

Please consider making dataclasses define an empty __post_init__
by Neil Girdhar Feb. 20, 2022

Feb. 20, 2022

This was discussed here: https://bugs.python.org/issue46757 This came up because I had a dataclass subclass to flax.linen.Module that had its own post-init. When I figured out that I needed to call super, I did so. Later, when I refactored my code to remove the superclass, the super-call was broken. It have been much simpler to always call super in __post_init__ from the start. The proposal on the bug report was to get in the habit of doing: if hasattr(super(), "__post_init__"): super().__post_init__() or: try: post_init = super().__post_init__ except AttributeError: pass else: post_init() I think it will be difficult to convince projects to do this. Best, Neil

1 0

Requirements.txt inside virtual environment
by Vishesh Mangla Feb. 20, 2022

Feb. 20, 2022

Consider the following scenario: 1) You have to format your pc. 2) You copy your python projects to your hard disk along with the virtual environment(I keep the virtual environment in the project folder only). 3)When you copy your projects back after successfully formatting your pc, the useless virtualenv lies there and you can't get the requirements.txt now and have to install all packages one by one. It could be great if pip or python or anyone would by default store a requirements.txt file inside the virtual env and on `pip install xyz` would append `xyz` to that file.

9 17

Creating ranges with ellipsis
by Soni L. Feb. 18, 2022

Feb. 18, 2022

This might be a silly idea but, would it be a good idea to have ...[a:b:c] return a range(a, b, c)?

6 6

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 18, 2022

Feb. 18, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, such a class would look like this: class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify() push = heapq.heappush pop = heapq.heappop pushpop = heapq.heappushpop replace = heapq.heapreplace heapify = heapq.heapify # This could be a simple wrapper as well, but I had the following thoughts anyways, so here they are def nsmallest(self, n, key=None): # heapq.nsmallest makes a *max* heap of the first n elements, # while we know that self is already a min heap, so we can # make the max heap construction faster self[:n] = reversed(self[:n]) return heapq.nsmallest(n, self, key) # do we define nlargest on a minheap?? Wrapping around the C builtin functions (which aren't descriptors) would be a bit harder, but not much so: from functools import partial class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify = partial(heapq.heapify, self) self.push = partial(heapq.heappush, self) ... self.heapify() Thoughts?

17 32

Regex pattern matching
by Valentin Berlier Feb. 18, 2022

Feb. 18, 2022

Hi, I've been thinking that it would be nice if regex match objects could be deconstructed with pattern matching. For example, a simple .obj parser could use it like this: match re.match(r"(v|f) (\d+) (\d+) (\d+)", line): case ["v", x, y, z]: print("Handle vertex") case ["f", a, b, c]: print("Handle face") Sequence patterns would extract groups directly. Mapping patterns could be used to extract named groups, which would be nice for simple parsers/tokenizers: match re.match(r"(?P<number>\d+)|(?P<add>\+)|(?P<mul>\*)", line): case {"number": str(value)}: return Token(type="number", value=int(value)) case {"add": str()}: return Token(type="add") case {"mul": str()}: return Token(type="mul") Right now, match objects aren't proper sequence or mapping types though, but that doesn't seem too complicated to achieve. If this is something that enough people would consider useful I'm willing to look into how to implement this.

3 3

Re: Creating a class template file generator inside Python language
by Avanish Gupta Feb. 16, 2022

Feb. 16, 2022

Thanks for the previous mail. I would like to highlight the value my thought offers to the developers. They often have to write classes while writing a module. In the class, they are supposed to write classes and objects. In a class, typically the attributes are private, and we have getters and setters to access and modify them. My thought is to automate the process of creating such class files with command line. For this, I have made my initial attempt to create a module that does so. Here is a short video in which I am demonstrating it. I am also attaching the details of my terminal to get a bit of more details. I am looking forward to hearing from you if I should work on integrating it within the Python Language features, or I should go some other way round, or this is not a very useful tool for many people. *avanishcodes@avanishcodes*:*~*$ echo "No Student.py file here" No Student.py file here*avanishcodes@avanishcodes*:*~*$ ls*Academics* *Documents* *node_modules* *Pictures* *snap* *Videos**bin* *Downloads* package.json *Projects* Student.py*Desktop* *Music* package-lock.json *Public* *Templates**avanishcodes@avanishcodes*:*~*$ rm Student.py *avanishcodes@avanishcodes*:*~*$ ls*Academics* *Documents* *node_modules* *Pictures* *snap**bin* *Downloads* package.json *Projects* *Templates**Desktop* *Music* package-lock.json *Public* *Videos**avanishcodes@avanishcodes*:*~*$ pip install classgen Requirement already satisfied: classgen in ./.local/lib/python3.8/site-packages (0.0.7)*avanishcodes@avanishcodes*:*~*$ python3 -m classgen Student member1,member2,member3,member4,member5,member6 Class Student generated successfully.*avanishcodes@avanishcodes*:*~*$ cat Student.py #(class) Student class Student: """ This class is used to represent a Student. Attributes: member1: The member1 of the Student. member2: The member2 of the Student. member3: The member3 of the Student. member4: The member4 of the Student. member5: The member5 of the Student. member6: The member6 of the Student. Methods: get_member1(self): Gets the member1 of the Student. get_member2(self): Gets the member2 of the Student. get_member3(self): Gets the member3 of the Student. get_member4(self): Gets the member4 of the Student. get_member5(self): Gets the member5 of the Student. get_member6(self): Gets the member6 of the Student. set_member1(self, member1): Sets the member1 of the Student. set_member2(self, member2): Sets the member2 of the Student. set_member3(self, member3): Sets the member3 of the Student. set_member4(self, member4): Sets the member4 of the Student. set_member5(self, member5): Sets the member5 of the Student. set_member6(self, member6): Sets the member6 of the Student. """ _member1: None _member2: None _member3: None _member4: None _member5: None _member6: None def __init__(self, member1, member2, member3, member4, member5, member6): """ Initializes a Student object. Params: member1: The member1 of the Student. member2: The member2 of the Student. member3: The member3 of the Student. member4: The member4 of the Student. member5: The member5 of the Student. member6: The member6 of the Student. """ self._member1 = member1 self._member2 = member2 self._member3 = member3 self._member4 = member4 self._member5 = member5 self._member6 = member6 def get_member1(self): """ Gets the member1 of the Student. Returns: get_member1: The member1 of the Student. """ return self._member1 def get_member2(self): """ Gets the member2 of the Student. Returns: get_member2: The member2 of the Student. """ return self._member2 def get_member3(self): """ Gets the member3 of the Student. Returns: get_member3: The member3 of the Student. """ return self._member3 def get_member4(self): """ Gets the member4 of the Student. Returns: get_member4: The member4 of the Student. """ return self._member4 def get_member5(self): """ Gets the member5 of the Student. Returns: get_member5: The member5 of the Student. """ return self._member5 def get_member6(self): """ Gets the member6 of the Student. Returns: get_member6: The member6 of the Student. """ return self._member6 def set_member1(self, member1): """ Sets the member1 of the Student. Params: member1: The member1 of the Student. """ self._member1 = member1 def set_member2(self, member2): """ Sets the member2 of the Student. Params: member2: The member2 of the Student. """ self._member2 = member2 def set_member3(self, member3): """ Sets the member3 of the Student. Params: member3: The member3 of the Student. """ self._member3 = member3 def set_member4(self, member4): """ Sets the member4 of the Student. Params: member4: The member4 of the Student. """ self._member4 = member4 def set_member5(self, member5): """ Sets the member5 of the Student. Params: member5: The member5 of the Student. """ self._member5 = member5 def set_member6(self, member6): """ Sets the member6 of the Student. Params: member6: The member6 of the Student. """ self._member6 = member6 *avanishcodes@avanishcodes*:*~*$ AvanishCodes - VMware Workstation 16 Player (No... <https://drive.google.com/file/d/1I4rbwjr-l5eNqbNzpnY8QOGrXP-c3t_B/view?usp=…> Thanks and Regards Avanish On Mon, Feb 14, 2022 at 4:59 AM Avanish Gupta <avanishcodes(a)gmail.com> wrote: > Hi all > I am Avanish, a junior from Indian Institute of Information Technology, > Surat. > I have written a simple script to generate class files based on command > line arguments. It is quite simple in nature, but if improved and packaged > as a part of the Python language feature, it has the potential to save a > lot of time for the developers across the globe. > Please check the package at https://pypi.org/project/classgen > If I can be given a chance to work on it, I would like to do it in much > more detail. I hope I am on the right track, and I seek the guidance of > your expertise and experience. > > > > Thanks and regards >

4 3

New convenience attribute pathlib.Path.stems
by Clay Gerrard Feb. 15, 2022

Feb. 15, 2022

>>> from pathlib import Path >>> p = Path('/etc/swift/object.ring.gz') >>> p.suffix '.gz' >>> p.suffixes ['.ring', '.gz'] >>> p.stem 'object.ring' >>> p.stems Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'PosixPath' object has no attribute 'stems' I think it would have been nice if .stems = ['object', '.ring'] The idiomatic answer for this persons definition of "true" stem: https://stackoverflow.com/questions/31890341/clean-way-to-get-the-true-stem… ... seemed to [ab]use with_suffix(‘’), but .stems[0] might be more obvious. What do you think? -- Clay Gerrard -- Clay Gerrard 210 788 9431

1 0

Please consider mentioning property without setter when an attribute can't be set
by Neil Girdhar Feb. 15, 2022

Feb. 15, 2022

Hello, consider: class C: @property def f(self) -> int: return 2 class D(C): pass D().f = 2 Gives: Traceback (most recent call last): File "/home/neil/src/cmm/a.py", line 10, in <module> D().f = 2 AttributeError: can't set attribute 'f' This can be a pain to debug when the property is buried in a base class. Would it make sense to mention the reason why the attribute can't be set, namely that it's on a property without a setter? Best, Neil

7 11