Python-ideas August 2019

python-ideas@python.org

74 participants
51 discussions

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow 18 Feb '22

18 Feb '22

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, such a class would look like this: class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify() push = heapq.heappush pop = heapq.heappop pushpop = heapq.heappushpop replace = heapq.heapreplace heapify = heapq.heapify # This could be a simple wrapper as well, but I had the following thoughts anyways, so here they are def nsmallest(self, n, key=None): # heapq.nsmallest makes a *max* heap of the first n elements, # while we know that self is already a min heap, so we can # make the max heap construction faster self[:n] = reversed(self[:n]) return heapq.nsmallest(n, self, key) # do we define nlargest on a minheap?? Wrapping around the C builtin functions (which aren't descriptors) would be a bit harder, but not much so: from functools import partial class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify = partial(heapq.heapify, self) self.push = partial(heapq.heappush, self) ... self.heapify() Thoughts?

17 32

Make fnmatch.filter accept a tuple of patterns
by Andre Delfino 26 Mar '21

26 Mar '21

Frequently, while globbing, one needs to work with multiple extensions. I’d like to propose for fnmatch.filter to handle a tuple of patterns (while preserving the single str argument functionality, alas str.endswith), as a first step for glob.i?glob to accept multiple patterns as well. Here is the implementation I came up with: https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-mu… If this is deemed reasonable, I’ll write tests and documentation updates. Any opinion?

9 15

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman 12 Mar '21

12 Mar '21

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman

12 35

Add command-line option to unittest for enabling post-mortem debugging
by Dominik Vilsmeier 09 Jan '21

09 Jan '21

Consider the following example: import unittest def foo(): for x in [1, 2, 'oops', 4]: print(x + 100) class TestFoo(unittest.TestCase): def test_foo(self): self.assertIs(foo(), None) if __name__ == '__main__': unittest.main() If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`. However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-mortem debugging. `--failfast` doesn't help, the exception is still swallowed. Since I am not aware of a solution that enables post-mortem debugging in such a case (without modifying the test scripts, please correct me if one exists), I propose adding a command-line option to `unittest` for [running test cases in debug mode](https://docs.python.org/3/library/unittest.html#unittest.TestCase.deb… so that post-mortem debugging can be used. P.S.: There is also [this SO question](https://stackoverflow.com/q/4398967/3767239) on a similar topic.

1 1

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley 16 Dec '20

16 Dec '20

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

Add the imath module
by Serhiy Storchaka 22 Mar '20

22 Mar '20

What are your thoughts about adding a new imath module for integer mathematics? It could contain the following functions: * factorial(n) Is just moved from the math module, but non-integer types are rejected. Currently math.factorial() accepts also integer floats like 3.0. It looks to me, the rationale was that at the time when math.factorial() was added, all function in the math module worked with floats. But now we can revise this decision. * gcd(n, m) Is just moved from the math module. * as_integer_ration(x) Equivalents to: def as_integer_ration(x): if hasattr(x, 'as_integer_ration'): return x.as_integer_ration() else: return (x.numerator, x.denominator) * binom(n, k) Returns factorial(n) // (factorial(k) * factorial(n-k)), but uses more efficient algorithm. * sqrt(n) Returns the largest integer r such that r**2 <= n and (r+1)**2 > n. * isprime(n) Tests if n is a prime number. * primes() Returns an iterator of prime numbers: 2, 3, 5, 7, 11, 13,... Are there more ideas?

14 38

New explicit methods to trim strings
by Alex Grigoryev 21 Mar '20

21 Mar '20

Following the discussion here (https://link.getmailspring.com/link/7D84D131-65B6-4EF7-9C43-51957F9DFAA9@ge…) I propose to add 3 new string methods: str.trim, str.ltrim, str.rtrim Another option would be to change API for str.split method to work correctly with sequences. In [1]: def ltrim(s, seq): ...: return s[len(seq):] if s.startswith(seq) else s ...: In [2]: def rtrim(s, seq): ...: return s[:-len(seq)] if s.endswith(seq) else s ...: In [3]: def trim(s, seq): ...: return ltrim(rtrim(s, seq), seq) ...: In [4]: s = 'mailto:maria@gmail.com' In [5]: ltrim(s, 'mailto:') Out[5]: 'maria(a)gmail.com' In [6]: rtrim(s, 'com') Out[6]: 'mailto:maria@gmail.' In [7]: trim(s, 'm') Out[7]: 'ailto:maria@gmail.co'

28 162

Re: [Python-ideas] Allow star unpacking within an slice expression
by Neil Girdhar 07 Mar '20

07 Mar '20

I didn't think of this when we were discussing 448. I ran into this today, so I agree with you that it would be nice to have this. Best, Neil On Monday, December 4, 2017 at 1:02:09 AM UTC-5, Eric Wieser wrote: > > Hi, > > I've been thinking about the * unpacking operator while writing some numpy > code. PEP 448 allows the following: > > values = 1, *some_tuple, 2 > object[(1, *some_tuple, 2)] > > It seems reasonable to me that it should be extended to allow > > item = object[1, *some_tuple, 2] > item = object[1, *some_tuple, :] > > Was this overlooked in the original proposal, or deliberately rejected? > > Eric >

2 2

Specify number of items to allocate for array.array() constructor
by Sven Rahmann 22 Feb '20

22 Feb '20

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

14 20

Fixed point format for numbers with locale based separators
by Łukasz Stelmach 20 Dec '19

20 Dec '19

Hi, I would like to present two pull requests[1][2] implementing fixed point presentation of numbers and ask for comments. The first is mine. I learnt about the second after publishing mine. The only format using decimal separator from locale data for float/complex/decimal numbers at the moment is "n" which behaves like "g". The drawback of these formats, I would like to overcome, is the inability to print numbers ranging more than one order of magnitude with the same number of decimal digits without "manually" (with some additional custom code) adjusting precission. The other option is to "manually" replace "." as printed by "f" with a local decimal separator. Neither of these option is appealing to my. Formatting 1.23456789 * n (LC_ALL=3Dpl_PL.UTF-8) | n | ".2f" | ".3n" | |---+----------+----------| | 1 | 1.23 | 1,23 | | 2 | 12.35 | 12,3 | | 3 | 123.46 | 123 | | 4 | 1234.57 | 1,23e+03 | In the application I want to create I am going to present users numbers ranging up to 3 orders of magnitude and I (my users) want them to be presented consistently with regards to number of decimal digits AND I want to conform to rules of languages of my users. And I would like to avoid the exponent notation by all means. I can't say much about James Emerton's implementation or his intentions, but please take a look at our patches and give your comments so either of us or together we can implement this feature. PS. In theory both implementations could be merged because James chose to use "l" to use LC_MONETARY category and I chose "m" to use LC_NUMERIC. [1] https://github.com/python/cpython/pull/11405 [2] https://github.com/python/cpython/pull/8612 -- Miłego dnia, Łukasz Stelmach

5 9

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas August 2019