Mailman 3 April 2017 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Implicit string literal concatenation considered harmful?
by Guido van Rossum March 14, 2018

March 14, 2018

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' … [View More]

51 165

dict(default=int)
by Steven Piantadosi May 30, 2017

May 30, 2017

Hi All, I find importing defaultdict from collections to be clunky and it seems like having a default should just be an optional keyword to dict. Thus, something like, d = dict(default=int) would be the same as from collections import defaultdict d = defaultdict(int) Any thoughts? Thanks, ++Steve

15 19

Augmented assignment syntax for objects.
by Erik May 2, 2017

May 2, 2017

Hi. I suspect that this may have been discussed to death at some point in the past, but I've done some searching and I didn't come up with much. Apologies if I'm rehashing an old argument ;) I often find myself writing __init__ methods of the form: def __init__(self, foo, bar, baz, spam, ham): self.foo = foo self.bar = bar self.baz = baz self.spam = spam self.ham = ham This seems a little wordy and uses a lot of vertical space on the screen. Occasionally, I have considered … [View More]

25 78

Proposed PEP 484 addition: describe a way of annotating decorated declarations
by Guido van Rossum May 1, 2017

May 1, 2017

There's a PR to the peps proposal here: https://github.com/python/peps/pull/242 The full text of the current proposal is below. The motivation for this is that for complex decorators, even if the type checker can figure out what's going on (by taking the signature of the decorator into account), it's sometimes helpful to the human reader of the code to be reminded of the type after applying the decorators (or a stack thereof). Much discussion can be found in the PR. Note that we ended up … [View More]

2 2

Re: [Python-ideas] Augmented assignment syntax for objects.
by Tin Tvrtković April 30, 2017

April 30, 2017

I'm gonna take a shot at elaborating this point. We Python programmers often tout Python as a high-level, high-productivity language where complex and useful things are easy to do. However, vanilla Python classes are anything but high level; they're basically glorified dictionaries that require an awful lot of boilerplate for basic features. __init__ verbosity is just one of the symptoms of this. I'm going to posit we need declarative classes. (This is what a library like attrs provides, … [View More]basically.) For a class to be declarative, it needs to be possible to inspect the class for its attributes and more. I'm putting forward three examples. These examples are based on attrs since that's what I consider to be the best way of having declarative classes in Python today. First example. Here's a slide from a recent talk I've given to a local Python meetup: http://tinche.github.io/talks/attrs/#/8/5 This is a class decorator that, when applied to any attrs class, will generate a simple shallow copy() method for that class. This copy method will be faster than copy.copy() and IMO less surprising since it will actually invoke __init__ (and all logic inside). This isn't production code, just a proof of concept for the talk. Second example: I'm working on a library that can turn dictionaries into attrs classes (i.e. turn unstructured data into structured data). Example: http://cattrs.readthedocs.io/en/latest/structuring.html#attrs-classes. When dealing with nested classes the library not only needs to know exactly which fields a class has but also the types of those fields, recursively (for which I'm planning to use attrs metadata). This functionality is useful when preparing data to be stored or sent elsewhere, since serialization libraries and clients in other languages expect json/msgpack etc. Third example: I work at a mobile games company. The backends are Python, the games are written in other languages. The backends and the games share data structures - requests, responses, the save game, game data. I want to have a single source of truth for these structures, so I want to generate classes/structs in other languages from the backend Python classes. All of these examples require declarative classes to do properly. Other approaches are, I suppose, possible in some cases - like inspecting __init__ and whatnot. But I claim the underlying issue is that we should be declaring classes differently, both to enable these new, high-level use cases and because declarative classes are much more readable and contain less boilerplate than what we have now, and this is (for me) Python is about. Message: 1 > Date: Fri, 28 Apr 2017 09:28:55 +1000 > From: Steven D'Aprano <steve(a)pearwood.info> > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Augmented assignment syntax for objects. > Message-ID: <20170427232853.GC22525(a)ando.pearwood.info> > Content-Type: text/plain; charset=iso-8859-1 > > On Wed, Apr 26, 2017 at 08:52:46PM -0400, Juancarlo A?ez wrote: > > > In my experience, what Python is lacking is a way to declare attributes > > outside of the constructor. Take a look at how it's done in C#, Swisft, > or > > Go. > > Since you apparently already know how they do it, how about telling us > (1) how they do it, and (2) *why* they do it? > > > > Object attributes outside of the constructor would solve things more > > relevant than the vertical space used when assigning constructor > parameters > > to attributes. > > Solve which things? > > > > For example, multiple inheritance is well designed in > > Python, except that it often requires constructors with no parameters, > > which leads to objects with no default attributes. > > Can you elaborate? > > > > -- > Steve > > [View Less]

5 8

Discourage operator.__dunder__ functions
by Steven D'Aprano April 27, 2017

April 27, 2017

Notice that I said *discourage* rather than *deprecate*. Quoting the documentation: The operator module exports a set of efficient functions corresponding to the intrinsic operators of Python. For example, operator.add(x, y) is equivalent to the expression x+y. The function names are those used for special class methods; variants without leading and trailing __ are also provided for convenience. https://docs.python.org/3/library/operator.html I propose four … [View More]

7 7

Binary arithmetic does not always call subclasses first
by Stephan Hoyer April 27, 2017

April 27, 2017

I recently filed this as a bug, and was asked to repost to python-dev or python-ideas for greater visibility: http://bugs.python.org/issue30140 Without further ado, here is my original report: --------------------------- We are writing a system for overloading NumPy operations (see PR [1] and design doc [2]) that is designed to copy and extend Python's system for overloading binary operators. The reference documentation on binary arithmetic [3] states: > Note: If the right operand's type … [View More]is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations. However, this isn't actually done if the right operand merely inherits from the left operand's type. In practice, CPython requires that the right operand defines a different method before it defers to it. Note that the behavior is different for comparisons, which defer to subclasses regardless of whether they implement a new method [4]. I think this behavior is a mistake and should be corrected. It is just as useful to write generic binary arithmetic methods that are well defined on subclasses as generic comparison operations. In fact, this is exactly the design pattern we propose for objects implementing special operators like NumPy arrays (see NDArrayOperatorsMixin in [1] and [2]). Here is a simple example, of a well-behaved type that implements addition by wrapping its value and that returns NotImplemented when the other operand has the wrong type: class A: def __init__(self, value): self.value = value def __add__(self, other): if not isinstance(other, A): return NotImplemented return type(self)(self.value + other.value) __radd__ = __add__ def __repr__(self): return f'{type(self).__name__}({self.value!r})' class B(A): pass class C(A): def __add__(self, other): if not isinstance(other, A): return NotImplemented return type(self)(self.value + other.value) __radd__ = __add__ A does not defer to subclass B: >>> A(1) + B(1) A(2) But it does defer to subclass C, which defines new methods (literally copied/pasted) for __add__/__radd__: >>> A(1) + C(1) C(2) With the current behavior, special operator implementations need to explicitly account for the possibility that they are being called from a subclass by returning NotImplemented. My guess is that this is rarely done, which means that most of these methods are broken when used with subclasses, or subclasses needlessly reimplement these methods. Can we fix this logic for Python 3.7? [1] https://github.com/numpy/numpy/pull/8247 [2] https://github.com/charris/numpy/blob/406bbc652424fff332f49b0d2f2e5aedd8191… [3] https://docs.python.org/3/reference/datamodel.html#object.__ror__ [4] http://bugs.python.org/issue22052 ----------------------------- Mark Dickinson's response: This is probably worth bringing up on the python-dev or python-ideas mailing lists for greater visibility. I can't think of any plausible non-historical reason why it makes sense for comparisons to behave one way and arithmetic operators another. Altering this might be a PEP-level change, though. The "Coercion rules" section[1] of the Python 2.7 docs is a bit more explicit about the intent: """ Exception to the previous item: if the left operand is an instance of a built-in type or a new-style class, and the right operand is an instance of a proper subclass of that type or class and overrides the base’s __rop__() method, the right operand’s __rop__() method is tried before the left operand’s __op__() method. """ so the check for an override was clearly intentional, rather than an implementation convenience or accident. (It's also clearly intentional in the source and comments.) The 3.x docs don't have the "and overrides" language; I haven't figured out why and when that language changed. [1] https://docs.python.org/release/2.7.6/reference/datamodel.html#coercion-rul… [View Less]

7 10

Re: [Python-ideas] Augmented assignment syntax for objects.
by Tin Tvrtković April 25, 2017

April 25, 2017

You might want to check out attrs (http://attrs.readthedocs.io/en/stable/). It can generate the __init__ for you, and much much more. Date: Tue, 25 Apr 2017 14:33:49 +0200 > From: George Fischhof <george(a)fischhof.hu> > To: Paul Moore <p.f.moore(a)gmail.com> > Cc: Python-Ideas <python-ideas(a)python.org>, "Steven D'Aprano" > <steve(a)pearwood.info> > Subject: Re: [Python-ideas] Augmented assignment syntax for objects. > Message-ID: > … [View More]

1 0

Re: [Python-ideas] Callable Enum values
by Ethan Furman April 21, 2017

April 21, 2017

[redirecting back to the list] On 04/19/2017 11:15 PM, Stephan Hoyer wrote: > Ethan and Steven, > > Thanks for your feedback on this one. I agree that it probably doesn't make sense for the standard library. > > I'm still not really happy with any of the standard approaches for choosing a function based on an enum value -- they > all seem pretty verbose/ugly -- but clearly I'm bothered by this more than most, and the standard library is not a good > place for novel … [View More]

3 4

Thread-safe generators
by Serhiy Storchaka April 17, 2017

April 17, 2017

When use a generator from different threads you can get a ValueError "generator already executing". Getting this exception with the single thread is a programming error, it in case of different threads it could be possible to wait until other thread finish executing the generator. The generator can be made thread-safe after wrapping it in a class that acquire a lock before calling the generator's __next__ method (for example see [1]). But this is not very efficient of course. I wondering … [View More]

8 12

Callable Enum values
by Stephan Hoyer April 16, 2017

April 16, 2017

Enums are great. They allow you cleanly define a set of statically defined options. One way that I've found myself using enums recently is for dispatching (as keys in a dictionary) between different interchangeable functions or classes. My code looks something like this: from enum import Enum def foo(...): ... def bar(...): ... class Implementation(Enum): FOO = 1 BAR = 2 _IMPLEMENTATIONS = { Implementation.FOO: foo, Implementation.BAR: bar, } def … [View More]call_implementation(implementation, *args, **kwargs): return _IMPLEMENTATIONS[implementation](*args, **kwargs) The first part of this blog post does a nice job of summarizing the general use case: http://lukasz.langa.pl/8/single-dispatch-generic-functions/ Obviously, enums are better than strings, because they're static declared and already grouped together. But it would be nice if we could do one better, by eliminating the dictionary, moving the dictionary values to the enum and making the enum instances. You might try this by writing a small subclass of Enum: class CallableEnum(Enum): def __call__(self, *args, **kwargs): return self.value(*args, **kwargs) class Implementation(CallableEnum): FOO = foo BAR = bar def call_implementation(implementation, *args, **kwargs): return implementation(*args, **kwargs) This looks great, with the Enum serving essentially as a typed namespace for a group of related functions, but unfortunately it this doesn't work. The problem is that when you assign a function to an Enum, it treats it as a method instead of an enum value: http://stackoverflow.com/questions/40338652/how-to-define-enum-values-that-… Instead, you need to wrap function values in a callable class, e.g., from functools import partial class Implementation(CallableEnum): FOO = partial(foo) BAR = partial(bar) This is OK, but definitely uglier and more error prone than necessary. It's easy to forget to add a partial, which results in an accidental method declaration. It would be nice to have a CallableEnum class that works like Enum, but adds the __call_ method and doesn't allow defining any new methods: all functions assigned in the class body become Enum instances instead of methods. I would write this myself and throw it up on pypi, but there doesn't seem to be any way to do this short of delving into the guts of enum.EnumMeta <https://github.com/python/cpython/blob/0dc5c3169dcd4853612d11ed8c92b12fa210…>. Given that I think that others will also find this pattern useful, rather than forking the enum module I would like to add this into the standard library instead. Note: this proposal would also allow defining enum values using "def" inside the Enum body. But I think this is actually pretty clean, aside from general enum confusion over the fact that Implementation.FOO is Enum instance, not the method. class Implementation(CallableEnum): def FOO(...): ... def BAR(...): ... [View Less]

3 2

"import me" to display some summary of the current python installation
by Kamal Mustafa April 12, 2017

April 12, 2017

On the same theme with "import this" and "import antigravity", "import me" I think can be both some fun way to introduce Python and also offer a helpful functionality when troubleshooting python installation with someone new to the language. Currently, we have to use some myriad approach like "show me the output of `which python`" to know the basic info of the python installation someone having trouble or asking question.

7 11

Proposal: making __str__ count in time's class
by Francesco Franchina April 9, 2017

April 9, 2017

Hello everyone, I'm shortly writing to you about a reflection I lately made upon the current functioning of __str__ for the time's class. Before expressing my thought and proposal, I want to make sure we all agree on a simple and clear fact: the __str__ magic method is used to give a literal and human-readable representation to the object (unlike __repr__). Generally this is true across the python panorama. It's not true for the time class, for example. *>>> import time>>&… [View More]

4 3

Re: [Python-ideas] Make `import a.b.c as m` is equivalent to `m = sys.modules['a.b.c']`
by Victor Varvariuc April 9, 2017

April 9, 2017

The issue: http://bugs.python.org/issue30024 <http://bugs.python.org/issue30024> > On 9 Apr 2017, at 10:05, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > On 9 April 2017 at 16:13, Victor Varvariuc <victor.varvariuc(a)gmail.com> wrote: >> I think there are can be several ways to achieve the result. >> >> The question is if what I've presented is a valid concern. I think this would make the language more consistent and help reducing number of … [View More]

1 0

Re: [Python-ideas] Make `import a.b.c as m` is equivalent to `m = sys.modules['a.b.c']`
by Victor Varvariuc April 9, 2017

April 9, 2017

Thanks for the reply. CC'ing the mailing list. > On 9 Apr 2017, at 10:05, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > On 9 April 2017 at 16:13, Victor Varvariuc <victor.varvariuc(a)gmail.com> wrote: >> I think there are can be several ways to achieve the result. >> >> The question is if what I've presented is a valid concern. I think this would make the language more consistent and help reducing number of errors related to import without breaking … [View More]

1 0

Make `import a.b.c as m` is equivalent to `m = sys.modules['a.b.c']`
by Victor Varvariuc April 8, 2017

April 8, 2017

Hi there. I asked a question <http://stackoverflow.com/questions/41845671/import-as-in-python-3> on Stackoverflow: (Pdb) import brain.utils.mail (Pdb) import brain.utils.mail as mail_utils *** AttributeError: module 'brain.utils' has no attribute 'mail' I always thought that import a.b.c as m is roughly equivalent to m = sys.modules['a.b.c']. Why AttributeError? Python 3.6 I was pointed out <http://stackoverflow.com/a/24968941/248296> that this is a somewhat weird behavior of … [View More]

2 1

Relative import of self or parent package?
by C Anthony Risinger April 8, 2017

April 8, 2017

I occasionally want to do something like this: import .. as parent or: import . as self The pattern is a little less useful for namespace packages (which I've been trying to use almost exclusively, along with relative imports) than it is for __init__.py based packages, but there is still some utility. Is there a reason one can't import dots directly and bind them to a name? I did not find any text around this and the closest I've come up with is: from .. import __name__ as … [View More]

2 1

What about regexp string litterals : re".*" ?
by Simon D. April 3, 2017

April 3, 2017

Hello, After some french discussions about this idea, I subscribed here to suggest adding a new string litteral, for regexp, inspired by other types like : u"", r"", b"", br"", f""… The regexp string litteral could be represented by : re"" It would ease the use of regexps in Python, allowing to have some regexp litterals, like in Perl or JavaScript. We may end up with an integration like : >>> import re >>> if re".k" in 'ok': ... print "ok" ok >>> … [View More]

14 24

pathlib.Path.walk
by Michel Desmoulin April 3, 2017

April 3, 2017

Like os.walk, but from a Path instance. We have Path.iterdir, but it's not recursive. Which you use either os.scandir, or os.walk. In any case, you end up doing: import os import pathlib directory = pathlib.Path(get_dir()) # do things with directory for root, dirs, files os.walk(directory): root = Path(root) for f in files: f = root / f # do something with file Which is not that bad, but you waste a lot of time discovering how to do that since you look first … [View More]

2 2

Introduce BaseTimeoutError
by Ram Rachum April 3, 2017

April 3, 2017

Today I got burned because I had code that did this: except TimeoutError: When it should have done this: except socket.timeout: There's also another timeout error class in asyncio. Initially I thought, why not make them all use the same exception class? But Guido objects to that: I considered this, and decided against unifying the two TimeoutErrors. First the builtin TimeoutError is specifically a subclass of OSError representing the case where errno is ETIMEDOUT. But asyncio.… [View More]

5 4

Construct a matrix from a list: matrix multiplication
by Chris Angelico April 1, 2017

April 1, 2017

This keeps on coming up in one form or another - either someone multiplies a list of lists and ends up surprised that they're all the same, or is frustrated with the verbosity of the alternatives. Can we use the matmul operator for this? class List(list): def __matmul__(self, other): return [copy.copy(x) for x in self for _ in range(other)] >>> x = List([[0]*4]) @ 2 >>> x [[0, 0, 0, 0], [0, 0, 0, 0]] >>> x[0][0] = 1 >>> x [[1, 0, 0, 0], [0, 0, … [View More]

5 5

Re: [Python-ideas] unify usage of mutable and immutable objects
by 语言破碎处 April 1, 2017

April 1, 2017

> To me, 'pop' implies mutation. Tuples do not have a pop method, and it > is not obvious to me that either tuples or frozensets should. What are > the use cases that are not better served by converting to list or set? > -- > Terry Jan Reedy 1) coverting to set or list is O(n) in time 2) if I have to keep the old copy, standard set solution will be O(n) both in time and space! working examples: 1) priority queue: insert and pop occur 2) share immutable data … [View More]

3 3