Python-ideas September 2020

python-ideas@python.org

87 participants
51 discussions

Allow arbitrary expressions in decorator syntax
by Paolo Lammens 19 Sep '20

19 Sep '20

The current grammar for decorators is decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE This restricts the decorator syntax to dotted names with an optional call with an argument list: @spam.eggsdef f(): ... @ham('foo, 'bar')def g(): ... I love the amount of generality and abstraction in Python, but I think this is a place where there could be more of that: I think arbitrary expressions should be allowed; the value of that expression is what gets used as the decorator, using its *call* method (raising TypeError if the object is not callable). This would allow things like: @(lambda x: x)def f(): ... (this currently raises a SyntaxError). Note that I didn’t come up with this because I have a particular application in mind, it just irks me that this isn’t possible in Python :-) ! Here is a perhaps slightly more stimulating example: @(lambda factory: factory())def foo(): # this is a one-use factory for real_foo call_count = 0 # essentially a static variable def real_foo(): nonlocal call_count call_count += 1 return call_count return real_foo This is more a “purity” thing than a “practicality” thing, but I don’t think it has any negative effects on practicality or on the usual use of decorators.

6 10

Magic attribute for attribute names retrieving
by Joseph Perez 18 Sep '20

18 Sep '20

A lot of libraries use string for attribute names to do some "dynamic" things. A typical example are SQLAlchemy [validators](https://docs.sqlalchemy.org/en/13/orm/mapped_attributes.html#si…: ```python from sqlalchemy.orm import validates class EmailAddress(Base): __tablename__ = 'address' id = Column(Integer, primary_key=True) email = Column(String) @validates('email') # Here def validate_email(self, key, address): assert '@' in address return address ``` In this example of SQLAlchemy documentation, email validator use `"email"` string in order to associate the validator to the `email` column. However this dynamic things don't play well with all static tools like linters, but especially IDE (for usages finding, navigation, refactoring, etc.), and of course type checkers. This issue could be solved with a "magic attribute" `__attrs__` (name can be discussed), used the following way: ```python @dataclass class Foo: bar: int foo = Foo(0) assert getattr(foo, Foo.__attrs__.bar) == 0 assert getattr(foo, foo.__attrs__.bar) == 0 ``` To make it usable in class declaration, the `__attrs__` symbol should be added to class declaration namespace: ```python class Foo: bar: int @validator(__attrs__.bar) def validate(self): ... ``` No check would be done by `__attrs__`, they are let to linters which would integrate this language feature and check for the presence of the attribute in the class/instance. And that becomes more interesting because type checkers could use this feature to check dynamic attribute retrieving. A special type `Attribute[Owner, T]` could be defined to be used by type checkers such as `getattr` signature become: ``` # default parameter omitted for concision def getattr(obj: Owner, attr: Attribute[Owner, T], /) -> T: ... ``` (of course, `getattr` can still be used with strings, as the relation between `Attribute` and `str` is explained later) It could allow to add typing to function like the following: ```python Key = TypeVar("Key", bound=Hashable) def dict_by_attr(elts: Collection[T], key_attr: Attribute[T, Key]) -> Mapping[Key, T]: return {getattr(elt, key_attr): elt for elt in elts} ``` Concerning the implementation of this feature would be very straightforward, `__attrs__` being defined as a instance of a class: ```python class Attrs: __slots__ = [] def __getattribute__(self, name): return name ``` Thus, `Foo.__attrs__.bar` would be simply equal to `"bar"`; `Attribute` would be a special type, but backed by a `str`. hence there is no need to modify `getattr` implementation or existing code using it. `Attribute` type should then be a kind of `_SpecialForm`, compatible with string by the "relation" `str` <=> `Attribute[Any, Any]` The only modifications in the langage would be to add the `Attrs` class, an `__attrs__` field to `type` and in class definition namespace when it is evaluated. The rest of the work (checking the attribute presence, type checking with `Attribute[Owner, T]`) should be done by external tools (Pycharm, Mypy, etc.) to handle the feature. To sum up, `__attrs__` would be a pseudo-static wrapper to attribute name retrieving in order to benefit of static tools (refactoring, usages finding, navigation, type checking), with a straightforward and backward compatible implementation. And I don't see that as a "niche" feature, because a lot of libraries could actually benefit from it (and SQLAlchemy, Pydantic, etc. are not a small libraries). Joseph

3 5

__typing__: new top level special variable to indicate typing enabled
by rea.aft＠gmail.com 18 Sep '20

18 Sep '20

In short, the idea is simple to add a special variable, available in the global scope, so you can do this: if __typing__: import typing import another_heavy_typing_lib It has similar semantics as typing.TYPE_CHECKING, but it would be available in any context without the need to import anything, just like __name__, __package__ or __loader__. By default, it would be false (like False ou None). When running static checkers and related tools it would be a true value. Additionally, It would be able to be set (to some value than the default), in the command line invocation, like `-X typing=someval` or `-X typing`. Maybe some environment variable too, like PYTHONTYPING. And leaving it as writable would allow some tool to change it in the runtime. The name of it could be anything, the `typing` suggestion is just lack of creativity. The implementation should be pretty straightforward, it wouldn't have any impact on existing users, performance, etc. And it would allow to use static typing in a slight more confident way in regards of worrying with any risk or penalty about it.

1 0

Allow required parameters after optional in dataclasses
by Laurie Opperman 18 Sep '20

18 Sep '20

I would like to do evil and nasty things with my dataclasses: inherit and extend them. In particular, I want to allow non-defaulted fields (ie required parameters to `__init__`) to be specified after defaulted fields (ie optional parameters), specifically in sub-classes (both parent and children are usable dataclasses). This is simply to reduce re-definition of fields between similar types (I wasn't able to come up with an elegant design using composition for my use-case). The technical problem with allowing this currently is that we can't have required positional arguments after optional arguments in the dataclass's `__init__`, but I have a PR which will make those required arguments keyword-only. I did some searching, but I wasn't able to find much discussion on this decision in the initial implementation of `dataclasses`. Bug tracker: https://bugs.python.org/issue36077 GitHub PR: https://github.com/python/cpython/pull/17322 Discourse link (currently void of discussion): https://discuss.python.org/t/make-succeeding-non-default-init-parameters-ke… Laurie

1 0

Re: f-strings as assignment targets
by Alexis Masson 18 Sep '20

18 Sep '20

> TL;DR: I propose the following behavior: > > >>> s = "She turned me into a newt." > >>> f"She turned me into a {animal}." = s > >>> animal > 'newt' > > >>> f"A {animal}?" = s > Traceback (most recent call last): > File "<pyshell#2>", line 1, in <module> > f"A {animal}?" = s > ValueError: f-string assignment target does not match 'She turned me into a newt.' > > >>> f"{hh:d}:{mm:d}:{ss:d}" = "11:59:59" > >>> hh, mm, ss > (11, 59, 59) I do really like the idea, but assigning to a literal really makes it feel wrong... I think the best way to implement this would be through a str method that would return a dict (so that variables don't "magically" appear out of nowhere). Something like : >>> "She turned me into a newt.".parse("She turned me into a {animal}.") {"animal":"newt"} This, in addition with locals().update(_), feels much better to me. Furthermore, it would allow other string-like classes, such as bytes or bytearray, to use that feature. Note that this modified version uses regular strings, not f-strings, but is that a loss ? It may even be a good thing, as f-strings are about formatting; here, it's parsing we're concerned about. We lose the invariant with = and ==, but only just. Second point, the type conversion : I think that should be left to the user on a second, separate step. There's just too many types that would need support, and we haven't even discussed user-defined types yet ! I think we should keep it simple, and stick to strings. Additionnally ,supporting !r as eval(...) looks rad to my hyped self, but it overlooks a major security risk (see all the stuff that's been discussed about pickle being unsafe). I don't think such an unsafe feature should be allowed at the core of Python. Anyway, the feature overall looks promising ! Alexis

4 6

Re: Python-ideas Digest, Vol 166, Issue 76
by Alexis Masson 17 Sep '20

17 Sep '20

> On Fri, Sep 18, 2020 at 2:13 AM Alexis Masson<a.masson555(a)ntymail.com> wrote: >> This, in addition with locals().update(_), feels much better to me. Furthermore, it would allow other string-like classes, such as bytes or bytearray, to use that feature. >> > But locals().update() isn't a supported operation, except in the > situation where locals() is globals(). So what you're suggesting would > work fine in the REPL but not in any production usage. > > ChrisA That surprises me. I put a quick test to check : > def f() : > print(locals()) > locals().update(dict(a=3)) > print(locals()) > > f() prints : > ================= RESTART: ****/test.py ================ > {} > {'a': 3} > >>> So maybe the specs don't force it, but under the current implementation, it seems to work. I agree that it's bad practice anyway; a correct solution to the original question might surely involve something akin to :for key, value in parsed.items() : > for key, value in parsed.items() : > > exec(f"{key} = {value}") > I like the idea of needing to format a string in the process of parsing another ;) Anyway, back on topic !

4 6

Re: A shortcut to load a JSON file into a dict : json.loadf
by William Pickard 17 Sep '20

17 Sep '20

os.fspath exists for a reason.

1 0

Decorators for class non function properties
by redradist＠gmail.com 14 Sep '20

14 Sep '20

Hi all, Seems like this topic was previously raised, but what if we add possibility to decorate non function properties in class: ```python class Neuron: @linear_activation activation ```

17 71

Access (ordered) dict by index; insert slice
by Hans Ginzel 14 Sep '20

14 Sep '20

Date: Fri, 26 Jun 2020 18:47:44 +0200 From: Hans Ginzel <hans(a)matfyz.cz> To: Hans Ginzel <hans(a)artax.karlin.mff.cuni.cz> Subject: Access (ordered) dict by index; insert slice Hello, thank you for making dict ordered. Is it planned to access key,value pair(s) by index? See https://stackoverflow.com/a/44687752/2556118 for example. Both for reading and (re)writing? Is it planned to insert pair(s) on exact index? Or generally to slice? See splice() in Perl, https://perldoc.perl.org/functions/splice.html. Use case: Represent database table metadata (columns). It is useful as to access columns both by name and by index as to insert column on specific position, https://dev.mysql.com/doc/refman/8.0/en/alter-table.html, “ALTER TABLE ADD COLUMN [FIRST |AFTER col]” (consider default order or table storage size optimisation by aligning). Thank you in advance, Hans PS1: Named tuples cannot be used, are immutable. PS2: See https://metacpan.org/pod/perlref#Pseudo-hashes:-Using-an-array-as-a-hash

32 182

Curious : Why staticmethod if classmethods can do everything a static method can?
by The Nomadic Coder 14 Sep '20

14 Sep '20

This question is to further my understanding of the internals. It seems to me that a classmethod can do everything a staticmethod can, and additionally is not limited by inheritance. Why does python have them as two different constructs?

12 15

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas September 2020