Python-ideas September 2020

python-ideas@python.org

87 participants
50 discussions

Magic attribute for attribute names retrieving

by Joseph Perez

A lot of libraries use string for attribute names to do some "dynamic" things. A typical example are SQLAlchemy [validators](https://docs.sqlalchemy.org/en/13/orm/mapped_attributes.html#si…: ```python from sqlalchemy.orm import validates class EmailAddress(Base): __tablename__ = 'address' id = Column(Integer, primary_key=True) email = Column(String) @validates('email') # Here def validate_email(self, key, address): assert '@' in address return address ``` In this example of SQLAlchemy documentation, email validator use `"email"` string in order to associate the validator to the `email` column. However this dynamic things don't play well with all static tools like linters, but especially IDE (for usages finding, navigation, refactoring, etc.), and of course type checkers. This issue could be solved with a "magic attribute" `__attrs__` (name can be discussed), used the following way: ```python @dataclass class Foo: bar: int foo = Foo(0) assert getattr(foo, Foo.__attrs__.bar) == 0 assert getattr(foo, foo.__attrs__.bar) == 0 ``` To make it usable in class declaration, the `__attrs__` symbol should be added to class declaration namespace: ```python class Foo: bar: int @validator(__attrs__.bar) def validate(self): ... ``` No check would be done by `__attrs__`, they are let to linters which would integrate this language feature and check for the presence of the attribute in the class/instance. And that becomes more interesting because type checkers could use this feature to check dynamic attribute retrieving. A special type `Attribute[Owner, T]` could be defined to be used by type checkers such as `getattr` signature become: ``` # default parameter omitted for concision def getattr(obj: Owner, attr: Attribute[Owner, T], /) -> T: ... ``` (of course, `getattr` can still be used with strings, as the relation between `Attribute` and `str` is explained later) It could allow to add typing to function like the following: ```python Key = TypeVar("Key", bound=Hashable) def dict_by_attr(elts: Collection[T], key_attr: Attribute[T, Key]) -> Mapping[Key, T]: return {getattr(elt, key_attr): elt for elt in elts} ``` Concerning the implementation of this feature would be very straightforward, `__attrs__` being defined as a instance of a class: ```python class Attrs: __slots__ = [] def __getattribute__(self, name): return name ``` Thus, `Foo.__attrs__.bar` would be simply equal to `"bar"`; `Attribute` would be a special type, but backed by a `str`. hence there is no need to modify `getattr` implementation or existing code using it. `Attribute` type should then be a kind of `_SpecialForm`, compatible with string by the "relation" `str` <=> `Attribute[Any, Any]` The only modifications in the langage would be to add the `Attrs` class, an `__attrs__` field to `type` and in class definition namespace when it is evaluated. The rest of the work (checking the attribute presence, type checking with `Attribute[Owner, T]`) should be done by external tools (Pycharm, Mypy, etc.) to handle the feature. To sum up, `__attrs__` would be a pseudo-static wrapper to attribute name retrieving in order to benefit of static tools (refactoring, usages finding, navigation, type checking), with a straightforward and backward compatible implementation. And I don't see that as a "niche" feature, because a lot of libraries could actually benefit from it (and SQLAlchemy, Pydantic, etc. are not a small libraries). Joseph

1 year

__typing__: new top level special variable to indicate typing enabled

by rea.aft＠gmail.com

In short, the idea is simple to add a special variable, available in the global scope, so you can do this: if __typing__: import typing import another_heavy_typing_lib It has similar semantics as typing.TYPE_CHECKING, but it would be available in any context without the need to import anything, just like __name__, __package__ or __loader__. By default, it would be false (like False ou None). When running static checkers and related tools it would be a true value. Additionally, It would be able to be set (to some value than the default), in the command line invocation, like `-X typing=someval` or `-X typing`. Maybe some environment variable too, like PYTHONTYPING. And leaving it as writable would allow some tool to change it in the runtime. The name of it could be anything, the `typing` suggestion is just lack of creativity. The implementation should be pretty straightforward, it wouldn't have any impact on existing users, performance, etc. And it would allow to use static typing in a slight more confident way in regards of worrying with any risk or penalty about it.

1 year

Allow required parameters after optional in dataclasses

by Laurie Opperman

I would like to do evil and nasty things with my dataclasses: inherit and extend them. In particular, I want to allow non-defaulted fields (ie required parameters to `__init__`) to be specified after defaulted fields (ie optional parameters), specifically in sub-classes (both parent and children are usable dataclasses). This is simply to reduce re-definition of fields between similar types (I wasn't able to come up with an elegant design using composition for my use-case). The technical problem with allowing this currently is that we can't have required positional arguments after optional arguments in the dataclass's `__init__`, but I have a PR which will make those required arguments keyword-only. I did some searching, but I wasn't able to find much discussion on this decision in the initial implementation of `dataclasses`. Bug tracker: https://bugs.python.org/issue36077 GitHub PR: https://github.com/python/cpython/pull/17322 Discourse link (currently void of discussion): https://discuss.python.org/t/make-succeeding-non-default-init-parameters-ke… Laurie

1 year

Re: f-strings as assignment targets

by Alexis Masson

> TL;DR: I propose the following behavior: > > >>> s = "She turned me into a newt." > >>> f"She turned me into a {animal}." = s > >>> animal > 'newt' > > >>> f"A {animal}?" = s > Traceback (most recent call last): > File "<pyshell#2>", line 1, in <module> > f"A {animal}?" = s > ValueError: f-string assignment target does not match 'She turned me into a newt.' > > >>> f"{hh:d}:{mm:d}:{ss:d}" = "11:59:59" > >>> hh, mm, ss > (11, 59, 59) I do really like the idea, but assigning to a literal really makes it feel wrong... I think the best way to implement this would be through a str method that would return a dict (so that variables don't "magically" appear out of nowhere). Something like : >>> "She turned me into a newt.".parse("She turned me into a {animal}.") {"animal":"newt"} This, in addition with locals().update(_), feels much better to me. Furthermore, it would allow other string-like classes, such as bytes or bytearray, to use that feature. Note that this modified version uses regular strings, not f-strings, but is that a loss ? It may even be a good thing, as f-strings are about formatting; here, it's parsing we're concerned about. We lose the invariant with = and ==, but only just. Second point, the type conversion : I think that should be left to the user on a second, separate step. There's just too many types that would need support, and we haven't even discussed user-defined types yet ! I think we should keep it simple, and stick to strings. Additionnally ,supporting !r as eval(...) looks rad to my hyped self, but it overlooks a major security risk (see all the stuff that's been discussed about pickle being unsafe). I don't think such an unsafe feature should be allowed at the core of Python. Anyway, the feature overall looks promising ! Alexis

1 year

Re: Python-ideas Digest, Vol 166, Issue 76

by Alexis Masson

> On Fri, Sep 18, 2020 at 2:13 AM Alexis Masson<a.masson555(a)ntymail.com> wrote: >> This, in addition with locals().update(_), feels much better to me. Furthermore, it would allow other string-like classes, such as bytes or bytearray, to use that feature. >> > But locals().update() isn't a supported operation, except in the > situation where locals() is globals(). So what you're suggesting would > work fine in the REPL but not in any production usage. > > ChrisA That surprises me. I put a quick test to check : > def f() : > print(locals()) > locals().update(dict(a=3)) > print(locals()) > > f() prints : > ================= RESTART: ****/test.py ================ > {} > {'a': 3} > >>> So maybe the specs don't force it, but under the current implementation, it seems to work. I agree that it's bad practice anyway; a correct solution to the original question might surely involve something akin to :for key, value in parsed.items() : > for key, value in parsed.items() : > > exec(f"{key} = {value}") > I like the idea of needing to format a string in the process of parsing another ;) Anyway, back on topic !

1 year

Re: A shortcut to load a JSON file into a dict : json.loadf

by William Pickard

os.fspath exists for a reason.

1 year

Decorators for class non function properties

by redradist＠gmail.com

Hi all, Seems like this topic was previously raised, but what if we add possibility to decorate non function properties in class: ```python class Neuron: @linear_activation activation ```

1 year

Access (ordered) dict by index; insert slice

by Hans Ginzel

Date: Fri, 26 Jun 2020 18:47:44 +0200 From: Hans Ginzel <hans(a)matfyz.cz> To: Hans Ginzel <hans(a)artax.karlin.mff.cuni.cz> Subject: Access (ordered) dict by index; insert slice Hello, thank you for making dict ordered. Is it planned to access key,value pair(s) by index? See https://stackoverflow.com/a/44687752/2556118 for example. Both for reading and (re)writing? Is it planned to insert pair(s) on exact index? Or generally to slice? See splice() in Perl, https://perldoc.perl.org/functions/splice.html. Use case: Represent database table metadata (columns). It is useful as to access columns both by name and by index as to insert column on specific position, https://dev.mysql.com/doc/refman/8.0/en/alter-table.html, “ALTER TABLE ADD COLUMN [FIRST |AFTER col]” (consider default order or table storage size optimisation by aligning). Thank you in advance, Hans PS1: Named tuples cannot be used, are immutable. PS2: See https://metacpan.org/pod/perlref#Pseudo-hashes:-Using-an-array-as-a-hash

1 year

Curious : Why staticmethod if classmethods can do everything a static method can?

by The Nomadic Coder

This question is to further my understanding of the internals. It seems to me that a classmethod can do everything a staticmethod can, and additionally is not limited by inheritance. Why does python have them as two different constructs?

1 year

argparse.py in version 3.8.2 Action=Custom ActionClass needs an option to have methods

by rangapv08＠gmail.com

Hi, Got the argparse() in version 3.8.2 and above the action=CusotmClass can have a user defined custom subclass and override the functionality in the def __init__() method. Along with this can we also have a built in custom method/methods that I can define under this custom Class, so that instead of calling the Action itself I have the option of calling the Action.Method() during the add_argument defintition. Regards Ranga

1 year

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas September 2020