collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict
currently dicts have insert-or-update semantics, e.g.:
dict([((), 1), ((), 2)]) {(): 2}
this is fine. however, in many cases insert or insert-or-ignore are more useful: (hypothetical examples)
InsertOrIgnoreDict([((), 1), ((), 2)]) {(): 1}
InsertDict([((), 1), ((), 2)]) Traceback: [...] ValueError: key exists
these are useful when dealing with custom config file formats, config overrides, among other things. additionally we could also get an UpdateDict, a *view* into a dict that only allows updating existing entries:
a = {'a': 1} b = {'a': 2, 'b': 3} UpdateDict(a).update(b) a {'a': 2}
but I don't see an obvious usefulness to this one. it'd just be for consistency with other SQL-esque dict wrappers (like the above InsertOrIgnoreDict and InsertDict).
I just tried playing with this idea: from collections import UserDict class InsertOrIgnoreDict(UserDict): __setitem__ = UserDict.setdefault print(InsertOrIgnoreDict([(1, 2), (3, 4)])) It caused an infinite chain of exceptions: Traceback (most recent call last): File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", line 845, in setdefault return self[key] File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", line 1003, in __getitem__ raise KeyError(key) KeyError: 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", line 845, in setdefault return self[key] File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", line 1003, in __getitem__ raise KeyError(key) KeyError: 1 During handling of the above exception, another exception occurred: ... I thought that was amusing. Anyway, one could come up with infinitely many variations of potentially useful semantics. Why include any of them in the language instead of letting users implement what they need? On Thu, Apr 16, 2020 at 2:37 PM Soni L. <fakedme+py@gmail.com> wrote:
currently dicts have insert-or-update semantics, e.g.:
dict([((), 1), ((), 2)]) {(): 2}
this is fine. however, in many cases insert or insert-or-ignore are more useful: (hypothetical examples)
InsertOrIgnoreDict([((), 1), ((), 2)]) {(): 1}
InsertDict([((), 1), ((), 2)]) Traceback: [...] ValueError: key exists
these are useful when dealing with custom config file formats, config overrides, among other things.
additionally we could also get an UpdateDict, a *view* into a dict that only allows updating existing entries:
a = {'a': 1} b = {'a': 2, 'b': 3} UpdateDict(a).update(b) a {'a': 2}
but I don't see an obvious usefulness to this one. it'd just be for consistency with other SQL-esque dict wrappers (like the above InsertOrIgnoreDict and InsertDict). _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7PCWDO... Code of Conduct: http://python.org/psf/codeofconduct/
what about a dict.setdefaults (note the s) that takes in an iterable or a dict, and uses insert-or-ignore behaviour? (unfortunately a lot of iterables and all generators are hashable, and we can't break that.) I could really use something like that tbh. On 2020-04-16 9:47 a.m., Alex Hall wrote:
I just tried playing with this idea:
from collections import UserDict
class InsertOrIgnoreDict(UserDict): __setitem__ = UserDict.setdefault
print(InsertOrIgnoreDict([(1, 2), (3, 4)]))
It caused an infinite chain of exceptions:
Traceback (most recent call last): File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", line 845, in setdefault return self[key] File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", line 1003, in __getitem__ raise KeyError(key) KeyError: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", line 845, in setdefault return self[key] File "/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", line 1003, in __getitem__ raise KeyError(key) KeyError: 1
During handling of the above exception, another exception occurred:
...
I thought that was amusing.
Anyway, one could come up with infinitely many variations of potentially useful semantics. Why include any of them in the language instead of letting users implement what they need?
On Thu, Apr 16, 2020 at 2:37 PM Soni L. <fakedme+py@gmail.com <mailto:fakedme%2Bpy@gmail.com>> wrote:
currently dicts have insert-or-update semantics, e.g.:
>>> dict([((), 1), ((), 2)]) {(): 2}
this is fine. however, in many cases insert or insert-or-ignore are more useful: (hypothetical examples)
>>> InsertOrIgnoreDict([((), 1), ((), 2)]) {(): 1}
>>> InsertDict([((), 1), ((), 2)]) Traceback: [...] ValueError: key exists
these are useful when dealing with custom config file formats, config overrides, among other things.
additionally we could also get an UpdateDict, a *view* into a dict that only allows updating existing entries:
>>> a = {'a': 1} >>> b = {'a': 2, 'b': 3} >>> UpdateDict(a).update(b) >>> a {'a': 2}
but I don't see an obvious usefulness to this one. it'd just be for consistency with other SQL-esque dict wrappers (like the above InsertOrIgnoreDict and InsertDict). _______________________________________________ Python-ideas mailing list -- python-ideas@python.org <mailto:python-ideas@python.org> To unsubscribe send an email to python-ideas-leave@python.org <mailto:python-ideas-leave@python.org> https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7PCWDO... Code of Conduct: http://python.org/psf/codeofconduct/
On 2020-04-16 11:43 a.m., Steven D'Aprano wrote:
On Thu, Apr 16, 2020 at 11:17:33AM -0300, Soni L. wrote:
what about a dict.setdefaults (note the s)
Do you have any idea of how much confusion and many silent errors will be caused by having methods setdefault and setdefaults on the same class?
we can't break setdefault (particularly for tuple keys), do you have a better idea?
On Thu, Apr 16, 2020 at 01:11:46PM -0300, Soni L. wrote:
we can't break setdefault (particularly for tuple keys), do you have a better idea?
Do nothing? I don't have to suggest a better idea, since its not me proposing a change. I don't think any change is needed. It is up to you to firstly justify that a change is needed, and only then justify a specific response to that need. An anecdote... when I first discovered Python back in 1.5 days, and specifically the dict.update method, practically the first thing I did was wonder why update replaced existing keys. What if I wanted it to keep the existing key, or raise an exception? So I immediately wrote a pair of functions to do both those things. It's been close to 20 years now and so long as I can remember, I have never, not once, used either of those variants I don't even know whether I even still have them (not that it would be hard to recreate them). If you personally would use them, I'm happy for you. But unless there is likely to be either moderate or widespread use among the community, or at least some uses in the stdlib, those functions belong in your own personal toolbox, not the stdlib. -- Steven
Steven D'Aprano wrote:
Do nothing?
I don't have to suggest a better idea, since its not me proposing a change. I don't think any change is needed. It is up to you to firstly justify that a change is needed, and only then justify a specific response to that need.
[snip]
If you personally would use them, I'm happy for you. But unless there is likely to be either moderate or widespread use among the community, or at least some uses in the stdlib, those functions belong in your own personal toolbox, not the stdlib.
+1 to doing nothing here, at least based on the current proposal. Soni, I would strongly recommend spending some more time to come up with specific, concrete use cases for feature proposals in the future (or in this thread). Preferably with some detailed "before and after" examples using *non-trivial* real world code, and some explanations as to why you think it should be included in the standard library instead of just something implemented locally or in a 3rd party package. Without doing the above, it's very likely going to end up being an infinite loop of seeing the same repeated counter-arguments on your proposals. If those points can't be answered and defended, it's highly unlikely the proposal would be accepted in the first place; regardless of how interesting the ideas may seem. On Thu, Apr 16, 2020 at 10:37 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Apr 16, 2020 at 01:11:46PM -0300, Soni L. wrote:
we can't break setdefault (particularly for tuple keys), do you have a better idea?
Do nothing?
I don't have to suggest a better idea, since its not me proposing a change. I don't think any change is needed. It is up to you to firstly justify that a change is needed, and only then justify a specific response to that need.
An anecdote... when I first discovered Python back in 1.5 days, and specifically the dict.update method, practically the first thing I did was wonder why update replaced existing keys. What if I wanted it to keep the existing key, or raise an exception? So I immediately wrote a pair of functions to do both those things.
It's been close to 20 years now and so long as I can remember, I have never, not once, used either of those variants I don't even know whether I even still have them (not that it would be hard to recreate them).
If you personally would use them, I'm happy for you. But unless there is likely to be either moderate or widespread use among the community, or at least some uses in the stdlib, those functions belong in your own personal toolbox, not the stdlib.
-- Steven _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SPY2DR... Code of Conduct: http://python.org/psf/codeofconduct/
On Apr 16, 2020, at 07:19, Soni L. <fakedme+py@gmail.com> wrote:
what about a dict.setdefaults (note the s) that takes in an iterable or a dict, and uses insert-or-ignore behaviour? (unfortunately a lot of iterables and all generators are hashable, and we can't break that.)
I could really use something like that tbh.
Then tell us the use cases that you have in mind. I can think of lots of common uses for “override this dict with that one”, such as the typical “defaults, config file, environment, argv” thing that most command line tools do. But I can already write that, in idiomatic and readable code (and in fact with variations—update, |, or ChainMap); why would I want to be able to write the same thing in another way that’s conceptually backward, probably less efficient, and doesn’t work in Python 3.8? If the issue is just that you have those default values in a list of pairs rather than a dict, it’s trivial to just construct a dict from that. Maybe there are good use cases where the usual way would feel backward or wrong and setdefaults would be nicer. But if so, you need to show some. And they have to be pretty common. You can obviously already write that as `for k, v in defaults: d.setdefault(k, v)`, which is acceptable if it comes up once a year, but demands improvement if people are (or should be, if they only realized it) writing it all over the place. (If the API were obviously symmetrical, the bar might be a lot lower. See the Haskell dict update operators—I’ll bet I’ve often used the “backward” ones and not even realized it, because they don’t look backward or unusual, and because Haskell in general encourages that kind of symmetrical thinking everywhere, to the extent that even “pass this arg to that function” is almost as commonplace as “call this function with that arg”. But I don’t think Python could get there from here, even if it were a good idea.) More generally, it seems like you think Python needs to include everything anyone can think of that might theoretically be useful somewhere. But it’s designed to only include (almost) everything people need regularly and can’t just trivially and build for themselves, because aiming for that sweet spot keeps Python discoverable and Python code readable—and similarly for Rust, Swift, Ruby, and other languages that attempt to go that way, even if none of them (including Python) pulls it off perfectly. A kitchen sink language is even worse than a no-batteries-included language—it isn’t just harder to design and implement, it’s harder to use. You have more docs to wade through to find anything, more subtle distinctions to think through to decide, more stuff to remember when you come back to the language after time spent doing other stuff, more trouble reading code written by other people who internalized a different subset of the kitchen sink, etc.
On 2020-04-16 2:42 p.m., Andrew Barnert wrote:
On Apr 16, 2020, at 07:19, Soni L. <fakedme+py@gmail.com> wrote:
what about a dict.setdefaults (note the s) that takes in an iterable or a dict, and uses insert-or-ignore behaviour? (unfortunately a lot of iterables and all generators are hashable, and we can't break that.)
I could really use something like that tbh.
Then tell us the use cases that you have in mind.
I can think of lots of common uses for “override this dict with that one”, such as the typical “defaults, config file, environment, argv” thing that most command line tools do. But I can already write that, in idiomatic and readable code (and in fact with variations—update, |, or ChainMap); why would I want to be able to write the same thing in another way that’s conceptually backward, probably less efficient, and doesn’t work in Python 3.8? If the issue is just that you have those default values in a list of pairs rather than a dict, it’s trivial to just construct a dict from that.
Maybe there are good use cases where the usual way would feel backward or wrong and setdefaults would be nicer. But if so, you need to show some.
And they have to be pretty common. You can obviously already write that as `for k, v in defaults: d.setdefault(k, v)`, which is acceptable if it comes up once a year, but demands improvement if people are (or should be, if they only realized it) writing it all over the place.
(If the API were obviously symmetrical, the bar might be a lot lower. See the Haskell dict update operators—I’ll bet I’ve often used the “backward” ones and not even realized it, because they don’t look backward or unusual, and because Haskell in general encourages that kind of symmetrical thinking everywhere, to the extent that even “pass this arg to that function” is almost as commonplace as “call this function with that arg”. But I don’t think Python could get there from here, even if it were a good idea.)
More generally, it seems like you think Python needs to include everything anyone can think of that might theoretically be useful somewhere. But it’s designed to only include (almost) everything people need regularly and can’t just trivially and build for themselves, because aiming for that sweet spot keeps Python discoverable and Python code readable—and similarly for Rust, Swift, Ruby, and other languages that attempt to go that way, even if none of them (including Python) pulls it off perfectly. A kitchen sink language is even worse than a no-batteries-included language—it isn’t just harder to design and implement, it’s harder to use. You have more docs to wade through to find anything, more subtle distinctions to think through to decide, more stuff to remember when you come back to the language after time spent doing other stuff, more trouble reading code written by other people who internalized a different subset of the kitchen sink, etc.
I prefer putting things in order of priority in my ConfigManagers and RepoListManagers and whatnot. mainly because in the RepoListManager, I keep the ConfigManager as the first thing in the list. wouldn't be fun if I had to keep it last, as I need to modify that list regularly. I also just concatenate the iterators of all those configs and repo lists together :v
participants (5)
-
Alex Hall -
Andrew Barnert -
Kyle Stanley -
Soni L. -
Steven D'Aprano