Delete dictionary entry if key exists using -= operator via __isub__()

*Background* It is frequently desirable to delete a dictionary entry if the key exists. It is necessary to check that the key exists or, alternatively, handle a KeyError: for, where `d` is a `dict`, and `k` is a valid hashable key, `del d[k]` raises KeyError if `k` does not exist. Example: ``` if k in d: del d[k] ``` *Idea* Use the `-=` operator with the key as the right operand to delete a dictionary if the key exists. *Demonstration-of-Concept* ``` class DemoDict(dict): def __init__(self, obj): super().__init__(obj) def __isub__(self, other): if other in self: del self[other] return self if __name__ == '__main__': given = {'a': 1, 'b': 2, 'c': 3, 'd': 4} demo = DemoDict(given) demo -= 'c' assert(demo == {'a': 1, 'b': 2, 'd': 4}) ```

You could use the pop method on dictionary (with a default value of None for example) to remove the key only if it exists. The method returns the value associated with this key but you can ignore it. I'm not sure the __isub__ implementation would be easy to understand as it's weird to make a subtraction between a dict and a str. Even sets only support subtraction between sets. (Also you have Counter objects from collections module that implement a kind of subtraction between dicts) Le jeu. 28 avr. 2022 à 15:19, <zmvictor@gmail.com> a écrit :
-- Antoine Rozo

Good points, Antoine! The intent is to mutate the dictionary with no side effects if a key exists — and without the extra code that comes along with a control structure, a try except, or positing a None to throw away. For me, pop() says "give me something." Since the intent is to mutate the dictionary only and not "get something," I'd prefer (i) not to use a function that returns a value and (ii) not to specify a return value None just so that it can be thrown away. Perhaps if pop()'s default return value were None—as it is with, say, get()—I would use pop() as you describe. But pop() doesn't have a default value unless you give it one, and pop() always returns a value. Where does this land with PEP 20? I think the use of pop() as you suggest lands on the implicit side of things and is not as readable: the reader has to ask, "what are we doing with the default value? Oh. Nothing. It's to delete a dict entry." However, pop() with the default value of None is practical, and practicality does beat purity. I agree that operator overloading needs to be consistent and easy to understand, and in that regard -= with the dict key probably does not withstand the PEP 20 tests. Furthermore, in support of your point, the language has moved in the direction of set operations between dicts using operators: for example, + for union of dicts. To satisfy set- and key-based operations, one could overload the augmented arithmetic assignment operators so that -= with a string would delete the entry with a matching key, while -= with a dict would delete the entry with a matching key and value. Again, that may be too recherché or hard-to-explain (cf PEP 20). Perhaps, then, it would be best to leave dict alone and just overload operators in subclasses for specific uses!

On Fri, 29 Apr 2022 at 04:03, Zach Victor <zmvictor@gmail.com> wrote:
Where does this land with PEP 20? I think the use of pop() as you suggest lands on the implicit side of things and is not as readable: the reader has to ask, "what are we doing with the default value? Oh. Nothing. It's to delete a dict entry." However, pop() with the default value of None is practical, and practicality does beat purity.
"Implicit" does not mean "code that I dislike". The pop method is exactly what it appears to be: a way to remove something, with either an error or a default if it's not found. It then returns the thing. Ignoring a function's return value is perfectly normal. There are all manner of functions which you use all the time, and it's not a problem to have them return something you usually don't care about (like f.write() returning how much it wrote). ChrisA

I agree that "implicit" does not mean "code that one dislikes." The intent is "delete entry if key exists." Is that implicit or explicit? Positing a default value to discard it as a return value is, arguably, what makes the intent of that construction implicit. I say "arguably," because that is the material that is under consideration (not whether one likes it).

On Fri, 29 Apr 2022 at 07:15, Zach Victor <zmvictor@gmail.com> wrote:
I agree that "implicit" does not mean "code that one dislikes." The intent is "delete entry if key exists." Is that implicit or explicit?
"[R]emove specified key and return the corresponding value", with a default if there isn't one. Is that explicit enough?
Positing a default value to discard it as a return value is, arguably, what makes the intent of that construction implicit. I say "arguably," because that is the material that is under consideration (not whether one likes it).
You're welcome to create your own function to wrap it up, if you really think that that's a problem: def discard(dict, key): _ignoreme = dict.pop(key, None) del _ignoreme # because it's not explicit enough to just abandon an object Is that an improvement? ChrisA

The original idea was actually not to write function, but thank you for telling me that I am welcome to do so—that is very kind of you and, indeed, welcoming. I don't know whether to consider it an improvement. It is a nice function.

On Thu, Apr 28, 2022 at 01:18:09AM -0000, zmvictor@gmail.com wrote:
Frequently? I don't think I've ever needed to do that. Can you give an example of real code that does this?
The simplest one-liner to delete a key if and only if it exists is with the `pop` method: mydict.pop(key, None) # Ignore the return result. That may not be the most efficient way. I expect that the most efficient way will depend on whether the key is more likely to exist or not: # Not benchmarked, so take my predictions with a pinch of salt. # Probably fastest if the key is usually present. try: del mydict[key] except KeyError: pass # Probably fastest if the key is usually absent. if key in mydict: del mydict[key] So we already have three ways to delete only an existing key from a dict, "optimised" (in some sense) for three scenarios: - key expected to be present; - key expected to be absent; - for convenience (one-liner). With three existing solutions to this problem, it is unlikely that a fourth solution will be blessed by building it into the dict type itself. (Although of course you can add it to your own subclasses.) Especially not when that solution is easily mistaken for something else: d -= 1 # Are we subtracting 1, or deleting key 1? Alone, that is not a fatal problem, but given that there are three other satisfactory solutions to the task of deleting an existing key, even minor or trivial problems push the cost:benefit ratio into the negative. By the way:
If the only purpose of a method is to call super and inherit from its parent class(es), as in the above, then you don't need to define the method at all. Just leave it out, and the parent's `__init__` will be called. -- Steve

On Fri, 29 Apr 2022 at 07:44, Steven D'Aprano <steve@pearwood.info> wrote:
Especially not when that solution is easily mistaken for something else:
d -= 1 # Are we subtracting 1, or deleting key 1?
IMO it would make more sense to spell it as subtraction of two similar types: d -= {1} # supporting sets would be nice d -= {1: ...} If subtraction on dicts is ever supported, I doubt it would allow subtracting one key from the dict - it would make much more sense to subtract another dictionary. But for the single elimination case, pop() is perfect for the job. Oh, and since we're making evidence-free predictions about performance, I'm going to put in my guesses too: "if key in mydict: del mydict[key]" will be slower than pop(), and the try/except could be about comparable, but could be slightly slower or faster. (If we were placing bets, I'd bet on pop() being the fastest of the three overall, but that's a weaker prediction.) Now we just need someone to make the ultimate benchmark so we can find out :) ChrisA

Agree — it is perhaps a bit strange to use an operator for something other than dictionary-to-dictionary operations. And, yes, benchmarking is the only way to know—but I think your hypothesis is a good one.

Can you give an example of real code that does this? In general, parsing JSON data and normalizing it. The existing architecture has microservices consuming PubSub messages for which the data-payload is JSON. In some cases, deleting dictionary keys is convenient (as opposed to, say, building a new dict and dumping it), as a way to prepare it for the downstream consumer.
I should say, there is no inherent need for a new way to delete a dictionary entry. This was merely an "idea" to a list of "ideas"! I appreciate the fair criticism and consideration, along with that of Antoine and Chris. I agree with you that there is probably no need for a fourth solution.

You could use the pop method on dictionary (with a default value of None for example) to remove the key only if it exists. The method returns the value associated with this key but you can ignore it. I'm not sure the __isub__ implementation would be easy to understand as it's weird to make a subtraction between a dict and a str. Even sets only support subtraction between sets. (Also you have Counter objects from collections module that implement a kind of subtraction between dicts) Le jeu. 28 avr. 2022 à 15:19, <zmvictor@gmail.com> a écrit :
-- Antoine Rozo

Good points, Antoine! The intent is to mutate the dictionary with no side effects if a key exists — and without the extra code that comes along with a control structure, a try except, or positing a None to throw away. For me, pop() says "give me something." Since the intent is to mutate the dictionary only and not "get something," I'd prefer (i) not to use a function that returns a value and (ii) not to specify a return value None just so that it can be thrown away. Perhaps if pop()'s default return value were None—as it is with, say, get()—I would use pop() as you describe. But pop() doesn't have a default value unless you give it one, and pop() always returns a value. Where does this land with PEP 20? I think the use of pop() as you suggest lands on the implicit side of things and is not as readable: the reader has to ask, "what are we doing with the default value? Oh. Nothing. It's to delete a dict entry." However, pop() with the default value of None is practical, and practicality does beat purity. I agree that operator overloading needs to be consistent and easy to understand, and in that regard -= with the dict key probably does not withstand the PEP 20 tests. Furthermore, in support of your point, the language has moved in the direction of set operations between dicts using operators: for example, + for union of dicts. To satisfy set- and key-based operations, one could overload the augmented arithmetic assignment operators so that -= with a string would delete the entry with a matching key, while -= with a dict would delete the entry with a matching key and value. Again, that may be too recherché or hard-to-explain (cf PEP 20). Perhaps, then, it would be best to leave dict alone and just overload operators in subclasses for specific uses!

On Fri, 29 Apr 2022 at 04:03, Zach Victor <zmvictor@gmail.com> wrote:
Where does this land with PEP 20? I think the use of pop() as you suggest lands on the implicit side of things and is not as readable: the reader has to ask, "what are we doing with the default value? Oh. Nothing. It's to delete a dict entry." However, pop() with the default value of None is practical, and practicality does beat purity.
"Implicit" does not mean "code that I dislike". The pop method is exactly what it appears to be: a way to remove something, with either an error or a default if it's not found. It then returns the thing. Ignoring a function's return value is perfectly normal. There are all manner of functions which you use all the time, and it's not a problem to have them return something you usually don't care about (like f.write() returning how much it wrote). ChrisA

I agree that "implicit" does not mean "code that one dislikes." The intent is "delete entry if key exists." Is that implicit or explicit? Positing a default value to discard it as a return value is, arguably, what makes the intent of that construction implicit. I say "arguably," because that is the material that is under consideration (not whether one likes it).

On Fri, 29 Apr 2022 at 07:15, Zach Victor <zmvictor@gmail.com> wrote:
I agree that "implicit" does not mean "code that one dislikes." The intent is "delete entry if key exists." Is that implicit or explicit?
"[R]emove specified key and return the corresponding value", with a default if there isn't one. Is that explicit enough?
Positing a default value to discard it as a return value is, arguably, what makes the intent of that construction implicit. I say "arguably," because that is the material that is under consideration (not whether one likes it).
You're welcome to create your own function to wrap it up, if you really think that that's a problem: def discard(dict, key): _ignoreme = dict.pop(key, None) del _ignoreme # because it's not explicit enough to just abandon an object Is that an improvement? ChrisA

The original idea was actually not to write function, but thank you for telling me that I am welcome to do so—that is very kind of you and, indeed, welcoming. I don't know whether to consider it an improvement. It is a nice function.

On Thu, Apr 28, 2022 at 01:18:09AM -0000, zmvictor@gmail.com wrote:
Frequently? I don't think I've ever needed to do that. Can you give an example of real code that does this?
The simplest one-liner to delete a key if and only if it exists is with the `pop` method: mydict.pop(key, None) # Ignore the return result. That may not be the most efficient way. I expect that the most efficient way will depend on whether the key is more likely to exist or not: # Not benchmarked, so take my predictions with a pinch of salt. # Probably fastest if the key is usually present. try: del mydict[key] except KeyError: pass # Probably fastest if the key is usually absent. if key in mydict: del mydict[key] So we already have three ways to delete only an existing key from a dict, "optimised" (in some sense) for three scenarios: - key expected to be present; - key expected to be absent; - for convenience (one-liner). With three existing solutions to this problem, it is unlikely that a fourth solution will be blessed by building it into the dict type itself. (Although of course you can add it to your own subclasses.) Especially not when that solution is easily mistaken for something else: d -= 1 # Are we subtracting 1, or deleting key 1? Alone, that is not a fatal problem, but given that there are three other satisfactory solutions to the task of deleting an existing key, even minor or trivial problems push the cost:benefit ratio into the negative. By the way:
If the only purpose of a method is to call super and inherit from its parent class(es), as in the above, then you don't need to define the method at all. Just leave it out, and the parent's `__init__` will be called. -- Steve

On Fri, 29 Apr 2022 at 07:44, Steven D'Aprano <steve@pearwood.info> wrote:
Especially not when that solution is easily mistaken for something else:
d -= 1 # Are we subtracting 1, or deleting key 1?
IMO it would make more sense to spell it as subtraction of two similar types: d -= {1} # supporting sets would be nice d -= {1: ...} If subtraction on dicts is ever supported, I doubt it would allow subtracting one key from the dict - it would make much more sense to subtract another dictionary. But for the single elimination case, pop() is perfect for the job. Oh, and since we're making evidence-free predictions about performance, I'm going to put in my guesses too: "if key in mydict: del mydict[key]" will be slower than pop(), and the try/except could be about comparable, but could be slightly slower or faster. (If we were placing bets, I'd bet on pop() being the fastest of the three overall, but that's a weaker prediction.) Now we just need someone to make the ultimate benchmark so we can find out :) ChrisA

Agree — it is perhaps a bit strange to use an operator for something other than dictionary-to-dictionary operations. And, yes, benchmarking is the only way to know—but I think your hypothesis is a good one.

Can you give an example of real code that does this? In general, parsing JSON data and normalizing it. The existing architecture has microservices consuming PubSub messages for which the data-payload is JSON. In some cases, deleting dictionary keys is convenient (as opposed to, say, building a new dict and dumping it), as a way to prepare it for the downstream consumer.
I should say, there is no inherent need for a new way to delete a dictionary entry. This was merely an "idea" to a list of "ideas"! I appreciate the fair criticism and consideration, along with that of Antoine and Chris. I agree with you that there is probably no need for a fourth solution.
participants (7)
-
Antoine Rozo
-
Chris Angelico
-
David Mertz, Ph.D.
-
Greg Ewing
-
Steven D'Aprano
-
Zach Victor
-
zmvictor@gmail.com