On Wed, Feb 27, 2019 at 8:50 AM Rhodri James wrote:

On 27/02/2019 16:25, João Matos wrote:

...

I would like to propose that instead of using this (applies to Py3.5 and upwards) dict_a = {**dict_a, **dict_b}

we could use dict_a = dict_a + dict_b

or even better dict_a += dict_b

While I don't object to the idea of concatenating dictionaries, I feel obliged to point out that this last is currently spelled dict_a.update(dict_b)

This is likely to be controversial. But I like the idea. After all, we have `list.extend(x)` ~~ `list += x`. The key conundrum that needs to be solved is what to do for `d1 + d2` when there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`. -- --Guido van Rossum (python.org/~guido)

The key conundrum that needs to be solved is what to do for `d1 + d2` when there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

This would mean that addition, at least in this particular instance, is not a commutative operation. Are there other places in Python where this is the case? ~ George On Wed, Feb 27, 2019 at 10:06 AM Guido van Rossum wrote:

On Wed, Feb 27, 2019 at 8:50 AM Rhodri James wrote:

...

On 27/02/2019 16:25, João Matos wrote:

...

I would like to propose that instead of using this (applies to Py3.5 and upwards) dict_a = {**dict_a, **dict_b}

we could use dict_a = dict_a + dict_b

or even better dict_a += dict_b

While I don't object to the idea of concatenating dictionaries, I feel obliged to point out that this last is currently spelled dict_a.update(dict_b)

This is likely to be controversial. But I like the idea. After all, we have `list.extend(x)` ~~ `list += x`. The key conundrum that needs to be solved is what to do for `d1 + d2` when there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 2019-02-27 17:37, Guido van Rossum wrote:

On Wed, Feb 27, 2019 at 9:34 AM George Castillo mailto:gmcastil@gmail.com> wrote:

The key conundrum that needs to be solved is what to do for `d1 + d2` when there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

This would mean that addition, at least in this particular instance, is not a commutative operation.  Are there other places in Python where this is the case?

Yes there are. 'a' + 'b' is not the same as 'b' + 'a'.

For non-numbers we only require + to be associative, i.e. a + b + c == (a + b) + c == a + (b + c).

That is satisfied for this proposal. Are there any advantages of using '+' over '|'?

On 2019-03-02 22:02, francismb wrote:

On 2/27/19 7:14 PM, MRAB wrote:

...

Are there any advantages of using '+' over '|'? or for e.g. '<=' (d1 <= d2) over '+' (d1 + d2)

'<=' is for comparison, less-than-or-equal (in the case of sets, subset, which is sort of the same kind of thing). Using it for anything else in Python would be too confusing.

Are there any advantages of using '+' over '|'? or '<-' (d1 <- d2) meaning merge priority (overriding policy for equal keys) on the right dict, and may be '->' (d1 -> d2) merge priority on

On 2/27/19 7:14 PM, MRAB wrote: the left dict over '+' (d1 + d2) ? E.g.:

...

...

d1 = {'a':1, 'b':1 } d2 = {'a':2 } d3 = d1 -> d2 d3 {'a':1, 'b':1 }

...

...

d1 = {'a':1, 'b':1 } d2 = {'a':2 } d3 = d1 <- d2 d3 {'a':2, 'b':1 }

Regards, --francis

"foo" + "bar" != "bar" + "foo" On Wed, Feb 27, 2019, 12:35 PM George Castillo wrote:

The key conundrum that needs to be solved is what to do for `d1 + d2` when

...

there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

This would mean that addition, at least in this particular instance, is not a commutative operation. Are there other places in Python where this is the case?

~ George

On Wed, Feb 27, 2019 at 10:06 AM Guido van Rossum wrote:

...

On Wed, Feb 27, 2019 at 8:50 AM Rhodri James wrote:

...

On 27/02/2019 16:25, João Matos wrote:

...

I would like to propose that instead of using this (applies to Py3.5 and upwards) dict_a = {**dict_a, **dict_b}

we could use dict_a = dict_a + dict_b

or even better dict_a += dict_b

While I don't object to the idea of concatenating dictionaries, I feel obliged to point out that this last is currently spelled dict_a.update(dict_b)

This is likely to be controversial. But I like the idea. After all, we have `list.extend(x)` ~~ `list += x`. The key conundrum that needs to be solved is what to do for `d1 + d2` when there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

OK, you're it. Please write a PEP for this. On Wed, Feb 27, 2019 at 3:53 PM Steven D'Aprano wrote:

On Wed, Feb 27, 2019 at 10:34:43AM -0700, George Castillo wrote:

...

...

The key conundrum that needs to be solved is what to do for `d1 + d2`

when

...

there are overlapping keys. I propose to make d2 win in this case, which is what happens in `d1.update(d2)` anyways. If you want it the other way, simply write `d2 + d1`.

This would mean that addition, at least in this particular instance, is not a commutative operation. Are there other places in Python where this is the case?

Strings, bytes, lists, tuples.

In this case, I wouldn't call it dict addition, I would call it a union operator. That suggests that maybe we match sets and use | for union.

That also suggests d1 & d2 for the intersection between two dicts, but which value should win?

More useful than intersection is, I think, dict subtraction: d1 - d2 being a new dict with the keys/values from d1 which aren't in d2.

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

On Wed, Feb 27, 2019 at 10:42 AM Michael Selik wrote:

On Wed, Feb 27, 2019 at 10:22 AM Anders Hovmöller wrote:

...

I dislike the asymmetry with sets:

...

{1} | {2} {1, 2}

To me it makes sense that if + works for dict then it should for set too.

/ Anders

...

On 27 Feb 2019, at 17:25, João Matos wrote:

Hello,

I would like to propose that instead of using this (applies to Py3.5 and upwards) dict_a = {**dict_a, **dict_b}

we could use dict_a = dict_a + dict_b

The dict subclass collections.Counter overrides the update method for adding values instead of overwriting values.

https://docs.python.org/3/library/collections.html#collections.Counter.updat...

Counter also uses +/__add__ for a similar behavior.

>>> c = Counter(a=3, b=1) >>> d = Counter(a=1, b=2) >>> c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3})

At first I worried that changing base dict would cause confusion for the subclass, but Counter seems to share the idea that update and + are synonyms.

Great, this sounds like a good argument for + over |. The other argument is that | for sets *is* symmetrical, while + is used for other collections where it's not symmetrical. So it sounds like + is a winner here. -- --Guido van Rossum (python.org/~guido)

On Wed, Feb 27, 2019 at 11:06 AM João Matos wrote:

Great. Because I don't program in any other language except Python, I can't make the PR (with the C code). Maybe someone who program in C can help?

First we need a PEP, and for a PEP you need a core dev interested in sponsoring the PEP. And I'm not it. Is there a core dev who is interested in sponsoring or co-authoring this PEP? -- --Guido van Rossum (python.org/~guido)

I’d like to try my hand at implementing this, if nobody else is interested. I should be able to have something up today. Brandt

On Feb 27, 2019, at 11:05, João Matos wrote:

Hello,

Great. Because I don't program in any other language except Python, I can't make the PR (with the C code). Maybe someone who program in C can help?

Best regards,

João Matos

...

On 27-02-2019 18:48, Guido van Rossum wrote:

...

On Wed, Feb 27, 2019 at 10:42 AM Michael Selik wrote:

...

On Wed, Feb 27, 2019 at 10:22 AM Anders Hovmöller wrote: I dislike the asymmetry with sets:

...

{1} | {2} {1, 2}

To me it makes sense that if + works for dict then it should for set too.

/ Anders

...

On 27 Feb 2019, at 17:25, João Matos wrote:

Hello,

I would like to propose that instead of using this (applies to Py3.5 and upwards) dict_a = {**dict_a, **dict_b}

we could use dict_a = dict_a + dict_b

The dict subclass collections.Counter overrides the update method for adding values instead of overwriting values.

https://docs.python.org/3/library/collections.html#collections.Counter.updat...

Counter also uses +/__add__ for a similar behavior.

>>> c = Counter(a=3, b=1) >>> d = Counter(a=1, b=2) >>> c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3})

At first I worried that changing base dict would cause confusion for the subclass, but Counter seems to share the idea that update and + are synonyms.

Great, this sounds like a good argument for + over |. The other argument is that | for sets *is* symmetrical, while + is used for other collections where it's not symmetrical. So it sounds like + is a winner here.

-- --Guido van Rossum (python.org/~guido)

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

27.02.19 20:48, Guido van Rossum пише:

On Wed, Feb 27, 2019 at 10:42 AM Michael Selik mailto:mike@selik.org> wrote > The dict subclass collections.Counter overrides the update method for adding values instead of overwriting values.

https://docs.python.org/3/library/collections.html#collections.Counter.updat...

Counter also uses +/__add__ for a similar behavior.

    >>> c = Counter(a=3, b=1)     >>> d = Counter(a=1, b=2)     >>> c + d # add two counters together:  c[x] + d[x]     Counter({'a': 4, 'b': 3})

At first I worried that changing base dict would cause confusion for the subclass, but Counter seems to share the idea that update and + are synonyms.

Great, this sounds like a good argument for + over |. The other argument is that | for sets *is* symmetrical, while + is used for other collections where it's not symmetrical. So it sounds like + is a winner here.

Counter uses + for a *different* behavior!

...

...

Counter(a=2) + Counter(a=3) Counter({'a': 5})

I do not understand why we discuss a new syntax for dict merging if we already have a syntax for dict merging: {**d1, **d2} (which works with *all* mappings). Is not this contradicts the Zen?

On Thu, Feb 28, 2019 at 07:40:25AM -0500, James Lu wrote:

I agree with Storchaka here. The advantage of existing dict merge syntax is that it will cause an error if the object is not a dict or dict-like object, thus preventing people from doing bad things.

What sort of "bad things" are you afraid of? -- Steven

28.02.19 23:19, Greg Ewing пише:

Serhiy Storchaka wrote:

...

I do not understand why we discuss a new syntax for dict merging if we already have a syntax for dict merging: {**d1, **d2} (which works with *all* mappings).

But that always returns a dict. A '+' operator could be implemented by other mapping types to return a mapping of the same type.

And this opens a non-easy problem: how to create a mapping of the same type? Not all mappings, and even not all dict subclasses have a copying constructor.

On Mar 1, 2019, at 11:31 AM, Guido van Rossum wrote:

There's a compromise solution for this possible. We already do this for Sequence and MutableSequence: Sequence does *not* define __add__, but MutableSequence *does* define __iadd__, and the default implementation just calls self.update(other). I propose the same for Mapping (do nothing) and MutableMapping: make the default __iadd__ implementation call self.update(other).

Usually, it's easy to add methods to classes without creating disruption, but ABCs are more problematic. If MutableMapping grows an __iadd__() method, what would that mean for existing classes that register as MutableMapping but don't already implement __iadd__? When "isinstance(m, MutableMapping)" returns True, is it a promise that the API is fully implemented? Is this something that mypy could would or should complain about?

Anyways, the main reason to prefer d1+d2 over {**d1, **d2} is that the latter is highly non-obvious except if you've already encountered that pattern before

I concur. The latter is also an eyesore and almost certain to be a stumbling block when reading code. That said, I'm not sure we actually need a short-cut for "d=e.copy(); d.update(f)". Code like this comes-up for me perhaps once a year. Having a plus operator on dicts would likely save me five seconds per year. If the existing code were in the form of "d=e.copy(); d.update(f); d.update(g); d.update(h)", converting it to "d = e + f + g + h" would be a tempting but algorithmically poor thing to do (because the behavior is quadratic). Most likely, the right thing to do would be "d = ChainMap(e, f, g, h)" for a zero-copy solution or "d = dict(ChainMap(e, f, g, h))" to flatten the result without incurring quadratic costs. Both of those are short and clear. Lastly, I'm still bugged by use of the + operator for replace-logic instead of additive-logic. With numbers and lists and Counters, the plus operator creates a new object where all the contents of each operand contribute to the result. With dicts, some of the contents for the left operand get thrown-away. This doesn't seem like addition to me (IIRC that is also why sets have "|" instead of "+"). Raymond

On Sat, 2 Mar 2019 at 19:15, Raymond Hettinger wrote:

...

On Mar 1, 2019, at 11:31 AM, Guido van Rossum wrote:

There's a compromise solution for this possible. We already do this for Sequence and MutableSequence: Sequence does *not* define __add__, but MutableSequence *does* define __iadd__, and the default implementation just calls self.update(other). I propose the same for Mapping (do nothing) and MutableMapping: make the default __iadd__ implementation call self.update(other).

Usually, it's easy to add methods to classes without creating disruption, but ABCs are more problematic. If MutableMapping grows an __iadd__() method, what would that mean for existing classes that register as MutableMapping but don't already implement __iadd__? When "isinstance(m, MutableMapping)" returns True, is it a promise that the API is fully implemented? Is this something that mypy could would or should complain about?

Just to clarify the situation, currently Mapping and MutableMapping are not protocols from both runtime and mypy points of view. I.e. they don't have the structural __subclasshook__() (as e.g. Iterable), and are not declared as Protocol in typeshed. So to implement these (and be considered a subtype by mypy) one needs to explicitly subclass them (register() isn't supported by mypy). This means that adding a new method will not cause any problems here, since the new method will be non-abstract with a default implementation that calls update() (the same way as for MutableSequence). The only potential for confusion I see is if there is a class that de-facto implements current MutableMapping API and made a subclass (at runtime) of MutableMapping using register(). Then after we add __iadd__, users of that class might expect that __iadd__ is implemented, while it might be not. This is however OK I think, since register() is already non type safe. Also there is a simple way to find if there are any subclassses of MutableMapping in typeshed that don't have __iadd__: one can *try* declaring MutableMapping.__iadd__ as abstract, and mypy will error on all such subclasses. -- Ivan

01.03.19 21:31, Guido van Rossum пише:

On Thu, Feb 28, 2019 at 10:30 PM Serhiy Storchaka mailto:storchaka@gmail.com> wrote: And this opens a non-easy problem: how to create a mapping of the same type? Not all mappings, and even not all dict subclasses have a copying constructor.

There's a compromise solution for this possible. We already do this for Sequence and MutableSequence: Sequence does *not* define __add__, but MutableSequence *does* define __iadd__, and the default implementation just calls self.update(other). I propose the same for Mapping (do nothing) and MutableMapping: make the default __iadd__ implementation call self.update(other).

This LGTM for mappings. But the problem with dict subclasses still exists. If use the copy() method for creating a copy, d1 + d2 will always return a dict (unless the plus operator or copy() are redefined in a subclass). If use the constructor of the left argument type, there will be problems with subclasses with non-compatible constructors (e.g. defaultdict).

Anyways, the main reason to prefer d1+d2 over {**d1, **d2} is that the latter is highly non-obvious except if you've already encountered that pattern before, while d1+d2 is what anybody familiar with other Python collection types would guess or propose. And the default semantics for subclasses of dict that don't override these are settled with the "d = d1.copy(); d.update(d2)" equivalence.

Dicts are not like lists or deques, or even sets. Iterating dicts produces keys, but not values. The "in" operator tests a key, but not a value. It is not that I like to add an operator for dict merging, but dicts are more like sets than sequences: they can not contain duplicated keys and the size of the result of merging two dicts can be less than the sum of their sizes. Using "|" looks more natural to me than using "+". We should look at discussions for using the "|" operator for sets, if the alternative of using "+" was considered, I think the same arguments for preferring "|" for sets are applicable now for dicts. But is merging two dicts a common enough problem that needs introducing an operator to solve it? I need to merge dicts maybe not more than one or two times by year, and I am fine with using the update() method. Perhaps {**d1, **d2} can be more appropriate in some cases, but I did not encounter such cases yet.

On Mon, Mar 4, 2019, 8:30 AM Serhiy Storchaka wrote:

But is merging two dicts a common enough problem that needs introducing an operator to solve it? I need to merge dicts maybe not more than one or two times by year, and I am fine with using the update() method. Perhaps {**d1, **d2} can be more appropriate in some cases, but I did not encounter such cases yet.

Like other folks in the thread, I also want to merge dicts three times per year. And every one of those times, itertools.ChainMap is the right way to do that non-destructively, and without copying.

On 3/4/19 10:44 AM, Steven D'Aprano wrote:

If you know ahead of time which order you want, you can simply reverse it:

# prefs = site_defaults + user_defaults + document_prefs prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))

but that seems a little awkward to me, and reads backwards. I'm used to thinking reading left-to-right, not right-to-left.

I read that as use document preferences first, then user defaults, then site defautls, exactly as I'd explain the functionality to someone else. So maybe we're agreeing: if you think in terms of updating a dictionary of preferences, then maybe it reads backwards, but if you think of implementing features, then adding dictionaries of preferences reads backwards.

On 3/4/19 5:11 PM, Steven D'Aprano wrote:

On Mon, Mar 04, 2019 at 11:56:54AM -0600, Dan Sommers wrote:

...

On 3/4/19 10:44 AM, Steven D'Aprano wrote:

...

If you know ahead of time which order you want, you can simply reverse it:

# prefs = site_defaults + user_defaults + document_prefs prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))

but that seems a little awkward to me, and reads backwards. I'm used to thinking reading left-to-right, not right-to-left.

I read that as use document preferences first, then user defaults, then site defautls, exactly as I'd explain the functionality to someone else.

If you explained it to me like that, with the term "use", I'd think that the same feature would be done three times: once with document prefs, then with user defaults, then site defaults.

Clearly that's not what you mean, so I'd then have to guess what you meant by "use", since you don't actually mean use. That would leave me trying to guess whether you meant that *site defaults* overrode document prefs or the other way.

I don't like guessing, so I'd probably explicitly ask: "Wait, I'm confused, which wins? It sounds like site defaults wins, surely that's not what you meant."

You're right: "use" is the wrong word. Perhaps "prefer" is more appropriate. To answer the question of which wins: the first one in the list [document, user, site] that contains a given preference in question. Users don't see dictionary updates; they see collections of preferences in order of priority. Documentation is hard. :-) Sorry.

...

So maybe we're agreeing: if you think in terms of updating a dictionary of preferences, then maybe it reads backwards, but if you think of implementing features, then adding dictionaries of preferences reads backwards.

Do you think "last seen wins" is backwards for dict.update() or for command line options?

As a user, "last seen wins" is clearly superior for command line options. As a programmer, because object methods operate on their underlying object, it's pretty obvious that d1.update(d2) starts with d1 and applies the changes expressed in d2, which is effectively "last seen wins." If I resist the temptation to guess in the face of ambiguity, though, I don't think that d1 + d2 is any less ambiguous than a hypothetical dict_update(d1, d2) function. When I see a + operator, I certainly don't think of one operand or the other winning.

* Dicts are not like sets because the ordering operators (<, <=, >, >=) are not defined on dicts, but they implement subset comparisons for sets. I think this is another argument pleading against | as the operator to combine two dicts. * Regarding how to construct the new set in __add__, I now think this should be done like this: class dict: <other methods> def __add__(self, other): new = self.copy() # A subclass may or may not choose to override new.update(other) return new AFAICT this will give the expected result for defaultdict -- it keeps the default factory from the left operand (i.e., self). * Regarding how often this is needed, we know that this is proposed and discussed at length every few years, so I think this will fill a real need. * Regarding possible anti-patterns that this might encourage, I'm not aware of problems around list + list, so this seems an unwarranted worry to me. -- --Guido van Rossum (python.org/~guido)

On Mon, Mar 4, 2019 at 12:12 PM Neil Girdhar wrote:

On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum wrote:

...

* Dicts are not like sets because the ordering operators (<, <=, >, >=)

are not defined on dicts, but they implement subset comparisons for sets. I think this is another argument pleading against | as the operator to combine two dicts.

...

I feel like dict should be treated like sets with the |, &, and - operators since in mathematics a mapping is sometimes represented as a set of pairs with unique first elements. Therefore, I think the set metaphor is stronger.

That ship has long sailed.

...

* Regarding how to construct the new set in __add__, I now think this should be done like this:

class dict: <other methods> def __add__(self, other): new = self.copy() # A subclass may or may not choose to override new.update(other) return new

I like that, but it would be inefficient to do that for __sub__ since it would create elements that it might later delete.

def __sub__(self, other): new = self.copy() for k in other: del new[k] return new

is less efficient than

def __sub__(self, other): return type(self)({k: v for k, v in self.items() if k not in other})

when copying v is expensive. Also, users would probably not expect values that don't end up being returned to be copied.

No, the values won't be copied -- it is a shallow copy that only increfs the keys and values. -- --Guido van Rossum (python.org/~guido)

Honestly I would rather withdraw the subtraction operators than reopen the discussion about making dict more like set. On Mon, Mar 4, 2019 at 12:33 PM Neil Girdhar wrote:

On Mon, Mar 4, 2019 at 3:22 PM Guido van Rossum wrote:

...

On Mon, Mar 4, 2019 at 12:12 PM Neil Girdhar

wrote:

...

...

On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum

wrote:

...

...

* Dicts are not like sets because the ordering operators (<, <=, >,

=) are not defined on dicts, but they implement subset comparisons for sets. I think this is another argument pleading against | as the operator to combine two dicts.

...

I feel like dict should be treated like sets with the |, &, and - operators since in mathematics a mapping is sometimes represented as a set of pairs with unique first elements. Therefore, I think the set metaphor is stronger.

That ship has long sailed.

Maybe, but reading through the various replies, it seems that if you are adding "-" to be analogous to set difference, then the combination operator should be analogous to set union "|". And it also opens an opportunity to add set intersection "&". After all, how do you filter a dictionary to a set of keys?

...

...

d = {'some': 5, 'extra': 10, 'things': 55} d &= {'some', 'allowed', 'options'} d {'some': 5}

...

...

...

* Regarding how to construct the new set in __add__, I now think this

should be done like this:

...

...

class dict: <other methods> def __add__(self, other): new = self.copy() # A subclass may or may not choose to

override

...

new.update(other) return new

I like that, but it would be inefficient to do that for __sub__ since it would create elements that it might later delete.

def __sub__(self, other): new = self.copy() for k in other: del new[k] return new

is less efficient than

def __sub__(self, other): return type(self)({k: v for k, v in self.items() if k not in other})

when copying v is expensive. Also, users would probably not expect values that don't end up being returned to be copied.

No, the values won't be copied -- it is a shallow copy that only increfs the keys and values.

Oh right, good point. Then your way is better since it would preserve any other data stored by the dict subclass.

...

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

On Mon, Mar 4, 2019 at 3:58 PM Christopher Barker wrote:

On Mon, Mar 4, 2019 at 12:41 PM Guido van Rossum wrote:

...

Honestly I would rather withdraw the subtraction operators than reopen the discussion about making dict more like set.

I think that's unfortunate.

+1

I think the "dicts are like more-featured" sets is a math-geek perspective, and unlikely to make things more clear for the bulk of users. And may make it less clear.

I'd say reddit has some pretty "common users", and they're having a discussion of this right now (https://www.reddit.com/r/Python/comments/ax4zzb/pep_584_add_and_operators_to...). The most popular comment is how it should be |. Anyway, I think that following the mathematical metaphors tends to make things more intuitive in the long run. Python is an adventure. You learn it for years and then it all makes sense. If dict uses +, yes, new users might find that sooner than |. However, when they learn set union, I think they will wonder why it's not consistent with dict union. The PEP's main justification for + is that it matches Counter, but counter is adding the values whereas | doesn't touch the values. I think it would be good to at least make a list of pros and cons of each proposed syntax.

We need to be careful -- there are a lot more math geeks on this list than in the general Python coding population.

Simply adding "+" is a non-critical nice to have, but it seems unlikely to really confuse anyone.

-CHB

-- Christopher Barker, PhD

Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Mon, 4 Mar 2019 at 20:42, Guido van Rossum wrote:

Honestly I would rather withdraw the subtraction operators than reopen the discussion about making dict more like set.

I'm neutral on dict addition, but dict subtraction seemed an odd extension to the proposal. Using b in a - b solely for its keys, and ignoring its values, seems weird to me. Even if dict1 - dict2 were added to the language, I think I'd steer clear of it as being too obscure. I'm not going to get sucked into this debate, but I'd be happy to see the subtraction operator part of the proposal withdrawn. Paul

Indeed the "obscure" argument should be thrown away. The `|` operator in sets seems to be evident for every one on this list but I would be curious to know how many people first got a TypeError doing set1 + set2 and then found set1 | set2 in the doc. Except for math geek the `|` is always something obscure.

...

Even if dict1 - dict2 were added to the language, I think I'd steer clear of it as being too obscure. Everything is obscure until people learn it and get used to it.

On 05/03/2019 09:42, Jimmy Girardet wrote:

Indeed the "obscure" argument should be thrown away.

The `|` operator in sets seems to be evident for every one on this list but I would be curious to know how many people first got a TypeError doing set1 + set2 and then found set1 | set2 in the doc.

Every. Single. Time. I don't use sets a lot (purely by happenstance rather than choice), and every time I do I have to go and look in the documentation because I expect the union operator to be '+'.

Except for math geek the `|` is always something obscure.

Two thirds of my degree is in maths, and '|' is still something I don't associate with sets. It would be unreasonable to expect '∩' and '∪' as the operators, but reasoning from '-' for set difference I always expect '+' and '*' as the union and intersection operators. Alas my hopes are always cruelly crushed :-) -- Rhodri James *-* Kynesim Ltd

On Mar 5, 2019, at 2:13 PM, Greg Ewing wrote:

Rhodri James wrote:

...

I have to go and look in the documentation because I expect the union operator to be '+'.

Anyone raised on Pascal is likely to find + and * more natural. Pascal doesn't have bitwise operators, so it re-uses + and * for set operations. I like the economy of this arrangement -- it's not as if there's any other obvious meaning that + and * could have for sets.

The language SETL (the language of sets) also uses + and * for set operations.¹ For us though, the decision to use | and & are set in stone. The time for debating the decision was 19 years ago.² Raymond ¹ https://www.linuxjournal.com/article/6805 ² https://www.python.org/dev/peps/pep-0218/

On Mon, Mar 04, 2019 at 03:33:36PM -0500, Neil Girdhar wrote:

Maybe, but reading through the various replies, it seems that if you are adding "-" to be analogous to set difference, then the combination operator should be analogous to set union "|".

That's the purpose of this discussion, to decide whether dict merging is more like addition/concatenation or union :-)

And it also opens an opportunity to add set intersection "&".

What should intersection do in the case of matching keys? I see the merge + operator as a kind of update, whether it makes a copy or does it in place, so to me it is obvious that "last seen wins" should apply just as it does for the update method. But dict *intersection* is a more abstract operation than merge/update. And that leads to the problem, what do you do with the values? {key: "spam"} & {key: "eggs"} # could result in any of: {key: "spam"} {key: "eggs"} {key: ("spam", "eggs")} {key: "spameggs"} an exception something else? Unlike "update", I don't have any good use-cases to prefer any one of those over the others.

After all, how do you filter a dictionary to a set of keys?

...

...

d = {'some': 5, 'extra': 10, 'things': 55} d &= {'some', 'allowed', 'options'} d {'some': 5}

new = d - (d - allowed) {k:v for (k,v) in d if k in allowed}

...

...

...

* Regarding how to construct the new set in __add__, I now think this should be done like this:

class dict: <other methods> def __add__(self, other): new = self.copy() # A subclass may or may not choose to override new.update(other) return new

I like that, but it would be inefficient to do that for __sub__ since it would create elements that it might later delete.

def __sub__(self, other): new = self.copy() for k in other: del new[k] return new

is less efficient than

def __sub__(self, other): return type(self)({k: v for k, v in self.items() if k not in other})

I don't think you should be claiming what is more or less efficient unless you've actually profiled them for speed and memory use. Often, but not always, the two are in opposition: we make things faster by using more memory, and save memory at the cost of speed. Your version of __sub__ creates a temporary dict, which then has to be copied in order to preserve the type. Its not obvious to me that that's faster or more memory efficient than building a dict then deleting keys. (Remember that dicts aren't lists, and deleting keys is an O(1) operation.) -- Steven

Adding the + operator for dictionaries feels like it would be a mistake in that it offers at most sugar-y benefits, but introduces the significant drawback of making it easier to introduced unintended errors. This would be the first instance of "addition" where the result can potentially lose/overwrite data (lists and strings both preserve the full extent of each operand; Counters include the full value from each operand, etc). Combining dictionaries is fundamentally an operation that requires more than one piece of information, because there's no single well-defined way to combine a pair of them. Off the top of my head, I can think of at least 2 different common options (replacement aka .update(), combination of values a la Counter). Neither of these is really a more valid "addition" of dictionaries. For specific dict-like subclasses, addition may make sense - Counter is a great example of this, because the additional context adds definition to the most logical method via which two instances would be combined. If anything, this seems like an argument to avoid implementing __ladd__ on dict itself, to leave the possibility space open for interpretation by more specific classes. *From: *Raymond Hettinger *Date: *Mon, Mar 4, 2019 at 9:53 PM *To: *Guido van Rossum *Cc: *python-ideas

...

On Mar 4, 2019, at 11:24 AM, Guido van Rossum wrote:

* Regarding how often this is needed, we know that this is proposed and discussed at length every few years, so I think this will fill a real need.

I'm not sure that conclusion follows from the premise :-) Some ideas get proposed routinely because they are obvious things to propose, not because people actually need them. One hint is that the proposals always have generic variable names, "d = d1 + d2", and another is that they are almost never accompanied by actual use cases or real code that would be made better. I haven't seen anyone in this thread say they would use this more than once a year or that their existing code was unclear or inefficient in any way. The lack of dict addition support in other languages (like Java example) is another indicator that there isn't a real need -- afaict there is nothing about Python that would cause us to have a unique requirement that other languages don't have.

FWIW, there are some downsides to the proposal -- it diminishes some of the unifying ideas about Python that I typically present on the first day of class:

* One notion is that the APIs nudge users toward good code. The "copy.copy()" function has to be imported -- that minor nuisance is a subtle hint that copying isn't good for you. Likewise for dicts, writing "e=d.copy(); e.update(f)" is a minor nuisance that either serves to dissuade people from unnecessary copying or at least will make very clear what is happening. The original motivating use case for ChainMap() was to make a copy free replacement for excessively slow dict additions in ConfigParser. Giving a plus-operator to mappings is an invitation to writing code that doesn't scale well.

* Another unifying notion is that the star-operator represents repeat addition across multiple data types. It is a nice demo to show that "a * 5 == a + a + a + a + a" where "a" is an int, float, complex, str, bytes, tuple, or list. Giving __add__() to dicts breaks this pattern.

* When teaching dunder methods, the usual advice regarding operators is to use them only when their meaning is unequivocal; otherwise, have a preference for named methods where the method name clarifies what is being done -- don't use train+car to mean train.shunt_to_middle(car). For dicts that would mean not having the plus-operator implement something that isn't inherently additive (it applies replace/overwrite logic instead), that isn't commutative, and that isn't linear when applied in succession (d1+d2+d3).

* In the advanced class where C extensions are covered, the organization of the slots is shown as a guide to which methods make sense together: tp_as_number, tp_as_sequence, and tp_as_mapping. For dicts to gain the requisite methods, they will have to become numbers (in the sense of filling out the tp_as_number slots). That will slow down the abstract methods that search the slot groups, skipping over groups marked as NULL. It also exposes method groups that don't typically appear together, blurring their distinction.

* Lastly, there is a vague piece of zen-style advice, "if many things in the language have to change to implement idea X, it stops being worth it". In this case, it means that every dict-like API and the related abstract methods and typing equivalents would need to grow support for addition in mappings (would it even make sense to add to shelve objects or os.environ objects together?)

That's my two cents worth. I'm ducking out now (nothing more to offer on the subject). Guido's participation in the thread has given it an air of inevitability so this post will likely not make a difference.

Raymond

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Mar 04, 2019 at 10:18:13PM -0800, Amber Yust wrote:

Adding the + operator for dictionaries feels like it would be a mistake in that it offers at most sugar-y benefits, but introduces the significant drawback of making it easier to introduced unintended errors.

What sort of errors? I know that some (mis-)features are "bug magnets" that encourage people to write buggy code, but I don't see how this proposal is worse than dict.update(). In one way it is better, since D + E returns a new dict, instead of over-writing the data in D. Ask any functional programmer, and they'll tell you that we should avoid side-effects.

This would be the first instance of "addition" where the result can potentially lose/overwrite data (lists and strings both preserve the full extent of each operand; Counters include the full value from each operand, etc).

I don't see why this is relevant to addition. It doesn't even apply to numeric addition! If I give you the result of an addition: 101 say, you can't tell what the operands were. And that's not even getting into the intricicies of floating point addition, which can violate associativity ``(a + b) + c`` is not necessarily equal to ``a + (b + c)`` and distributivity: ``x*(a + b)`` is not necessarily equal to ``x*a + x*b`` even for well-behaved, numeric floats (not NANs or INFs).

Combining dictionaries is fundamentally an operation that requires more than one piece of information, because there's no single well-defined way to combine a pair of them.

Indeed, But some ways are more useful than others.

Off the top of my head, I can think of at least 2 different common options (replacement aka .update(), combination of values a la Counter). Neither of these is really a more valid "addition" of dictionaries.

That's why we have subclasses and operator overloading :-) By far the most commonly requested behaviour for this is copy-and- update (or merge, if you prefer). But subclasses are free to define it as they will, including: - add values, as Counter already does; - raise an exception if there is a duplicate key; - "first seen wins" or anything else. -- Steven

04.03.19 15:29, Serhiy Storchaka пише:

Using "|" looks more natural to me than using "+". We should look at discussions for using the "|" operator for sets, if the alternative of using "+" was considered, I think the same arguments for preferring "|" for sets are applicable now for dicts.

See the Python-Dev thread with the subject "Re: Re: PEP 218 (sets); moving set.py to Lib" starting from https://mail.python.org/pipermail/python-dev/2002-August/028104.html

Serhiy Storchaka wrote:

And this opens a non-easy problem: how to create a mapping of the same type?

That's the responsibility of the class implementing the + operator. There doesn't have to be any guarantee that a subclass of it will automatically return an instance of the subclass (many existing types provide no such guarantee, e.g. + on strings), so whatever strategy it uses doesn't have to be part of its public API. -- Greg

The PEP should probably also propose d1-d2.

What would be the output of this? Does this return a new dictionary where keys in d2 are removed in d1 like sets?

...

...

d = dict((i, i) for i in range(5)) e = dict((i, i) for i in range(4, 10)) d {0: 0, 1: 1, 2: 2, 3: 3, 4: 4} e {4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9} d.items() - e.items() {(0, 0), (1, 1), (3, 3), (2, 2)} dict(d.items() - e.items()) {0: 0, 1: 1, 3: 3, 2: 2}

-- Regards, Karthikeyan S

On Thu, Feb 28, 2019 at 08:59:30PM -0800, Hasan Diwan wrote:

Do we really need a "+" and a "-" operation on dictionaries? [dictinstance.update({k:v}) for k,v in dictinstance.items()] does handle merges already.

I don;t think that does what you intended. That merges dictinstance with itself (a no-op!), but one item at a time, so in the slowest, most inefficient way possible. Writing a comprehension for its side-effects is an anti-pattern that should be avoided. You are creating a (potentially large) list of Nones which has to be created, then garbage collected.

And I'm assuming that "-" should return the difference -- set(d1.keys()) - set(d2.keys()), right?

No. That throws away the values associated with the keys. P.S. As per Guido's ~~command~~ request *wink* I'm writing a PEP for this. I should have a draft ready later this evening. -- Steven

On Fri, Mar 01, 2019 at 08:47:36AM +0200, Serhiy Storchaka wrote:

Currently Counter += dict works and Counter + dict is an error. With this change Counter + dict will return a value, but it will be different from the result of the += operator.

That's how list.__iadd__ works too: ListSubclass + list will return a value, but it might not be the same as += since that operates in place and uses a different dunder method. Why is it a problem for dicts but not a problem for lists?

Also, if the custom dict subclass implemented the plus operator with different semantic which supports the addition with a dict, this change will break it, because dict + CustomDict will call dict.__add__ instead of CustomDict.__radd__.

That's not how operators work in Python or at least that's not how they worked the last time I looked: if the behaviour has changed without discussion, that's a breaking change that should be reverted. Obviously I can't show this with dicts, but here it is with lists: py> class MyList(list): ... def __radd__(self, other): ... print("called subclass first") ... return "Something" ... py> [1, 2, 3] + MyList() called subclass first 'Something' This is normal, standard behaviour for Python operators: if the right operand is a subclass of the left operand, the reflected method __r*__ is called first.

Adding support of new operators to builting types is dangerous.

Explain what makes new operators more dangerous than old operators please.

...

I do not understand why we discuss a new syntax for dict merging if we already have a syntax for dict merging: {**d1, **d2} (which works with *all* mappings). Is not this contradicts the Zen?

But (as someone else pointed out) {**d1, **d2} always returns a dict, not the type of d1 and d2.

And this saves us from the hard problem of creating a mapping of the same type.

What's wrong with doing this? new = type(self)() Or the equivalent from C code. If that doesn't work, surely that's the fault of the subclass, the subclass is broken, and it will raise an exception. I don't think it is our responsibility to do anything more than call the subclass constructor. If that's broken, then so be it. Possibly relevant: I've always been frustrated and annoyed at classes that hardcode their own type into methods. E.g. something like: class X: def spam(self, arg): return X(eggs) # Wrong! Bad! Please use type(self) instead. That means that each subclass has to override every method: class MySubclass(X): def spam(self, arg): # Do nothing except change the type returned. return type(self)( super().spam(arg) ) This gets really annoying really quickly. Try subclassing int, for example, where you have to override something like 30+ methods and do nothing but wrap calls to super. -- Steven

01.03.19 12:44, Steven D'Aprano пише:

On Fri, Mar 01, 2019 at 08:47:36AM +0200, Serhiy Storchaka wrote:

...

Currently Counter += dict works and Counter + dict is an error. With this change Counter + dict will return a value, but it will be different from the result of the += operator.

That's how list.__iadd__ works too: ListSubclass + list will return a value, but it might not be the same as += since that operates in place and uses a different dunder method.

Why is it a problem for dicts but not a problem for lists?

Because the plus operator for lists predated any list subclasses.

...

Also, if the custom dict subclass implemented the plus operator with different semantic which supports the addition with a dict, this change will break it, because dict + CustomDict will call dict.__add__ instead of CustomDict.__radd__.

That's not how operators work in Python or at least that's not how they worked the last time I looked: if the behaviour has changed without discussion, that's a breaking change that should be reverted.

You are right.

What's wrong with doing this?

new = type(self)()

Or the equivalent from C code. If that doesn't work, surely that's the fault of the subclass, the subclass is broken, and it will raise an exception.

Try to do this with defaultdict. Note that none of builtin sequences or sets do this. For good reasons they always return an instance of the base type.

04.03.19 15:43, Serhiy Storchaka пише:

01.03.19 12:44, Steven D'Aprano пише:

...

On Fri, Mar 01, 2019 at 08:47:36AM +0200, Serhiy Storchaka wrote:

...

Also, if the custom dict subclass implemented the plus operator with different semantic which supports the addition with a dict, this change will break it, because dict + CustomDict will call dict.__add__ instead of CustomDict.__radd__.

That's not how operators work in Python or at least that's not how they worked the last time I looked: if the behaviour has changed without discussion, that's a breaking change that should be reverted.

You are right.

Actually there is still a problem if the first argument is an instance of dict subclass that does not implement __add__.

Counter also uses +/__add__ for a similar behavior.

>>> c = Counter(a=3, b=1) >>> d = Counter(a=1, b=2) >>> c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3})

At first I worried that changing base dict would cause confusion for the subclass, but Counter seems to share the idea that update and + are synonyms.

Counter is a moot analogy. Counter's + and - operators follow the rules of numbers addition and subtraction:

...

...

c = Counter({"a": 1}) c + Counter({"a": 5}) Counter({'a': 6}) c + Counter({"a": 5}) - Counter({"a": 4}) Counter({'a': 2})

Which also means that in most cases (c1 + c2) - c2 == c1 which is not something you would expect with the suggested "dictionary addition" operation. As a side note, this is not true in general for Counters because of how subtraction handles 0. E.g.

...

...

c0 = Counter({"a": 0}) c1 = Counter({"a": 1}) (c0 + c1) - c1 Counter() (c0 + c1) - c1 == c0 False

--- The current intuition of how + and - work don't apply literally to this suggestion: 1) numeric types are their own story 2) most built-in sequences imply concatenation for + and have no subtraction 3) numpy-like arrays behave closer to numbers 4) Counters mimic numbers in some ways and while addition reminds of concatenation (but order is not relevant) they also have subtraction 5) sets have difference which is probably the closest you expect from dict subtraction, but no + operator --- I understand the arguments against a | operator for dicts but I don't entirely agree with them. dict is obviously a different type of object than all the others I've mentioned, even mathematically, and there is no clear precedent. If sets happened to maintain insertion order, like dicts after 3.6/3.7, I would expect the union operator to also preserve the order. Before 3.6 we probably would have seen dicts as closer to sets from that point of view, and this suggested addition as closer to set union. The question of symmetry ({"a": 1} + {"a": 2}) is an important one and I would consider not enforcing one resolution in PEP 584, and instead leave this undefined (i.e. in the resulting dict, the value could be either 1 or 2, or just completely undefined to also be compatible with Counter-like semantics in the same PEP). This is something to consider carefully if the plan is to make the new operators part of Mapping. It's not obvious that all mappings should implement this the same way, and a survey of what is being done by other implementation of Mappings would be useful. On the other hand leaving it undefined might make it harder to standardize it later, once other implementations have defined their own behavior. This question is probably on its own a valid argument against the proposal. When it comes to dicts (and not Mappings in general) {**d1, **d2} or d.update() already have clearly-defined semantics. The new proposal for a merge() operation might be more useful. The added value would be the ability to add two mappings regardless of concrete type. But it's with Mappings in general that this proposal is the most problematic. On the other hand the subtraction operator is probably less controversial and immediately useful (the idiom to remove keys from a dictionary is not obvious).

On Fri, Mar 1, 2019 at 11:00 PM INADA Naoki wrote:

I dislike adding more operator overload to builtin types.

str is not commutative, but it satisfies a in (a+b), and b in (a+b). There are no loss.

In case of dict + dict, it not only sum. There may be loss value.

{"a":1} + {"a":2} = ?

In case of a.update(b), it's clear that b wins. In case of a + b, "which wins" or "exception raised on duplicated key?" is unclear to me.

Picking semantics can be done as part of the PEP discussion, and needn't be a reason for rejecting the proposal before it's even made. We have at least one other precedent to consider:

...

...

{1} | {1.0} {1} {1.0} | {1} {1.0}

I have absolutely no doubt that these kinds of questions will be thoroughly hashed out (multiple times, even) before the PEP gets to pronouncement. ChrisA

On Fri, Mar 01, 2019 at 09:58:08PM +0900, INADA Naoki wrote:

...

...

...

{1} <= ({1} | {1.0}) True {1.0} <= ({1} | {1.0}) True

So dict + dict is totally different than set | set. dict + dict has los at equality level.

Is that an invariant you expect to apply to other uses of the + operator? py> x = -1 py> x <= (x + x) False py> [999] <= ([1, 2, 3] + [999]) False -- Steven

On Fri, 1 Mar 2019 at 13:48, INADA Naoki wrote:

...

Is that an invariant you expect to apply to other uses of the + operator?

py> x = -1 py> x <= (x + x) False

py> [999] <= ([1, 2, 3] + [999]) False

Please calm down. I meant each type implements "sum" in semantics of the type, in lossless way. What "lossless" means is changed by the semantics of the type.

-1 + -1 = -2 is sum in numerical semantics. There are no loss.

TBH I don't understand what is lossless about numeric addition. What is the definition of lossless? Clearly some information is lost, since you can't uniquely restore two numbers you add from the result. Unless you define what lossless means, there will be just more misunderstandings. -- Ivan

On 3/1/2019 9:38 AM, INADA Naoki wrote:

Sorry, I'm not good at English enough to explain my mental model.

I meant no skip, no ignorance, no throw away.

In case of 1+2=3, both of 1 and 2 are not skipped, ignored or thrown away.

On the other hand, in case of {a:1, b:2}+{a:2}={a:2, b:2}, I feel {a:1} is skipped, ignored, or thrown away.  I used "lost" to explain it.

And I used "lossless" for "there is no lost".  Not for reversible.

If it isn't understandable to you, please ignore me.

I think Rémi’s comment is very similar to my thought.  Merging mapping is more complex than concatenate sequence and it seems hard to call it "sum".

I understand Inada to be saying that each value on the LHS (as shown above) affects the result on the RHS. That's the case with addition of ints and other types, but not so with the proposed dict addition. As he says, the {a:1} doesn't affect the result. The result would be the same if this key wasn't present in the first dict, or if the key had a different value. This doesn't bother me, personally. I'm just trying to clarify. Eric

Regards,

2019年3月1日(金) 23:19 Ivan Levkivskyi mailto:levkivskyi@gmail.com>:

On Fri, 1 Mar 2019 at 13:48, INADA Naoki mailto:songofacandy@gmail.com> wrote:

> > > Is that an invariant you expect to apply to other uses of the + > operator? > > py> x = -1 > py> x <= (x + x) > False > > py> [999] <= ([1, 2, 3] + [999]) > False >

Please calm down.  I meant each type implements "sum" in semantics of the type, in lossless way. What "lossless" means is changed by the semantics of the type.

-1 + -1 = -2 is sum in numerical semantics.  There are no loss.

TBH I don't understand what is lossless about numeric addition. What is the definition of lossless? Clearly some information is lost, since you can't uniquely restore two numbers you add from the result.

Unless you define what lossless means, there will be just more misunderstandings.

-- Ivan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

OK, thanks for explaining! So more formally speaking, you want to say that for other examples of '+' in Python x1 + y == x2 + y if and only if x1 == x2, while for the proposed '+' for dicts there may be many different x_i such that x_i + y gives the same result.

It's bit different thank my mind. I'm OK to violate " x1 + y == x2 + y if and only if x1 == x2", if it's not important for semantics of type of x1, x2, and y. Mapping is defined by key: value pairs. It's core part. I don't want to call operator losts key: value pair as "sum". That's why I thought this proposal is more serious abuse of + operator. By the way, in case of sequence, `len(a) + len(b) == len(a + b)`. In case of set, `len(a) + len(b) >= len(a | b)`. Proposed operation looks similar to `set | set` than `seq + seq` in this point of view. I don't propose | than +. I just mean difference between dict.update() and seq+seq is not smaller than difference between dict.update() and set|set. If | seems not fit to this operation, + seems not fit to this operation too. -- INADA Naoki

On 3/1/19 8:19 AM, Ivan Levkivskyi wrote:

On Fri, 1 Mar 2019 at 13:48, INADA Naoki wrote:

...

...

Is that an invariant you expect to apply to other uses of the + operator?

py> x = -1 py> x <= (x + x) False

py> [999] <= ([1, 2, 3] + [999]) False

Please calm down. I meant each type implements "sum" in semantics of the type, in lossless way. What "lossless" means is changed by the semantics of the type.

-1 + -1 = -2 is sum in numerical semantics. There are no loss.

TBH I don't understand what is lossless about numeric addition. What is the definition of lossless? Clearly some information is lost, since you can't uniquely restore two numbers you add from the result.

Unless you define what lossless means, there will be just more misunderstandings.

I don't mean to put words into anyone's mouth, but I think I see what IDANA Naoki means: in other cases of summation, the result somehow includes or contains both operands. In the case of summing dicts, though, some of the operands are "lost" in the process. I'm sure that I'm nowhere near as prolific as many of the members of this list, but I don't remember ever merging dicts (and a quick grep of my Python source tree confirms same), so I won't comment further on the actual issue at hand.

On Fri, Mar 1, 2019 at 10:10 PM Steven D'Aprano wrote:

On Fri, Mar 01, 2019 at 08:59:45PM +0900, INADA Naoki wrote:

...

I dislike adding more operator overload to builtin types.

str is not commutative, but it satisfies a in (a+b), and b in (a+b). There are no loss.

Is this an invariant you expect to apply for other classes that support the addition operator?

5 in (5 + 6)

I meant more high level semantics: "no loss". Not only "in". So my example about set used "<=" operator. 5 + 6 is sum of 5 and 6.

[1, 2, 3] in ([1, 2, 3] + [4, 5, 6])

Both of [1,2,3] and [4,5,6] are not lost in result.

Since it doesn't apply for int, float, complex, list or tuple, why do you think it must apply to dicts?

You misunderstood my "no loss" expectation.

...

In case of dict + dict, it not only sum. There may be loss value.

Yes? Why is that a problem?

It's enough reason to I dislike.

...

{"a":1} + {"a":2} = ?

Would you like to argue that Counter.__add__ is a mistake for the same reason?

In Counter's case, it's clear. In case of dict, it's unclear.

Counter(('a', 1)) + Counter(('a', 2)) = ?

For the record, what I expected the above to do turned out to be *completely wrong* when I tried it. I expected Counter({'a': 3}) but the actual results are Counter({'a': 2, 1: 1, 2: 1}).

It just because you misunderstood Counter's initializer argument. It's not relating to how overload + or | operator.

Every operation is going to be mysterious if you have never learned what it does:

from array import array a = array('i', [1, 2, 3]) b = array('i', [10, 20, 30]) a + b = ?

Without trying it or reading the docs, should that be an error, or concatenation, or element-wise addition?

I never say every operator must be expected by everyone. Don't straw man. -- INADA Naoki