[Python-ideas] Adding "+" and "+=" operators to dict

[Chris Barker]

On Thu, Feb 12, 2015 at 10:56 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> And the difference between:
>
>     default_settings + global_settings + user_settings
>
> versus:
>
>     dict(default_settings, global_settings, user_settings)
>
> is only four characters. If you're using a, b, c as variable names in
> production, you probably have bigger problems than an extra four
> characters :-)


and the difference between:

  a + b

and

  a.sum(b)

is only six characters.

I _think_ your point is that the math symbols are for math, so not
suggesting that we'd all be better off without any infix operators at all.
But that ship has sailed -- Python overloads infix operators. And it does
it for common usage for built-ins.

So adding + and += for dicts fits right in with all this.


Some people here find + to be a plausible operator. I don't. I react to
> using + for things which aren't addition in much the same way that I
> expect you would react if I suggested we used ** as the operator. "Why
> on earth would you want to use ** it's nothing like exponentiation!"
>

merging two dicts certainly is _something_ like addition. I'd agree about
re-using other arbitrary operators just because they exist.


Some languages may be able to optimize a + b + c + d to avoid making and
> throwing away the intermediate dicts, but in general Python cannot.



> So
> it is going to fall into the same trap as list addition does. While it
> is not common for this to lead to severe performance degradation, when
> it does happen the cost is *so severe* that we should think long and
> hard before adding any more O(N**2) booby traps into the language.


well, it seems there are various proposals in this thread to create a
shorthand for "merging two dicts into a new third dict" -- wouldn't those
create the same booby traps?

And teh booby traps aren't in code like:

a + b + c + d -- folks don't generally put 100s of items in a line like
that. The traps are when you do it in a loop:

al_dicts = {}
for d in a_bunch_of_dicts:
    start = start + d

( or sum() ) -- of course, in the above loop, if you had +=, you'd be fine.
So it may be that adding + for mutables is not the same trap, as long as
you add the +=, too. (can't put += in a comprehension, though)


But this:
"Some languages may be able to optimize a + b + c + d "

Got me thinking -- this is actually a really core performance problem for
numpy. In that case, the operators are really being used for math, so you
can't argue that they shouldn't be supported for that reason -- and we
really want readable code:

y = a*x**2 + b*x + c

really reads well, but it does create a lot of temporaries that kill
performance for large arrays. You can optimize that by hand by doing
somethign like:

y = x**2
y *= a
y += b*x
y += c

which really reads poorly!

So I've thought for years that we should have a "numpython" interpreter
that would parse out each expression, check its types, and if they were all
numpy arrays, generate an optimized version that avoided temporaries, maybe
even did nifty things like do the operations in cache-friendly blocks,
multi thread them, whatever. (there is a package called numexp that does
all these things already).


But maybe cPython itself could do an optimization step like that -- examine
an entire expression for types, and if they all support the right
operations, re-structure it in a more optimized way.

Granted, doing this is the general case would be pretty impossible but if
the hooks are there, then individual type-bases optimizations could be done
-- like the current interpreter has for adding strings.

OK -- gotten pretty OT here....

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150213/b6e45228/attachment-0001.html>