map/filter/reduce/lambda opinions and background unscientific mini-survey

[Steven D'Aprano]

On Sun, 03 Jul 2005 19:31:02 +0000, Ron Adam wrote:

> First on removing reduce:
> 
> 1. There is no reason why reduce can't be put in a functional module 

Don't disagree with that.

> or 
> you can write the equivalent yourself.  It's not that hard to do, so it 
> isn't that big of a deal to not have it as a built in.

Same goes for sum. Same goes for product, which doesn't have that many
common usages apart from calculating the geometric mean, and let's face
it, most developers don't even know what the geometric mean _is_. 

If you look back at past discussions about sum, you will see that there is
plenty of disagreement about how it should work when given non-numeric
arguments, eg strings, lists, etc. So it isn't so clear what sum should do.

> 2. Reduce calls a function on every item in the list, so it's 
> performance isn't much better than the equivalent code using a for-loop.

That is an optimization issue. Especially when used with the operator
module, reduce and map can be significantly faster than for loops.

>   *** (note, that list.sort() has the same problem. I would support 
> replacing it with a sort that uses an optional 'order-list' as a sort 
> key.  I think it's performance could be increased a great deal by 
> removing the function call reference. ***
> 
> 
> Second, the addition of sum & product:
> 
> 1. Sum, and less so Product, are fairly common operations so they have 
> plenty of use case arguments for including them.

Disagree about product, although given that sum is in the language, it
doesn't hurt to put product as well for completion and those few usages.

> 2. They don't need to call a pre-defined function between every item, so 
> they can be completely handled internally by C code. They will be much 
> much faster than equivalent code using reduce or a for-loop.  This 
> represents a speed increase for every program that totals or subtotals a 
> list, or finds a product of a set.

I don't object to adding sum and product to the language. I don't object
to adding zip. I don't object to list comps. Functional, er, functions
are a good thing. We should have more of them, not less.

>> But removing reduce is just removing
>> functionality for no other reason, it seems, than spite.
> 
> No, not for spite. It's more a matter of increasing the over all
> performance and usefulness of Python without making it more complicated.
>     In order to add new stuff that is better thought out, some things
> will need to be removed or else the language will continue to grow and
> be another visual basic.

Another slippery slope argument.

> Having sum and product built in has a clear advantage in both
> performance and potential frequency of use, where as reduce doesn't have
> the same performance advantage and most poeple don't use it anyway, so
> why have it built in if sum and product are?  

Because it is already there. 

> Why not just code it as a
> function and put it in your own module?

Yes, let's all re-invent the wheel in every module! Why bother having a
print statement, when it is so easy to write your own:

def myprint(obj):
    sys.stdout.write(str(obj))

Best of all, you can customize print to do anything you like, _and_ it is
a function.

>      def reduce( f, seq):
>          x = 0
>          for y in seq:
>              x = f(x,y)
>          return x

Because that is far less readable, and you take a performance hit.

> But I suspect that most people would just do what I currently do and
> write the for-loop to do what they want directly instead of using lambda
> in reduce.

That's your choice. I'm not suggesting we remove for loops and force you
to use reduce. Or even list comps.


-- 
Steven.