[Python-Dev] Fwd: summing a bunch of numbers (or "whatevers")

Jack Diederich jack@performancedrivers.com
Sat, 19 Apr 2003 19:57:20 -0400


On Sun, Apr 20, 2003 at 12:10:48AM +0100, Michael Hudson wrote:
> Jack Jansen <Jack.Jansen@oratrix.com> writes:
> 
> > Do you have any idea why your sum function is, uhm, three times
> > faster than the reduce(operator.add) version? Is the implementation
> > of reduce doing something silly, or are there shortcuts you can take
> > that reduce() can't?
> 
> I imagine it's the function calls; a trip through the call machinery,
> time packing and unpacking arguments, etc.  I haven't checked, though.

Browsing through bltinmodule.c (was 'builtinmodule.c' too long?) it is
mainly the overhead of calling PyEval_CallObject lots of times, which would
include parsing args, etc.  It tries to avoid creating the argument tuple
more than once by checking the refcount on every loop (I would think the
tuple would be generally be unpacked by the receiving function, but better
safe than sorry).

> > I'm asking because I think I would prefer reduce to give the speed
> > you want.  That way, we won't have people come asking for a prod()
> > function to match sum(), etc.
> 

I think reduce/filter/map could be improved by checking if their operative
function is in builtin or operator modules and calling the C directly.
operator.c is a just litany of macros.  This would add tiny one-time overhead 
for non builtins/operators.

Don't mistake my comments as volunteering ;)  I'm still plodding through _sre.c
(Did you know re's are used in setup.py?  break _sre.c and you can't compile
the source without using two trees, fun!)

-jackdied