[Python-Dev] Comparing closures and arguments (was Re: Scoping vs augmented assignment vs sets (Re: 'fast locals' in Python 2.5)

[Josiah Carlson]

"Phillip J. Eby" <pje at telecommunity.com> wrote:
> At 11:26 AM 6/14/2006 -0700, Josiah Carlson wrote:
> >Ok, so here's a bit of a benchmark for you.
> >
> >     def helper(x,y):
> >         return y
> >
> >     def fcn1(x):
> >         _helper = helper
> >         y = x+1
> >         for i in xrange(x):
> >             y = _helper(x,y)
> >
> >     def fcn2(x):
> >         y = x+1
> >         def _helper(x):
> >             return y
> >         for i in xrange(x):
> >             y = _helper(x)
> >
> >
> >Can you guess which one is faster?  I guessed, but I was wrong ;).
> >
> > >>> x = 1000000
> > >>> min([fcn1(x) for i in xrange(10)]), min([fcn2(x) for i in xrange(10)])
> >(0.53200006484985352, 0.59299993515014648)
> >
> >It turns out that passing two arguments to a helper function is actually
> >faster than passing one argument and pulling a second out of an
> >enclosing scope.
> 
> That claim isn't necessarily supported by your benchmark, which includes 
> the time to *define* the nested function 10 times, but calls it only 45 
> times!  Try comparing fcn1(1000) and fcn2(1000) - I suspect the results 
> will be somewhat closer, but probably still in favor of fcn1.

Please re-read the code and test as I have specified.  You seem to have
misunderstood something in there, as in the example I provide, _helper
is called 1,000,000 times (and is defined only once for each call of
fcn2) during each fcn1 or fcn2 call, and _helper is only defined once
inside of fcn2.

Further, reducing the passes to 1000 preserves the relative performance
measures as I had previously stated.

    >>> x = 1000
    >>> min([fcn1(x) for i in xrange(10)]), min([fcn2(x) for i in xrange(10)])
    (0.00051907656835226135, 0.00056413073832572991)
    >>> x = 10000
    >>> min([fcn1(x) for i in xrange(10)]), min([fcn2(x) for i in xrange(10)])
    (0.0037536511925964078, 0.0044071910377851964)
    >>> x = 100000
    >>> min([fcn1(x) for i in xrange(10)]), min([fcn2(x) for i in xrange(10)])
    (0.053770416317590275, 0.057610581942384442)
    >>> x = 1000000
    >>> min([fcn1(x) for i in xrange(10)]), min([fcn2(x) for i in xrange(10)])
    (0.54333500712479577, 0.58664054298870383)


> However, I suspect that the remaining difference in the results would be 
> due to the fact that the interpreter loop has a "fast path" function call 
> implementation that doesn't work with closures IIRC.  Perhaps someone who's 
> curious might try adjusting the fast path to support closures, and see if 
> it can be made to speed them up without slowing down other "fast path" calls.

That would be an interesting direction for improving speed.

 - Josiah