[Python-ideas] Interrogate alternate namespace keyword and concept

[Steven D'Aprano]

On Sat, 15 Aug 2009 06:29:11 am ilya wrote:
> That's a useful statistics, but the bottleneck is **only** because of
> parsing 'value +=5'.
>
> Here's how I time it:
>
> # insert my old program here...
>
>
> from timeit import Timer
> from codeop import compile_command
>
> def timing(something):
>     setup = 'from __main__ import test, interrogate, command, inc5'
>     best = sorted(Timer(something, setup).repeat(3, 1000))[0]

min(alist) is a more-direct, simpler, faster, easier to read way of 
calculating sorted(alist)[0].


>     print('{0!r} -> {1:.3} ms'.format(something, best))
>     print('# test.value =', test.value)
>
> command = '''
> for i in range(10):
>     value += 1
> '''

You're muddying the water by including a for-loop and call to range() 
inside the code snippet being tested. We're trying to compare your 
function interrogate(test, 'value += 1') with the standard call to 
test.value += 1. Why include the time required to generate a range() 
object, and iterate over it ten times, as part of the code snippet? All 
that does is mix up the time required to execute common code and the 
time required to execute the code we care about.

If it's not obvious why your approach is flawed, consider this extreme 
example:

Timer("time.sleep(1000); interrogate(test, 'value += 1')", ...)
Timer("time.sleep(1000); test.value += 1", ...)

The differences in speed between the interrogate call (using exec) and 
the direct access to test.value will be swamped by the time used by the 
common code. 

A more accurate measurement is to remove the "for i in..." part from 
command, and increase the number=1000 argument to Timer.repeat() to 
10000.


> inc5 = compile_command(command)

This is an unfair test. We're comparing directly accessing test.value 
versus indirectly accessing test.value using exec. Regardless of 
whether the caller compiles the statement manually before passing it to 
exec, or just passes it to exec to compile it automatically, the cost 
of that compilation has to be payed. Pulling that outside of the timing 
code just hides the true cost.

Pre-compiling before passing to exec is a good optimization for the 
cases where you need to exec the same code snippet over and over again. 
If you want to execute "for i in range(1000): exec(s)" then it makes 
sense to pull out the compilation of s outside of the loop. But 
generally when timing code snippets, the only purpose of the loop is to 
minimize errors and give more accurate results, so pulling out the 
compilation just hides some of the real cost.


> timing("interrogate(test, command)")
> timing(command.replace('value', 'test.value'))
> timing("interrogate(test, inc5)")
>
> Result:
>
> 15

Where does the 15 come from?

> 'interrogate(test, command)' -> 0.0908 ms
> # test.value = 30015
> '\nfor i in range(10):\n    test.value += 1\n' -> 0.00408 ms
> # test.value = 60015
> 'interrogate(test, inc5)' -> 0.00469 ms
> # test.value = 90015
>
> so interrogate() with additional precompiling introduces very little
> overhead. 

Only because you're ignoring the overhead of pre-compiling. A more 
accurate test would be:

timing("inc5 = compile_command(command); interrogate(test, inc5)")


> Though I agree it's inconvenient to write functions as 
> strings; 

If you think that's inconvenient, just try writing functions as code 
objects without calling compile :)



-- 
Steven D'Aprano