Easy questions from a python beginner

[wheres pythonmonks]

Funny... just spent some time with timeit:

I wonder why I am passing in strings if the callback overhead is so light...

More funny:  it looks like inline (not passed in) lambdas can cause
python to be more efficient!
>>> import random
>>> d = [ (['A','B'][random.randint(0,1)],x,random.gauss(0,1)) for x in xrange(0,1000000) ]
>>> def A1(): j = [ lambda t: (t[2]*t[1],t[2]**2+5) for t in d ]

>>> def A2(): j = [ (t[2]*t[1],t[2]**2+5) for t in d ]

>>> def A3(l): j = [ l(t) for t in d]

>>> import timeit
>>> timeit.timeit('A1()','from __main__ import A1,d',number=10);
2.2185971572472454
>>> timeit.timeit('A2()','from __main__ import A2,d',number=10);
7.2615454749912942
>>> timeit.timeit('A3(lambda t: (t[2]*t[1],t[2]**2+5))','from __main__ import A3,d',number=10);
9.4334241349350947

So: in-line lambda possible speed improvement.  in-line tuple is slow,
passed-in callback, slowest yet?

Is this possibly right?

Hopefully someone can spot the bug?

W





On Fri, Jul 23, 2010 at 4:10 AM, Steven D'Aprano
<steve at remove-this-cybersource.com.au> wrote:
> On Thu, 22 Jul 2010 22:47:11 -0400, wheres pythonmonks wrote:
>
>> Thanks for pointing out that swap (and my swap2) don't work everywhere
>> -- is there a way to get it to work inside functions?
>
> Not in CPython. In IronPython or Jython, maybe, I don't know enough about
> them. But even if you got it to work, it would be an implementation-
> dependent trick.
>
> [...]
>> I always think that it is the language's job to express
>> my thoughts...
>
> Ha, no, it's the language's job to execute algorithms. If it did so in a
> way that is similar to the way people think, that would be scary. Have
> you *seen* the way most people think???
>
> *wink*
>
>
>> I don't like to think that my thoughts are somehow
>> constrained by the language.
>
>
> Whether you "like" to think that way, or not, thoughts are influenced and
> constrained by language. While I don't accept the strong form of the
> Sapir-Whorf hypothesis (that some thoughts are *impossible* due to lack
> of language to express them, a position which has been discredited), a
> weaker form is almost certainly correct. Language influences thought.
>
> Turing Award winner and APL creator Kenneth E. Iverson gave a lecture
> about this theme, "Notation as a tool of thought", and argued that more
> powerful notations aided thinking about computer algorithms.
>
> Paul Graham also discusses similar ideas, such as the "blub paradox".
> Graham argues that the typical programmer is "satisfied with whatever
> language they happen to use, because it dictates the way they think about
> programs". We see this all the time, with people trying to write Java in
> Python, Perl in Python, and Ruby in Python.
>
> And Yukihiro Matsumoto has said that one of his inspirations for creating
> Ruby was the science fiction novel Babel-17, which in turn is based on
> the Sapir-Whorf Hypothesis.
>
>
>
>> The truth is that I don't intend to use these approaches in anything
>> serious.  However, I've been known to do some metaprogramming from time
>> to time.
>>
>> In a recent application, I pass in a list of callables (lambdas) to be
>> evaluated repeatedly.
>
> Are you aware that lambdas are just functions? The only differences
> between a "lambda" and a function created with def is that lambda is
> syntactically limited to a single expression, and that functions created
> with lambda are anonymous (they don't have a name, or at least, not a
> meaningful name).
>
>
>> Clearly, a superior solution is to pass a single lambda that returns a
>> list.
>
> I don't see why you say this is a superior solution, mostly because you
> haven't explained what the problem is.
>
>
>> [Less function call dispatches]
>
> How? You have to generate the list at some point. Whether you do it like
> this:
>
> functions = (sin, cos, tan)
> data = (2.3, 4.5, 1.2)
> result = [f(x) for f, x in zip(functions, data)]
>
> or like this:
>
> result = (lambda x, y, z: return (sin(x), cos(y), tan(z))
>    )(2.3, 4.5, 1.2)
>
> you still end up with the same number of function calls (four). Any
> execution time will almost certainly be dominated by the work done inside
> the lambda (sin, cos and tan) rather than the infrastructure. And unless
> you have profiled your code, you would be surprised as to where the
> bottlenecks are. Your intuitions from Perl will not guide you well in
> Python -- it's a different language, and the bottlenecks are different.
>
>
>> However, it might be more
>> efficient to avoid the function call overhead completely and pass-in a
>> string which is substituted into a string code block, compiled, and
>> executed.
>
> See, that's *exactly* what I mean about intuitions. No no no no!!! Using
> exec or eval in Python code is almost certainly a *pessimation*, not an
> optimization! I expect this will be an order of magnitude slower to
> parse, compile and execute a string than it is to execute a function.
> Using exec or friends to avoid the overhead of function calls is like
> pushing your car to work to avoid the overhead of having to get in and
> out of the car.
>
> But of course, don't take my word for it. Write your code and profile it,
> see where the bottlenecks are. I might be wrong.
>
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>