On 30/01/13 20:46, Wolfgang Maier wrote:
b) I have to say I was very impressed by the speed gains you report through the use of 'partial', which I had not thought of at all, I have to admit. However, I tested your suggestions and I think they both suffer from the same mistake: your condition is 'partial(lt,50)', but this is not met to begin with and results in an empty list at least for me. Have you two actually checked the output of the code or have you just timed it? I found that in order to make it work the comparison has to be made via 'partial(gt,50)'.
Yes, you are absolutely correct. I screwed that up badly. I can only take comfort that apparently so did Yuriy.
I don't often paste code in public without testing it, but when I do, it invariably turns out to be wrong.
With this modification the resulting list in your example would be [0,..,49] as it should be.
And now the big surprise in terms of runtimes: partial(lt,50) variant: 1.17 (but incorrect results) partial(gt,50) variant: 13.95 if cond or stop() variant: 9.86
I do not get such large differences. I get these:
py> min(t1.repeat(number=100000, repeat=5)) # cond or stop() 1.2582030296325684 py> min(t2.repeat(number=100000, repeat=5)) # takewhile and lambda 1.9907748699188232 py> min(t3.repeat(number=100000, repeat=5)) # takewhile and partial 1.8741891384124756
with the timers t1, t2, t3 as per my previous email.
I guess python is just smart enough to recognize that it compares against a constant value all the time, and optimizes the code accordingly (after all the if clause is a pretty standard thing to use in a comprehension).
No, it is much simpler than that. partial(lt, 50) is equivalent to:
lambda x: lt(50, x)
which is equivalent to 50 < x, *not* x < 50 like I expected.
So the function tests 50 < 0 on the first iteration, which is False, and takewhile immediately returns, giving you an empty list.
I was surprised that partial was *so much faster* than a regular function. But it showed me what I expected/wanted to see, and so I didn't question it. A lesson for us all.
So the reason for your reported speed-gain is that you actually broke out of the comprehension at the very first element instead of going through the first 50!