[Python-ideas] while conditional in list comprehension ??

[Shane Green]

Although it's a bit of a cheat, if you create a wrapper of the thing you're iterating, or don't mind closing it (it's probably best to wrap it unless you know what it is), both generators and list comprehensions can be "while iterated" using this approach: 

[item for item in items if condition or items.close()]


When I tested it earlier with a 1000 entries 5 times and had forgotten the parens on close(), it made it really obvious there would be times when the wrapping overhead wasn't a problem: 

On Jan 30, 2013, at 9:02 AM, Shane Green <shane.green at me.com> wrote:

> Nice catch.  New times, 
> 
> >>> timeit.timeit(var1)
> 8.533167123794556
> >>> timeit.timeit(var2)
> 9.067211151123047
> >>> timeit.timeit(var3)
> 12.966150999069214
> >>> timeit.timeit(var4)
> 
> And I accidentally ran this (without parens), so it was a regular comprehension: 
> def var5(count=1000):
>     seq = (i for i in xrange(count))
>     return [i for i in seq if i < 50 or seq.close]
> 
> >>> timeit.timeit(var5)
> 212.26763486862183
> 
> Then fixed it: 
> >>> timeit.timeit(var5)
> 10.280441045761108
> >>> 
> 
> 
> 
> 
> 
> 
> Shane Green
> 805-452-9666 | shane.green at me.com
> 
> Begin forwarded message:
> 
>> From: Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de>
>> Subject: RE: [Python-ideas] while conditional in list comprehension ??
>> Date: January 30, 2013 8:40:51 AM PST
>> To: 'Shane Green' <shane.green at me.com>
>> 
>> Careful! You’re using range() in the slow ones, but xrange() in the fast ones.
>> With the input seq being much longer than the output, differences in the time it takes to produce the range object may be important.
>>  
>> From: Shane Green [mailto:shane.green at me.com] 
>> Sent: Wednesday, January 30, 2013 5:37 PM
>> To: Wolfgang Maier
>> Subject: Re: [Python-ideas] while conditional in list comprehension ??
>>  
>> >>> def var1(count=1000):
>> ...     def _gen():
>> ...          for i in range(count):
>> ...               if i > 50: break
>> ...               yield i
>> ...     return list(_gen())
>> ... 
>> >>> def var2(count=1000):
>> ...     def stop():
>> ...          raise StopIteration
>> ...     return list(i for i in range(count) if i <= 50 or stop())
>> ... 
>> >>> def var3(count=1000):
>> ...     return [i for i in itertools.takewhile(lambda n: n <= 50, range(count))]
>> ... 
>> >>> def var4(count=1000):
>> ...     return [i for i in itertools.takewhile(functools.partial(operator.lt, 50)
>> ... 
>> >>> def var5(count=1000):
>> ...     seq = (i for i in xrange(count))
>> ...     return [i for i in seq if i < 50 or seq.close()]
>>  
>> >>> timeit.timeit(var1)
>> 19.118155002593994
>> >>> timeit.timeit(var2)
>>  
>> 19.217869997024536
>>  
>> >>> timeit.timeit(var5)
>> 10.251838207244873
>>  
>> >>> 
>>  
>>  
>>  
>>  
>> Shane Green
>> 805-452-9666 | shane.green at me.com
>>  
>> On Jan 30, 2013, at 8:17 AM, Wolfgang Maier <wolfgang.maier at biologie.uni-freiburg.de> wrote:
>> 
>> 
>> list(i for i in a if i < 5000 or a.close())
>>  
> 



Shane Green 
www.umbrellacode.com
408-692-4666 | shane at umbrellacode.com

On Jan 30, 2013, at 10:05 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

> On 30 January 2013 17:56, Yuriy Taraday <yorik.sar at gmail.com> wrote:
>> 
>> On Wed, Jan 30, 2013 at 1:46 PM, Wolfgang Maier
>> <wolfgang.maier at biologie.uni-freiburg.de> wrote:
>>> 
>>> your condition is 'partial(lt,50)', but this is not met to begin with and
>>> results in an empty list at least for me. Have you two actually checked
>>> the
>>> output of the code or have you just timed it?
>> 
>> Yeah. Shame on me. You're right. My belief in partial and operator module
>> has been shaken.
>> 
> 
> This is why I prefer this stop() idea to any of the takewhile()
> versions: regardless of performance it leads to clearer code, that can
> be understood more easily.
> 
> 
> Oscar
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130130/4a38f800/attachment.html>