[Tutor] parsing a "chunked" text file

[Karim Liateni]

Hello,

Thanks both of you for these useful information.

Regards
Karim

Hugo Arts wrote:
> On Thu, Mar 18, 2010 at 12:54 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>   
>> Karim Liateni, 04.03.2010 01:23:
>>
>> Yes, a *big* difference in the true sense of the word. Your code (assuming
>> you meant to write "... for line in ..." ) evaluates the entire list
>> comprehension before returning from the call. Steven's code returns a
>> generator that only handles one line (or a couple of empty lines) at a time.
>> So, assuming that this runs against a large file, Steven's code uses only a
>> constant amount of memory, compared to the whole file in your case, and is
>> likely also a lot faster than your code as it involves less looping.
>>
>>     
>
> Though, if you changed the brackets into parentheses, you'd get a
> generator expression, which *is* equivalent to Steven's version,
> except that it calls strip() twice, which is a bit wasteful.
>
> If the unnecessary extra call bothers you, you could do one of two things:
> 1) Learn how the yield keyword works. You should do this. It's an
> awesome feature, and you'll come across it many more times.
> 2) go functional and import itertools. ifilter with a generator
> expression, like so (pure functional programmers can also use imap
> instead of the generator expr., which might be faster. profile to be
> sure)
>
> def skip_blanks(lines):
>     return ifilter(None, (l.strip() for l in lines))
>
> Very short, has all the memory and speed benefits of the generator.
> Personally I really like terse functional programming like this,
> though I believe the general consensus in the python community is that
> imperative alternatives are usually clearer to read.
>
> If you want to know more about the yield keyword:
> A terse description (assumes that you know how iterators work) is
> here: http://docs.python.org/tutorial/classes.html#generators
> A more detailed description of iterators and generators can be found
> here: http://www.ibm.com/developerworks/library/l-pycon.html
>
> Hugo
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20100319/8e898a01/attachment.html>