[Python-ideas] Integrate some itertools into the Python syntax

[Michel Desmoulin]

No you didn't store the lines, the whole points is that they are
returning generators.

In that context, f[begin, end] does itertools.takewhile and dropwhile
and [:10000] is doing islice(0, 10000). None of those functions store
anything.

Please read the begining of thread.

Le 22/03/2016 20:41, Ethan Furman a écrit :
> On 03/22/2016 12:39 PM, Ethan Furman wrote:
>> On 03/22/2016 12:26 PM, Michel Desmoulin wrote:
>>> Le 22/03/2016 20:23, Ethan Furman a écrit :
>>>> On 03/22/2016 12:10 PM, Koos Zevenhoven wrote:
>>>>> On Tue, Mar 22, 2016 at 8:41 PM, Ethan Furman wrote:
>>>>>> On 03/22/2016 10:51 AM, Michel Desmoulin wrote:
>>>>>>
>>>>>>> def foo(p):
>>>>>>>        with open(p) as f:
>>>>>>>            def begin:
>>>>>>>                return x == "BEGIN SECTION"
>>>>>>>             def end:
>>>>>>>                return x == "STOP"
>>>>>>>            return f[begin, end][:10000]
>>>>>>>
>>>>>>> It's very clean, very convenient, very natural, and memory
>>>>>>> efficient.
>>>>>>
>>>>>>
>>>>>> Except the 10,000 limit doesn't happen until /after/ the end block is
>>>>>> reached -- which could be a million lines later.
>>>>>
>>>>>
>>>>> if f[begin, end] is a generator, the 10000 limit may happen before the
>>>>> end block is reached, which I think was the point.
>>>>
>>>> That is wrong, which was my point: the `[:10000]` doesn't take effect
>>>> until after `f[begin:end]` (whatever it is) is evaluated.
>>  >
>>> [begin, end] and [:10000] are applied one next() at a time.
>>>
>>> begin, then end, then :10000 for the first next(),
>>>
>>> then again in that order for the following next() call, etc.
>>>
>>> That's the whole point.
>>
>> That may be the point, but that is not what the above code does.  Since
>> you don't believe me, let's break it down:
>>
>> f[begin:end] -> grabs a section of p.  This could be 5 lines or 50000000
>>
>> [:10000] -> take the first 10000 of the previous result
>>
>> return -> send those (up to) 10000 lines back
> 
> In case my point still isn't clear:  if you just stored 49,990,000 lines
> just to throw them away, you are not being memory efficient.
> 
> -- 
> ~Ethan~
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/