Generators.

[Lie Ryan]

First, I apologize for rearranging your message out of order.

On 12/8/2009 5:29 AM, Jorge Cardona wrote:
>>> islice execute the function at the generator and drop the elements
>>> that aren't in the slice. I found that pretty weird, the way that i
>>> see generators is like an association between and indexing set (an
>>> iterator or another generator) and a computation that is made indexed
>>> by the indexing set, and islice is like a "transformation" on the
>>> indexing set,it doesn't matter the result of the function, the slice
>>> should act only on the indexing set, some other "transformation" like
>>> takewhile act on the result so, the execution it has to be made, but
>>> in the islice, or other "transformation" that act only in the indexing
>>> set, the function shouldn't be executed at each element, but only on
>>> that new set that result of the application of the "transformation" on
>>> the original set.
>>
>> that seems like an extremely lazy evaluation, I don't know if even a true
>> lazy language do that. Python is a strict language, with a few laziness
>> provided by generators, in the end it's still a strict language.
>>
>
> Yes, it looks like lazy evaluation, but, i don't see why there is not
> a better control over the iterable associated to a generator, even
> with Python that is a strict language, it will increase the
> functionality of it, and the performance too, imagine that you pass a
> function that takes 1 sec in run, and for several reason you can't
> slice before (as the smp function that i want to create), the final
> performance with the actual islice it gets really reduced
> Just creating the separation between those transformation that act on
> the index(islice, or tee) on those that act on the result(dropwhile,
> takewhile, etc.) the control could be fine enough to increase
> usability (that's the way i think right now), and you will be able to
> combine generator without lose performance.

Theoretically yes, but the semantic of generators in python is they work 
on an Iterable (i.e. objects that have __iter__), instead of a Sequence 
(i.e. objects that have __getitem__). That means semantically, 
generators would call obj.__iter__() and call the iter.__next__() and do 
its operation for each items returned by the iterator's iterable's 
__next__().

The lazy semantic would be hard to fit the current generator model 
without changing the semantics of generator to require a Sequence that 
supports indexing.

> Yes, it looks like lazy evaluation, but, i don't see why there is not
> a better control over the iterable associated to a generator, even
> with Python that is a strict language

You can control the laziness by making it explicitly lazy:

from functools import partial
def f(x):
     print("eval: %d"%x)
     return x

X = range(10)
g = (partial(f, x) for x in X)

print(list(x() for x in islice(g,0,None,2)))
# # or without partial:
# g = ((lambda: f(x)) for x in X)
# print(list(f() for f in islice(g,0,None,2)))

In a default-strict language, you have to explicitly say if you want 
lazy execution.

> What i want to do is a function that receive any kind of generator and
> execute it in several cores (after a fork) and return the data, so, i
> can't slice the set X before create the generator.

beware that a generator's contract is to return a valid iterator *once* 
only. You can use itertools.tee() to create more generators, but tee 
built a list of the results internally.