Generators.

[Taylor]

On Dec 7, 1:29 pm, Jorge Cardona <jorgeecard... at gmail.com> wrote:
> 2009/12/7 Lie Ryan <lie.1... at gmail.com>:
>
>
>
> > On 12/7/2009 7:22 AM, Jorge Cardona wrote:
>
> >> Hi,
>
> >> I was trying to create a function that receive a generator and return
> >> a list but that each elements were computed in a diferent core of my
> >> machine. I start using islice function in order to split the job in a
> >> way that if there is "n" cores each "i" core will compute the elements
> >> i,i+n,i+2n,..., but islice has a weird (to me) behavior, look:
>
> > it's nothing weird, python just do what you're telling it to:
>
> > transform all x in X with f(x)
>
> >> g = (f(x) for x in X)
>
> > then slice the result of that
>
> >> print(list(x for x in islice(g,0,None,2)))
>
> When i wrote that first line of code i thought that i was creating a
> generator that will later compute  the elements of X with function f,
> if i will like to transform them immediately i would use [] instead
> (), so, the result is not where i want to slice, even so, is exactly
> the same to slice, after or before, the only difference is that after
> the compute i'm losing 5 in unnecessary execution of the function f.
>
> > what you want to do is to slice before you transform:
> >>>> g = (x for x in islice(X, 0, None, 2))
> >>>> print(list(f(x) for x in g))
> > eval: 0
> > eval: 2
> > eval: 4
> > eval: 6
> > eval: 8
> > [0, 2, 4, 6, 8]
>
> What i want to do is a function that receive any kind of generator and
> execute it in several cores (after a fork) and return the data, so, i
> can't slice the set X before create the generator.
>
>
>
> >> islice execute the function at the generator and drop the elements
> >> that aren't in the slice. I found that pretty weird, the way that i
> >> see generators is like an association between and indexing set (an
> >> iterator or another generator) and a computation that is made indexed
> >> by the indexing set, and islice is like a "transformation" on the
> >> indexing set,it doesn't matter the result of the function, the slice
> >> should act only on the indexing set, some other "transformation" like
> >> takewhile act on the result so, the execution it has to be made, but
> >> in the islice, or other "transformation" that act only in the indexing
> >> set, the function shouldn't be executed at each element, but only on
> >> that new set that result of the application of the "transformation" on
> >> the original set.
>
> > that seems like an extremely lazy evaluation, I don't know if even a true
> > lazy language do that. Python is a strict language, with a few laziness
> > provided by generators, in the end it's still a strict language.
>
> Yes, it looks like lazy evaluation, but, i don't see why there is not
> a better control over the iterable associated to a generator, even
> with Python that is a strict language, it will increase the
> functionality of it, and the performance too, imagine that you pass a
> function that takes 1 sec in run, and for several reason you can't
> slice before (as the smp function that i want to create), the final
> performance with the actual islice it gets really reduced
> Just creating the separation between those transformation that act on
> the index(islice, or tee) on those that act on the result(dropwhile,
> takewhile, etc.) the control could be fine enough to increase
> usability (that's the way i think right now), and you will be able to
> combine generator without lose performance.
>
> >> Well, it works for what i need, but is not very neat, and i think that
> >> there it should be a formal way to act on the base indexing iterator,
> >> such way exists? Is there a better approach to get what i need?
>
> > Reverse your operation.
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> --
> Jorge Eduardo Cardona
> jorgeecard... at gmail.com
> jorgeecardona.blogspot.com
> ------------------------------------------------
> Linux registered user  #391186
> Registered machine    #291871
> ------------------------------------------------

What would you have your islice do for the following generator?

def fib(n):
    a,b=0,1
    for x in range(n):
        a,b=b,a+b
        yield a

In order for some value it yields to be correct, it needs execute all
the previous times. If it happens that some of the results aren't
used, they are thrown out. As is, using your islice (on, say, fib(10))
gives a KeyError for the key '.0'.
Point is, generators don't work that way. You aren't guaranteed to be
able to jump around, only to find the next value. If you need to split
a particular generator up into parts that can be computed separately,
your best bet is probably to rewrite that generator.