[Tutor] when is a generator "smart?"

[eryksun]

On Sat, Jun 1, 2013 at 11:58 PM, Jim Mooney <cybervigilante at gmail.com> wrote:
>
> def uneven_squares(x,y):
>     squarelist = (c**2 for c in range(x,y) if c%2 != 0)
>     return squarelist #returning a generator
>
> print(list(uneven_squares(10,10000))[2:10]) #slows as y gets bigger, then dies

I think you said you switched back to using Python 2.x. If that's
still the case, remember to use xrange() for a case like this.

2.x range() is a list factory function:

    >>> r = range(10, 10000)
    >>> itr = iter(r)
    >>> type(r), type(itr)
    (<type 'list'>, <type 'listiterator'>)

xrange is a type:

    >>> xr = xrange(10, 10000)
    >>> itxr = iter(xr)
    >>> type(xr), type(itxr)
    (<type 'xrange'>, <type 'rangeiterator'>)


Instead of using syntactic sugar like a generator expression, use a
generator function so you can see what it's doing:

    def uneven_squares(x, y):
        for c in xrange(x, y):
            if c % 2 != 0:
                yield c ** 2

    >>> g = uneven_squares(10, 10000)
    >>> type(g)
    <type 'generator'>

The compiler flags the code as a generator. Evaluation of generator
code is special-cased. It does all of the normal call setup, creates a
frame to run the code, but then instead of evaluating the frame it
returns a new generator.

The next() method (3.x __next__) evaluates the frame up to a yield:

    >>> g.next()
    121
    >>> next(g)  # use built-in next()
    169

When the frame returns instead of yielding, the generator raises
StopIteration. For example:

    def gen():
        yield 1
        return
        yield 2 # never gets here

    >>> list(gen())
    [1]

Others have already mentioned using itertools.islice(). This takes an
iterable/iterator, plus arguments for (start, stop, step) like
built-in slice(). In case you're unfamiliar with the latter, it
creates a a slice object for use in subscription. For example:

    >>> r = range(10)
    >>> s = slice(5)  # [:5]
    >>> r[s]
    [0, 1, 2, 3, 4]
    >>> s = slice(5, None)  # [5:]
    >>> r[s]
    [5, 6, 7, 8, 9]
    >>> s = slice(None, 5, 2)  # [:5:2]
    >>> r[s]
    [0, 2, 4]
    >>> s = slice(None, None, 3)  # [::3]
    >>> r[s]
    [0, 3, 6, 9]

Since many iterators don't support subscription, let alone slicing,
itertools.islice is a convenient alternative:

    >>> g = uneven_squares(10, 10000)

    >>> tuple(islice(g, 5)) # start=0,stop=5,step=1
    (121, 169, 225, 289, 361)

    >>> set(islice(g, 3))
    set([441, 625, 529])

islice returns an iterator, so it can be used efficiently to build
other sequence types, such as tuple() and set().

The second islice starts at 21**2 because the previous islice stopped
at 19**2, i.e.t he "start" and "stop" arguments are relative to the
current position in the iteration sequence.