Geneator/Iterator Nesting Problem - Any Ideas? 2.4

Steven Bethard steven.bethard at gmail.com
Fri Mar 4 11:31:22 EST 2005


ChaosKCW wrote:
> For reasons of pure asthetics and my own learning I wanted it to look
> like this:
> 
> </B>
> def resultsetbatchgen(cursor, size=100):
>     for results in (recordbatch for recordbatch in
> cursor.fetchmany(size)):
>         for rec in results:
>             yield rec
> </B>

Note that this is equivalent to:

def resultsetbatchgen(cursor, size=100):
     for results in cursor.fetchmany(size):
         for rec in results:
             yield rec

That is, your generator expression isn't really doing anything.

>>>>cur.execute(<QUERY WITH MOER THAN 1000 records>)
>>>>recs = (recordbatch for recordbatch in cur.fetchmany(1000))
>>>>sum(map(lambda x: 1, (rec for rec in recs)))
> 1000

This should be equivalent to
...
 >>> recs = iter(cur.fetchmany(1000))
 >>> len(list(recs))

>>>>cur.execute(<QUERY WITH MOER THAN 1000 records>)
>>>>for results in (recordbatch for recordbatch in cur.fetchmany(1000)):
> ... 	print sum(map(lambda x: 1, (rec for rec in results)))

This is now equivalent to:
 >>> cur.execute(<QUERY WITH MOER THAN 1000 records>)
 >>> for rec in cur.fetchmany(1000):
... 	print len(list(results))

Note that you're counting the number of elements in record, which I'm 
guessing is 76?

I'm thinking you want to do something like:

def resultsetbatchgen(cursor, size=100):
     while True:
         results = cursor.fetchmany(size)
         if not results:
             break
         for rec in results:
             yield rec

I don't actually know how you tell when there isn't anything more to 
fetch, but if it returns an empty list, you could also write this like:

def resultsetbatchgen(cursor, size=100):
     for results in iter(lambda: cursor.fetchmany(size), [])
         for rec in results:
             yield rec


HTH,

STeVe



More information about the Python-list mailing list