[Twisted-Python] iterators/generator
Hello. I'd like to write my 'user' level code with generators, and thus, was thinking that new code could perhaps be at least generator friendly... what do you think: 1. We add the following code fragment somewhere in a python.util or equivalent: # primative support iterators for 2.1 try: StopIteration = StopIteration iter = iter except: class StopIteration(Exception): pass class _ListIterator: def __init__(self,lst): self.lst = list(lst) def next() if self.lst: return self.lst.pop(0) else: raise StopIteration def iter(lst): if type(lst) == type([]) or type(lst) == type(tuple()): return _ListIterator(lst) else: return lst 2. Then, code within twisted which could possible accept an iterator or generator could be written like this. import python.twisted.util try: itr = util.iter(getIterator()) val = itr.next() while 1: doSomething(val) val = itr.next() except util.StopIteration: pass Just thoughts... using generators from 'user' land is really very useful, and it'd be nice if lower levels of Twisted allowed for this by a mechanism similar to the above. Clark
On Mon, 10 Mar 2003 06:34:29 +0000
"Clark C. Evans"
Hello. I'd like to write my 'user' level code with generators, and thus, was thinking that new code could perhaps be at least generator friendly... what do you think:
I'd rather you just use 2.1 style iterators (i.e. define a __getitem__) in code that goes into Twisted. class Iterator: index = 0 def __getitem__(self, index): if index != self.index: raise TypeError, "this is an iterator" self.index += 1 if self.hasMoreData: return self.getData() else: raise IndexError
On Mon, Mar 10, 2003 at 10:19:42AM -0500, Itamar Shtull-Trauring wrote: | I'd rather you just use 2.1 style iterators (i.e. define a __getitem__) | in code that goes into Twisted. Perfect. Thanks. Clark
On Mon, Mar 10, 2003 at 10:19:42AM -0500, Itamar Shtull-Trauring wrote: | I'd rather you just use 2.1 style iterators (i.e. define a __getitem__) | in code that goes into Twisted. | | class Iterator: | | index = 0 | | def __getitem__(self, index): | if index != self.index: raise TypeError, "this is an iterator" | self.index += 1 | if self.hasMoreData: | return self.getData() | else: | raise IndexError After some thinking, the above won't work for me: 1) The whole point of an iterator is to be able to ask it for the next() object; using the above idiom doesn't allow me to 'yield' easily between calls to next(); for more detail see the FlowIterator in the flow.py code 2) The goal is to move foward with 2.2 style iterators and generators, but to do so in a way that the code still works for 2.1 users. The code has grown up a bit... try: # if we are using 2.2, just make a local copy # of the StopIteration and iter() built-in StopIteration = StopIteration iter = iter except: # Simulate StopIteration and iter() for 2.1 # This can eventually be removed once we # stop supporting 2.1 class StopIteration(Exception): pass class _ListIterator: def __init__(self,lst): self.lst = list(lst) def next(): if self.lst: return self.lst.pop(0) else: raise StopIteration def iter(lst): from types import ListType, TupleType if type(lst) == type([]) or type(lst) == type(tuple()): return _ListIterator(lst) else: return lst.__iter__() In this way, code with the toolkit can be written like from <the-place-for-this> import iter, StopIteration traverse(someIterable): iterator = iter(someIterable) try: while 1: val = iterator.next() # process val, perhaps Yielding # or doing other things not possible # in a 'for val in someIterable: except StopIteration: pass The advantage, is that traverse is now 'generator' friendly but will work with lists and tuples for those in 2.1 land. Further, if someone implements __iter__ and next() in 2.1 it will still work. Its a win-win. And the best part, is that once we stop supporting 2.1, we can just kill the "from <the-place-for-this>" line and all is well. ;) Clark
On Mon, Mar 10, 2003 at 10:19:42AM -0500, Itamar Shtull-Trauring wrote: | > Hello. I'd like to write my 'user' level code with generators, | > and thus, was thinking that new code could perhaps be at least | > generator friendly... what do you think: | | def __getitem__(self, index): | if index != self.index: raise TypeError, "this is an iterator" | self.index += 1 | if self.hasMoreData: | return self.getData() | else: | raise IndexError I think that the code below is a fairly good 'first-cut' at supporting user-level 2.2 generators with code that works in Python 2.1 Anyone object committing the following to python.compat? Clark # # This compatibility hack allows for code to be written that # supports 2.2 iterator/generator semantics within Python 2.1 # This wraps 2.1 lists, mappings, and classes using the __getitem__ # style iterator to use iter/next # try: StopIteration = StopIteration iter = iter except: # Python 2.1 StopIteration = IndexError class _ListIterator: def __init__(self,lst): self.idx = 0 if getattr(lst,'keys',None): lst = lst.keys() self.lst = lst def next(self): idx = self.idx self.idx += 1 return self.lst[idx] def iter(lst): if hasattr(lst,'__iter__'): return lst.__iter__() else: return _ListIterator(lst) # if __name__ == '__main__: def dumpiter(itr): next = iter(itr).next try: while 1: print next() except StopIteration: pass dumpiter([1,2,3]) dumpiter({'one': 'value', 'two': 'twoval'})
participants (2)
-
Clark C. Evans
-
Itamar Shtull-Trauring