[CentralOH] Generator and DRY

[Steven Huwig]

I would solve the chunking a different way, but maybe it's too clever
by half for your code. It also has a little repetition. :-)

import itertools
def chunker(size, stream):
   chunk = list(itertools.islice(stream, size))
   while chunk:
       yield ''.join(chunk)
       chunk = list(itertools.islice(stream, size))

>>> stream = (c for c in "abcdefghijklmnopqrstuvwxyz")
>>> size = 5
>>> x = chunker(size, stream)
>>> x
<generator object at 0xb9b20>
>>> list(x)
['abcde', 'fghij', 'klmno', 'pqrst', 'uvwxy', 'z']
>>>

-- Steve

On Tue, Jul 22, 2008 at 5:37 PM, Mark Erbaugh <mark at microenh.com> wrote:
> Here are part of method I wrote that provides a generator:
>
>    def read(self, id, size=None):
>        i = gen_data.next()
>        try:
>            while True:
>                while len(buffer) >= read_size:
>                    yield(''.join(buffer[:read_size]))
>                    del buffer[:read_size]
>                i = gen_data.next()
>        except StopIteration:
>            pass
>        while len(buffer) >= read_size:
>            yield(''.join(buffer[:read_size]))
>            del buffer[:read_size]
>        if len(buffer):
>            yield(''.join(buffer))
>
>
> The purpose of this code is to take data from another generator
> (gen_data) and deliver it in chunks of the specified size (except the
> last).  Code not shown handles possible decompression (zlib) of the
> data.
>
> The interesting thing is that there is some duplicate code:
>
>                while len(buffer) >= read_size:
>                    yield(''.join(buffer[:read_size]))
>                    del buffer[:read_size]
>
> I couldn't come up with any way to rewrite the loop to eliminate the
> duplicate sections. Actually, I could by buffering the data, but I don't
> want to do that. The chunks could be quite large.
>
> If this were "normal" code, I could write an embedded function and just
> call function that in both places.
>
> However, since the embedded function contains a yield() call, it now
> becomes a generator itself and the re-factored code doesn't do work
> properly.
>
> Is there a way to create an embedded function where the yield() call is
> still tied to the parent function/method?
>
> Thanks,
> Mark
>
>
> _______________________________________________
> CentralOH mailing list
> CentralOH at python.org
> http://mail.python.org/mailman/listinfo/centraloh
>