Syncing up iterators with gaps
Chris Angelico
rosuav at gmail.com
Wed Sep 28 15:48:43 EDT 2016
On Thu, Sep 29, 2016 at 5:10 AM, Tim Chase
<python.list at tim.thechases.com> wrote:
> And I'd like to do something like
>
> for common_key, d1, d2, d3 in magic_happens_here(data1, data2, data3):
> for row in d1:
> process_a(common_key, row)
> for thing in d2:
> process_b(common_key, row)
> for thing in d3:
> process_c(common_key, row)
Assuming that the keys are totally ordered and the data sets are
sorted, this should work:
def magic_happens_here(*iters):
iters = [iter(it) for it in iters]
nexts = [next(it, (None,)) for it in iters]
while "moar stuff":
try: common_key = min(row[0] for row in nexts if row[0])
except ValueError: break # No moar stuff
outputs = [common_key]
for i in range(len(nexts)): # code smell, sorry
output = []
while nexts[i][0] == common_key:
output.append(nexts[i])
nexts[i] = next(iters[i], (None,))
outputs.append(output)
yield outputs
Basically, it takes the lowest available key, then takes everything of
that key and yields it as a unit.
Code not tested. Use at your own risk.
ChrisA
More information about the Python-list
mailing list