[Tutor] Iterator Merging

Stephen Nelson-Smith sanelson at gmail.com
Wed Nov 11 13:54:34 CET 2009


So, following Kent and Alan's advice, I've preprocessed my data, and
have code that produces 6 LogFile iterator objects:

>>> import magpie
>>> magpie.logs[1]
<magpie.LogFile instance at 0x2b8235debc20>
>>> dir(magpie.logs[1])
['__doc__', '__init__', '__iter__', '__module__', 'date', 'logfile',
'timestamp']

>>> for timestamp, entry in itertools.islice(magpie.logs[1], 3):
...   print timestamp, entry
...
[05/Nov/2009:04:02:13 +0000] 192.168.41.107 - - [05/Nov/2009:04:02:13
+0000] "GET http://sekrit.com/taxonomy/term/27908?page=111&item_884=1&year=66&form_id=dynamic_build_learning_objectives_form&level=121
HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
[05/Nov/2009:04:02:13 +0000] 66.249.165.22 - - [05/Nov/2009:04:02:13
+0000] "GET /taxonomy/term/27908?page=111&item_884=1&year=66&form_id=objectives_form&level=121
HTTP/1.1" 200 28736 "-" "Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
[05/Nov/2009:04:02:15 +0000] 162.172.185.126 - - [05/Nov/2009:04:02:15
+0000] "GET http://sekrit.com/sites/all/themes/liszt/images/backgrounds/grad_nav_5_h3.gif
HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible;)"

This is great.

So I have a list of 6 of these iterator objects.

Kent mentioned feeding them into an iterator merger.  I've got the
iterator merger in place too:

>>> from imerge import imerge
>>> imerge
<function imerge at 0x2b8235df6050>
>>> imerge([1,3,4],[2,7])
<generator object at 0x2b8235debf38>
>>> list(imerge([1,3,4],[2,7]))
[1, 2, 3, 4, 7]

What I'm trying to work out is how to feed the data I have - 6 streams
of timestamp, entry into imerge.

How can I do this?

S.

-- 
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com


More information about the Tutor mailing list