<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE></TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3429" name=GENERATOR></HEAD>
<BODY>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Hi Guys,</SPAN><SPAN
style="FONT-SIZE: 10pt"><?xml:namespace prefix = o ns =
"urn:schemas-microsoft-com:office:office" /><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt"> <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">I need to create a Log
Manager application that is able to accept log files with various line (e.g.,
error messages) formatting, and create log objects that could be put into a
data-base.</SPAN><SPAN style="FONT-SIZE: 10pt"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt"> <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial"><U>Some</U> of the
requirements of this Log Manager, from the perspective of a client program would
be:</SPAN><SPAN style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"> <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">1) Open a
log file (in line/error-message format A) in the log_files_collection
(where a log is actually a set of several files, some of which might be
compressed).</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">2) Open a
log file (in line/error-message format B) in the log_files_collection
(where a log is actually a set of several files, some of which might be
compressed). etc...</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">3) Given a log and a
set of error pattern objects, generate a set of error objects that correspond to
error messages in the log. Each error object represents a normalized error
type, with fields from the error message text processed into attributes that
allow easy cross-referencing into a database.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">4) Given an
error object, retrieve the 'n' log entries on either side (above or below) of
the error message.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">5) Given an error
object, retrieve the log entries for 'n' seconds on either side of the error
message.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">6) Retrieve the
start and end dates for a log.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">7) Generate a
density histogram for all messages within a specified time period for a
log.</SPAN><SPAN style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">8) Generate a
density histogram for messages matching a specified pattern within a specified
time period for a log.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">9) Retrieve all log
entries matching a specified pattern within a time period.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"> <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">Envisioning how the
Log Manager might do each of the above, produces requirements which
include:</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"> <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">1) The log
manager must be able to order a set of related log files so that the client sees
a single, time-ordered collection of log messages.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">2) The log
manager must be sequentially accessible, both forward and backward.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">3) The log
manager must be randomly accessible by an event index.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">4) The log
manager must be randomly accessible by event time.</SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">5) The log
manager must allow the client to define error patterns that support regular
expression matching, and the ability to convert values embedded in the log
message text into keys that can be used to look up items in a ConnectivityGraph
database.</SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial"></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: #003300; FONT-FAMILY: Arial">(These are not all
the requirements, but you get the idea)</SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Googling around, I
found pyparsing which may help with some of the above
requirements.<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Anyone can suggest
other “wheels” already built that I could use and save me “reinventing”
them?</SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Namely, are there
Python modules that could perform the above actions ?</SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Thanks,<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Ron.</SPAN><SPAN
style="FONT-SIZE: 10pt"><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN
style="FONT-SIZE: 10pt"><o:p> </o:p></SPAN></P></BODY></HTML>