Convert AWK regex to Python

[Giacomo Boffi]

J <jnr.gonzalez at googlemail.com> writes:

> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service command status; do echo "Service: $service, Command: $command, Status: $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command | grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log; done
>
> This AWK command gets lines which look like this:-
>
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D <G_CC_SMS_SERVICE_51408_656.O_ CC_SMS_SERVICE_51408_656-ServerThread-VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 80000004 Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
>
> And outputs lines like this:-
>
> CC_SMS_SERVICE_51408 submit_resp: 0
>

i see some discrepancies in the description of your problem

1. if i echo a properly quoted line "like this" above in the pipeline
   formed by the first three awk commands i get

$ echo $likethis | awk -F\- '{print $1,$NF}' \
                 | awk -F\. '{print$1,$NF}'  \
                 | awk '{print $1,$4,$5}'
2011 ) )
$ 
   not a triple 'service command status'

2. with regard to the final product, you script outputs lines like in

echo "Service: $service, [...]"

   and you say that it produces lines like

CC_SMS_SERVICE_51408 submit_resp: 


WHATEVER, the abnormous run time is due to the fact that for every
output line you rescan again and again the whole log file

IF i had understood what you want, imho you should run your data
through sort and uniq -c

$ awk -F\- '{print $1,$NF}' < $file \
        | awk -F\. '{print$1,$NF}'  \
        | awk '{print $1,$4,$5}'    | sort | uniq -c | format_program

uniq -c drops repeated lines from a sorted input AND prepends to each
line the count of equal lines in the original stream

hth
                                                                g