Why is text file processing SO slow?

[jaimewyant at my-deja.com]

I have a program that processes large files ~25 megs.  The algorithm
that I use is as follows:

while not EOF:
    readline
    store line in list

    if line == 'End of set' marker
        dump the list to appopriate file (produces 2 sep files)
        empty the list
    elsif line contains 'EDS' regexp
        set the eds flag


The above algorithm is only slightly stripped down.  The above program
written in Python takes 55 seconds to execute.  The same program
written in perl takes 15 seconds.  The C version takes 10 seconds.

Is it typically a BAD idea to use Python (1.5.2) to process text files?

TIA,
jaime


Sent via Deja.com http://www.deja.com/
Before you buy.