Need suggestion to speed up code...

[Dave Cole]

>>>>> "Roy" == Roy Smith <roy at panix.com> writes:

Roy> I need to split up a string into a list of fields.  The strings
Roy> are value lists from SQL statements, and look something like
Roy> this:

Roy> (1, 'foo', 'bar', 34, 3.14159, 'an imbedded comma, this sting
Roy> has', 'this one isn''t so easy either')

Roy> If it wasn't for the fact that I need to handle commas and quotes
Roy> imbedded in quoted strings, it would be trivial -- just a call to
Roy> string.split.  But, as it is, the best I can figure out is to
Roy> walk the string, character by character, keeping track of what
Roy> state I'm in (parsing an integer, parsing a floating point, or
Roy> parsing a quoted string).  It works, but profiling shows it's the
Roy> bottleneck in my whole program.

Roy> Anybody have any idea for a better way to do this?

Roy> If I really had to, I suppose I could write this bit in C as an
Roy> extension module, but I'd rather do a pure python implementation
Roy> to keep it portable.

I have a CSV parsing extension module which might be close enough to
what you need.

        http://www.object-craft.com.au/projects/csv/

If you need some small modification made to make it more general, I
would be happy to accommodate you.

- Dave

-- 
http://www.object-craft.com.au