Need suggestion to speed up code...

[Aahz Maruch]

In article <roy-54AC09.22222810052001 at news1.panix.com>,
Roy Smith  <roy at panix.com> wrote:
>
>I need to split up a string into a list of fields.  The strings are value 
>lists from SQL statements, and look something like this:
>
>(1, 'foo', 'bar', 34, 3.14159, 'an imbedded comma, this sting has', 'this 
>one isn''t so easy either')
>
>If it wasn't for the fact that I need to handle commas and quotes imbedded 
>in quoted strings, it would be trivial -- just a call to string.split.  
>But, as it is, the best I can figure out is to walk the string, character 
>by character, keeping track of what state I'm in (parsing an integer, 
>parsing a floating point, or parsing a quoted string).  It works, but 
>profiling shows it's the bottleneck in my whole program.

There are regular expressions that will handle this, but you should
probably consider one of the text processing tools available.  One of
the standard recommendations is mxTextTools.  If you take a look at the
Vaults of Parnassus, you're likely to find other options.
-- 
                      --- Aahz  <*>  (Copyright 2001 by aahz at pobox.com)

Androgynous poly kinky vanilla queer het Pythonista   http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6

"Everyone is entitled to an *informed* opinion."  --Harlan Ellison