speeding up string.split()

[Fredrik Aronsson]

In article <m27kz5k078.fsf at phosphorus.tucc.uab.edu>,
	Chris Green <cmg at uab.edu> writes:
> duncan at NOSPAMrcp.co.uk (Duncan Booth) writes:
> 
>> Chris Green <cmg at uab.edu> wrote in 
>> news:m2n182cs9c.fsf at phosphorus.tucc.uab.edu:
>> 
>> > Is there any way to speed up the following code?
> 
> Disappointed by the speed of splitting and then creating a dict, I
> got rid of the dict step since I knew it was positional in nature and
> then discovered that the split was a bottleneck.  I might be able to
> use split and map to do my bidding and will try that but I'm not very
> hopeful.

Fixed width fields?
In that case you can try splitting it by hand using slices
instead of using split.
I might be faster, but I havn't tried so YMMW.

>> 
>> 7. Use Perl, or C, or whatever else takes your fancy if speed is that 
>> critical.
> 
> I probably will have to use C to do the speed filterings and python to
> build an abstraction off that take the orignal approach.\

mxTextTools - Fast Text Manipulation Tools for Python
might be something for you. It's fast and written in c.
http://www.lemburg.com/files/python/mxTextTools.html

/Fredrik