Creating Fixed Length Records
drew at gremlinhosting.com
Thu Dec 9 19:03:54 CET 2004
Absolutely *perfect* case for a dose of Python string concatenation
performance theory or, How To Join Strings Together Fast:
On Wed, 2004-12-08 at 17:29 -0600, Greg Lindstrom wrote:
> I'm creating fixed-length record layouts for various record translations
> I need to perform. I have been using a home-grown object,
> "FixedLengthRecord" for about 4 years now and am very happy with it.
> Almost. The burr under my saddle occurs when I serialize the record.
> Currently, I create an empty text field and for each record in my
> database (all record layouts, data types, lengths, defaults, etc. are
> held in an SQL server) I concatenate the field to the text segment. I
> have been led to believe this is bad form because Python will copy the
> entire segment each time I add a field. Up until now, it was not a big
> deal because the segments had at most 20 fields. I have just been
> handed a record layout that is almost 5000 bytes long with 350 fields in
> it. Gulp!! Although I could let my object churn away on this bad boy,
> I'd like to know if there is a more pythonic way to serialize the record.
> One thought I had, which might lead to an addition to the language, was
> to use the struct module. If I could feed the pack method a format
> string then a tuple of values (instead of individual values), then I
> could create the format string once, then pass it a tuple with the
> values for that record. Just a thought.
> So, gurus, what are your suggestions to tame this record? Are there
> easier ways that I'm just not seeing?
Andrew James <drew at gremlinhosting.com>
More information about the Python-list