[CentralOH] Pickling data into a database

Mark Erbaugh mark at microenh.com
Wed Sep 3 20:32:36 CEST 2008

I'm working on a Python database application where the type and number
of fields that need to be saved vary from row to row. It is an
application that allows the user to configure my client's products to
their requirements. Depending on the specifics of the application, the
required data changes.  In the past, being an old database designer, I
just designed a database with enough columns to hold all the possible

However, the other day I had a "brainstorm".  Much of this data does not
need to be queried from the database.  I could streamline my database
design and store this variable data in Pickle'd format in a text field.
The data that does need to be queried would be kept in database fields
of the appropriate type.

One of the outputs of this application is a performance curve that needs
to be saved. In the current application, I create a one to many child
table that has the X and Y coordinates of the curve (one pair per row).
I could easily Pickle a sequence of tuples.  There are usually only a
dozen or fewer points on the curve (I think the worst case is about 40).

Is this a good approach?  What are the potential pitfalls?

One pitfall that just came to mind is that this limits the ability of
non-Python applications to process the data.  Maybe an alternative would
be to convert the data to XML instead of Pickling it?


More information about the CentralOH mailing list