[Tutor] Python database options

Brad Reisfeld brad.reisfeld@colostate.edu
Mon, 11 Feb 2002 06:52:20 -0700

I have inherited a large tab-delimited text file containing about 150,000
records, each of which has about 10 fields. I am interested in using some
sort of very simple 'database' to search within the file (i.e. search
through a field for a given item and retrieve the record). These 'searches'
will be part of another application and will need to be fast.

I know next to nothing about databases, but have, through some research
identified a number of options:

1. dictionary + pickle
2. dictionary + shelve
3. Gadfly
4. Berkeley DB + Python bindings
5. Postgres, MySQL + Python bindings

The first couple of options relate reading in the file, creating a
dictionary, and making it persistent. The Gadfly option looks good, but I'm
not sure whether or not it is still supported and works under later versions
of Python. The last couple of options are for 'true' database manager
packages: perhaps overkill for this application.

What are the pros and cons of the options? Are there other viable options?

I'm using Python 2.1 on Linux RH7.2.

Thank you for your help in sorting this out.