[ANNOUNCE] KirbyBase 1.0
jcribbs at twmi.rr.com
Wed Apr 9 06:21:45 CEST 2003
Announcing the initial release of KirbyBase.
It can be downloaded from http://www.netpromi.com/files/kirbybase1.0.zip
What is KirbyBase?
KirbyBase is a simple, pure-python, flat-file database management
system, with a few twists.
1. All data is kept in plain-text, newline delimited files that can be
edited by hand. This gives you the ability to make changes by just
opening the file up in a text editor, or you can use another programming
language to read the file in and do things with it.
2. KirbyBase can either be used as an embedded database or in a
client/server, multi-user mode. To switch from one mode to the other,
you only have to change one line in your program. Included in the
distribution are two sample database server scripts. One is
single-threaded, blocking, and pretty safe. The other one is an
experimental mult-threaded, non-blocking script. So far it has worked
well, but it probably needs more testing.
3. KirbyBase is not an "in-memory" database. The entire database is NOT
read into memory at initialization and then completely written back out
to disk when it is closed. KirbyBase tables are kept on disk and
accessed from disk when selecting, updating, inserting, and deleting.
Changes to the database are written immediately to disk. This has the
advantage of accomodating larger databases, because you aren't taking up
lots of memory to hold the entire database. Additionally, since changes
are immediately written to disk, the chance of lost data due to power
interruptions, crashes, etc. should be smaller.
4. KirbyBase allows you to specify field types, then it allows you to
use Python's regular expression syntax to search by string fields and
Python expressions to search by numeric fields. So, for exampe, to
search a table containing info on WWII fighter planes and select US
planes that had a maximun speed higher than 400 mph, you would say
db.select('plane.tbl', ['country', 'speed'], ['USA', '>400'])
I like doing this more than building SQL query strings in Python.
That's about it. There are a few other goodies, like automatic
auto-incrementing primary keys on each record and a kind of interesting
way to keep updates quick by trying to write back to the same spot in
the physical file, but it's mostly internal stuff.
What's it good for?
I'm not sure. :) If you are looking for an easy to use dbms that has a
syntax that is a little more "python friendly" than SQL, has an internal
data format that is easily accessible (text files), doesn't take up much
room (one file, 34K), that can be used embedded or client/server, and
that allows you to change how it works (source code is there for you to
see/modify), then you might find KirbyBase interesting. I guess I see
it fitting somewhere in between pysqlite and plain text files for your
Where did KirbyBase come from?
A lot of the ideas and concepts for KirbyBase came from a tcl dbms
script written by Dave Maggiano. Dave included this script on the cdrom
that accompanied his book "CGI Programming with Tcl". I bought the
book, which is excellent, and, in the course of reading it, discovered
his dbms script. I have a soft-spot for small, simple, cleverly
designed database management systems, and Dave's was one of the coolest
I have seen. The icing on the cake was that I could actually understand
and follow the source code! I'm no genius at programming, so that was
definitely a plus.
Well, I used his dbms in a few Tcl projects at work and it performed
flawlessly. However, when I started using other languages like Python
and Ruby, I had to leave Dave's tcl-based dbms behind. I told myself
that one day I would convert it into another language.
Well, lately I have been learning Python while writing a gui program
using pygtk. I really like the combination of Python and Gtk, so I
plunged in and decided to port Dave's dbms to Python. As I started to
do this, I noticed a lot of things that could be done differently
because, of course, Python is a different language. Also, because Dave
and I are unique human beings, I also took the opportunity to add some
functionality that Dave's script didn't have, and take some stuff away
that I didn't find as important. The end result is a program that uses
a lot of Dave's concepts, but has some ideas of its own.
I have tried, in vain, to contact Dave and get his blessing/permission
for all of this. I have looked through his book, source code, and cdrom
to try to find out what license his script is under, but I have been
unable to find anything. I have tried three different emails that were
supposed to be his, but received no response. I even posted a message
to the Tcl mailing list asking if anyone knew how I could contact Dave,
but no one did. So, Dave, if you read this...please contact me. I want
to, first of all, thank you for such an awesome piece of software, and,
secondly, make sure you are ok with what I am doing. I'm not directly
using any code you wrote, but I am definitely borrowing some concepts
from you and I want to make sure that is alright with you.
Where to go from here?
KirbyBase can be downloaded from
http://www.netpromi.com/files/kirbybase1.0.zip I don't know if it will
prove of value to anyone, but me. I do hope some of you could take a
look at it and give me feedback. I am sure that there are a lot of code
improvements that can be made and I would welcome any suggestions.
Right now, it performs pretty well on small databases, but starts to
drop off when doing large queries against 50,000+ record databases.
Mostly, writing KirbyBase fulfilled a promise I made to myself. I also
am using it for a few projects at work. If nothing else, I have learned
a LOT about Python, just by writing KirbyBase, and, it was actually kind
of fun doing it.
jcribbs at twmi.rr.com
More information about the Python-list