[ANNOUNCE] KirbyBase 1.0

Wed Apr 9 00:21:45 EDT 2003

Announcing the initial release of KirbyBase.

It can be downloaded from http://www.netpromi.com/files/kirbybase1.0.zip

What is KirbyBase?
---------------------

KirbyBase is a simple, pure-python, flat-file database management 
system, with a few twists.

1. All data is kept in plain-text, newline delimited files that can be 
edited by hand.  This gives you the ability to make changes by just 
opening the file up in a text editor, or you can use another programming 
language to read the file in and do things with it.

2. KirbyBase can either be used as an embedded database or in a 
client/server, multi-user mode.  To switch from one mode to the other, 
you only have to change one line in your program.  Included in the 
distribution are two sample database server scripts.  One is 
single-threaded, blocking, and pretty safe.  The other one is an 
experimental mult-threaded, non-blocking script.  So far it has worked 
well, but it probably needs more testing.

3. KirbyBase is not an "in-memory" database.  The entire database is NOT 
read into memory at initialization and then completely written back out 
to disk when it is closed.  KirbyBase tables are kept on disk and 
accessed from disk when selecting, updating, inserting, and deleting.  
Changes to the database are written immediately to disk.  This has the 
advantage of accomodating larger databases, because you aren't taking up 
lots of memory to hold the entire database.  Additionally, since changes 
are immediately written to disk, the chance of lost data due to power 
interruptions, crashes, etc.  should be smaller.

4. KirbyBase allows you to specify field types, then it  allows you to 
use Python's regular expression syntax to search by string fields and 
Python expressions to search by numeric fields.  So, for exampe, to 
search a table containing info on WWII fighter planes and select US 
planes that had a maximun speed higher than 400 mph, you would say 
something like:

     db.select('plane.tbl', ['country', 'speed'], ['USA', '>400'])

I like doing this more than building SQL query strings in Python.

That's about it.  There are a few other goodies, like automatic 
auto-incrementing primary keys on each record and a kind of interesting 
way to keep updates quick by trying to write back to the same spot in 
the physical file, but it's mostly internal stuff.

What's it good for?
-------------------

I'm not sure.  :)  If you are looking for an easy to use dbms that has a 
syntax that is a little more "python friendly" than SQL, has an internal 
data format that is easily accessible (text files), doesn't take up much 
room (one file, 34K), that can be used embedded or client/server, and 
that allows you to change how it works (source code is there for you to 
see/modify), then you might find KirbyBase interesting.  I guess I see 
it fitting somewhere in between pysqlite and plain text files for your 
dbms needs.

Where did KirbyBase come from?
-----------------------------------

A lot of the ideas and concepts for KirbyBase came from a tcl dbms 
script written by Dave Maggiano.  Dave included this script on the cdrom 
that accompanied his book "CGI Programming with Tcl".  I bought the 
book, which is excellent, and, in the course of reading it, discovered 
his dbms script.  I have a soft-spot for small, simple, cleverly 
designed database management systems, and Dave's was one of the coolest 
I have seen.  The icing on the cake was that I could actually understand 
and follow the source code!  I'm no genius at programming, so that was 
definitely a plus.

Well, I used his dbms in a few Tcl projects at work and it performed 
flawlessly.  However, when I started using other languages like Python 
and Ruby, I had to leave Dave's tcl-based dbms behind.  I told myself 
that one day I would convert it into another language.

Well, lately I have been learning Python while writing a gui program 
using pygtk.  I really like the combination of Python and Gtk, so I 
plunged in and decided to port Dave's dbms to Python.  As I started to 
do this, I noticed a lot of things that could be done differently 
because, of course, Python is a different language.  Also, because Dave 
and I are unique human beings, I also took the opportunity to add some 
functionality that Dave's script didn't have, and take some stuff away 
that I didn't find as important.  The end result is a program that uses 
a lot of Dave's concepts, but has some ideas of its own.

I have tried, in vain, to contact Dave and get his blessing/permission 
for all of this.  I have looked through his book, source code, and cdrom 
to try to find out what license his script is under, but I have been 
unable to find anything.  I have tried three different emails that were 
supposed to be his, but received no response.  I even posted a message 
to the Tcl mailing list asking if anyone knew how I could contact Dave, 
but no one did.  So, Dave, if you read this...please contact me.  I want 
to, first of all, thank you for such an awesome piece of software, and, 
secondly, make sure you are ok with what I am doing.  I'm not directly 
using any code you wrote, but I am definitely borrowing some concepts 
from you and I want to make sure that is alright with you.

Where to go from here?
------------------------

KirbyBase can be downloaded from 
http://www.netpromi.com/files/kirbybase1.0.zip I don't know if it will 
prove of value to anyone, but me.  I do hope some of you could take a 
look at it and give me feedback.  I am sure that there are a lot of code 
improvements that can be made and I would welcome any suggestions.  
Right now, it performs pretty well on small databases, but starts to 
drop off when doing large queries against 50,000+ record databases.  
Mostly, writing KirbyBase fulfilled a promise I made to myself.  I also 
am using it for a few projects at work.  If nothing else, I have learned 
a LOT about Python, just by writing KirbyBase, and, it was actually kind 
of fun doing it. 

Jamey Cribbs
jcribbs at twmi.rr.com