On Wed, Mar 29, 2006 at 11:47:10PM +0200, Thomas Wouters wrote:

Con:

...

* Competing Python wrappers exist * SQLite itself is updated frequently, let alone the wrappers * Build integration risks unknown, possible delay of 2.5? * Another external library to track and maybe have emergency updates of

All of these con arguments go for bsddb, too, and without sounding too negative about bsddb, I believe SQLite is a *much* better solution than BerkeleyDB, for roughly the same problem space. The same goes for pysqlite vs. bsddb. IMNSHO, SQLite and pysqlite are much easier to use correctly than BerkelyDB and bsddb, for simple and complex tasks. I may be biased against bsddb because I spent too much time hunting refleaks in it, but I'm not biased in favour of SQLite -- I'm a PostgreSQL user myself. ;-P

Agreed. sqlite is a joy to use. Its simple. It provides table structure that anyone can understand. BerkeleyDB is very powerful but requires a much more serious time investment to use usefully than sqlite for anything other than simple dictionary-like data storage. I wanted sqlite to exist for years. The intentionally undocumented bsddb.db.dbtables module i hacked together in early 2000 would never have been written had sqlite existed at the time. other things available at the time (gadfly anyone?) just didn't seem right.

probably a good thing. I would probably choose sqlite instead of shelve/anydbm/bsddb if it were part of the standard library, even though it's probably installed on all my machines anyway. I guess it's a psych thing.

As for people asking about deadlocks, well, I much rather explain about sqlite deadlocks than about BerkelyDB transactions.

yep.

skip@pobox.com wrote:

I haven't been tracking the pysqlite discussion either, but one con you missed is that regardless of pro #1 people will almost certainly apply it to problems for which it is ill-suited, reflectly poorly on both Python and SQLite.

the arguments keep getting more and more weird. is there *any* part of the standard Python distribution that cannot be applied to problems for which it is ill-suited? </F>

skip@pobox.com wrote:

The powers-that-be didn't want to support another database server (we already have Sybase) and didn't want our group's experimental data "polluting" the production database, so the folks who wanted it went the SQLite/pysqlite route. They were immediately bitten by the multiple reader/single writer limitation and they tried to cram too much data into it, so performance further sucked.

Firebird could be a solution to this. It can be used in a mode that doesn't need a server, and it has no trouble at all with concurrency or large amounts of data that I know of. In fact, a Firebird interface might be an alternative worth considering for the library. It would have most of the advantages of SQLite without these disadvantages. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing@canterbury.ac.nz +--------------------------------------+

Fredrik Lundh wrote:

Greg Ewing wrote:

...

Firebird could be a solution to this.

so a library that doesn't support multiple independent readers/writers on a single file at all is much better than one that does,

Where do you get that from? Firebird supports multiple readers/writers perfectly well. Greg

Fredrik Lundh wrote:

not according to the documentation (which says that the embedded library locks the database file, to prevent other independent processes from accessing the data).

I think that means other *non-Firebird* processes. Firebird itself uses a system of file locks and transactions to manage concurrent access by different Firebird applications. Just to make sure I wasn't imagining things, I ran two instances of a program of mine that uses Firebird, without server, accessing the same database, and both worked fine. They can independently access different parts of the database, and if they try to update the same part at the same time, deadlocks are reported as appropriate. On the whole I'm very impressed with Firebird. I've found it to be nothing less than sleek, efficient, powerful and reliable. I'd encourage anyone looking for a database, embedded or otherwise, to give it a go. -- Greg

At 04:00 PM 3/29/2006 -0600, skip@pobox.com wrote:

Is it not possible to distribute an empty db package which is then populated with various database eggs (or other similar installation things)?

I don't think I understand your question. If you are asking whether it's possible to have Java-like "namespace packages" in Python, the answer is yes. The stdlib module "pkgutil" supports this for regular filesystem packages, and the "pkg_resources" module in setuptools extends this support to zipfiles and eggs.

On Thursday 30 March 2006 12:15, skip@pobox.com wrote:

Someone was throwing around names like db.sqlite as the place to install pysqlite.

Dunno who originally suggested it, but the theory was that there's some issue with toplevel library namespace pollution. I'm not too stressed out one way or the other - but starting off with 'db.sqlite' (and then maybe moving/aliasing db.bsddb &c at a future point) doesn't seem like a bad idea.

That suggests other database interface modules like db.mysql, db.postgresql, db.sybase, etc. Given that we probably won't include all those as standard modules, we should make it easy for someone to install one or more of those modules via normal external mechanisms and have them appear seamlessly to the Python programmer. Then I begin to wonder why bother with db.sqlite at all. Why not just create an empty db package that does the pkgutil or pkg_resources dance and let people install all N database interfaces instead of just N-1?

The same could be said of vast amounts of the standard library. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.