Database experiences in Python: Good or Bad?

David Rushby woodsplitter at rocketmail.com
Mon Aug 12 13:06:26 CEST 2002


"Brad Clements" <bkc at Murkworks.com> wrote in message news:<3d51193d$1_8 at goliath.newsgroups.com>...
> I'm currently the maintainer for gvIB...
> Anyway ... I'm using Firebird (formerly was using Interbase) on Linux with
> Python and Zope in about 6 commercial projects.
> 
> I am starting to run into problems -- mostly due to the lack of threading
> support in the Interbase/Firebird client library (gds.so)
> 
> There are some problems with having multiple database connections from the
> same process...

  Yes, there are.  Maybe it'll irritate me (or you, or some other
crusader) enough to fix it one day.  My foremost free-time-priority
right now is to round out kinterbasdb's feature set (to get it
"squared away"--see this discussion:
http://sourceforge.net/forum/forum.php?thread_id=696302&forum_id=30917
) so that I can explore other areas of Firebird enhancement.  One
bothersome aspect of rearchitecting the client library is that
threading behavior is notoriously inconsistent across platforms, and
Firebird insists on supporting some bloody obscure platforms.

> ... even if these connections are used "one at a time" through a lock.

  Last time I checked (about fifteen months ago, when I corresponded
with you by e-mail), gvib released the GIL before most database API
calls.  Have you since experimented with a '"one at a time" through a
lock' approach?

  gvib's original approach (releasing the GIL) resulted in freezes for
me, which I eventually learned were because of the lack of
thread-safety in the Firebird client library.  Ultimately, gvib's
concurrency problems drove me to kinterbasdb, which had plenty of its
own instabilities, but no threading deadlocks.

  kinterbasdb is considerably less unstable now that I've sacrificied
my firstborn child to it (wink), but current and past versions of
kinterbasdb *never* release the GIL, so the GIL serializes all
database API calls within the process.  Though safe, that approach is
about as scalable as a twenty-five-year-old cart horse.  My RSN
enhancement plan calls for maintaining a separate lock to serialize
all database API calls within the process, while releasing the GIL to
allow non-kinterbasdb-calling Python threads to continue unimpeded
(see http://sourceforge.net/forum/message.php?msg_id=1611917 ).

  I've read that the Firebird Type 2 JDBC driver currently uses a
similar scheme.  Ultimately, of course, it's just an ugly hack and a
dead end (though not quite as dead as kinterbasdb's current total
serialization, or gvib's freezes).  So the Firebird client library
undeniably needs to be rearchitected.

> If I had the time, I'd try to fix gds itself, but I don't have any more free
> time and can't get my clients to pay for it.  
> 
> The firebird site isn't helpful, just lists it as a bug and I can't see any
> discussion on the issue.
> 
> I'd consider switching if I could find another database that supports
> transactions, same datatypes, good performance and low cost.

  Doesn't SAPDB meet those criteria?  I don't know how you define
"cost", and I haven't compared Firebird's performance to SAPDB's
(aside from the obvious fact that SAPDB is vastly more demanding in
the memory department, but also more SMP-scalable:
http://sourceforge.net/mailarchive/forum.php?thread_id=848329&forum_id=6330
).  SAPDB certainly supports transactions (including subtransactions)
and has a respectable set of datatypes.  SAPDB's client library is
almost entirely thread-safe, and its Python libraries take advantage
of that (to the extent that CPython can).

  Well, upon searching the SAPDB mailing list archives for "Python
thread safety", I found that you've apparently come to the same
conclusions.  Should I smile or frown?



More information about the Python-list mailing list