[DB-SIG] API Enhancements

[M.-A. Lemburg]

Andy Dustman wrote:
> 
> On Mon, 18 Jan 1999, M.-A. Lemburg wrote:
> 
> > 1. Threading
> >
> > The API spec should make some notes about the scope of thread
> > safety imposed on the interfaces. Thread safety could be given
> > at module level (threads all use the same module, but maintain
> > their own connections), connection level (threads share modules
> > and connections, cursors are not shared between threads) and
> > cursor level (threads share module, connections and cursors).
> >
> > The last level is probably not feasable, but I think connection
> > level could be reached.
> 
> This, of course, depends on what sort of threading support is built into
> the database. Part of the reason this question is being asked is because
> of a little e-mail discussion between me and Marc...
> 
> Basically, I have a server app where I have wanted to have a threaded
> server using mxODBC, but I could never use it because I always ran into a
> memory leak, so I had to switch to a forked model (not a big pain with
> SocketServer). However, M.-A. has now found the leak so I can think about
> doing this again with mxODBC.

Actually, I found a different leakage than the one announced on
c.l.p. They will both be fixed in the next release which is due later
this week.

> C extensions in Python are basically thread-safe by default, due to the
> global interpreter lock: While the C extension is running, it has the
> lock, so no other thread can run. This, by itself, would make a pretty
> useless threading implementation, so Guido has some macros in the Python
> API to give up and reacquire the lock around sections of code that would
> block for arbitrarily long periods of time: Py_BEGIN_ALLOW_THREADS and
> Py_END_ALLOW_THREADS. Judicious use of this construct makes a C extension
> module what I call thread-friendly. I guess you can manipulate some Python
> objects to some extent, but nobody else can be in a position to notice
> (like modifying tuples: You can do it, as long as nobody else knows it
> exists yet).
> 
> I believe that in practice, if the database's client library is
> thread-safe, then the Python interface module for it needs to be
> thread-friendly, or else there will be poor performance in threads: Even
> if each thread has it's own connection, only one can be using the database
> module at any given time, and database accesses will block all threads for
> arbitrarily long periods of time.
> 
> Generally I don't think threads should try to share connections mainly
> because even if it is thread-safe, it has some very strange implications
> for transactions. It may mess up client-server protocols as well. So
> sharing cursors becomes a moot point.

I've added those thread macros around most long running calls, e.g.
those preparing, executing and fetching data and connects, in mxODBC.
Since I'm not running threaded applications with it, someone else 
will have to check whether those macros do no harm to the data integrity
of the various supported databases.

-- 
Marc-Andre Lemburg                               Y2000: 347 days left
---------------------------------------------------------------------
          : Python Pages >>> http://starship.skyport.net/~lemburg/  :
           ---------------------------------------------------------