[DB-SIG] API Enhancements

Andy Dustman adustman@comstar.net
Mon, 18 Jan 1999 12:59:08 -0500 (EST)


On Mon, 18 Jan 1999, M.-A. Lemburg wrote:

> 1. Threading
> 
> The API spec should make some notes about the scope of thread
> safety imposed on the interfaces. Thread safety could be given
> at module level (threads all use the same module, but maintain
> their own connections), connection level (threads share modules
> and connections, cursors are not shared between threads) and
> cursor level (threads share module, connections and cursors).
> 
> The last level is probably not feasable, but I think connection
> level could be reached.

This, of course, depends on what sort of threading support is built into
the database. Part of the reason this question is being asked is because
of a little e-mail discussion between me and Marc...

Basically, I have a server app where I have wanted to have a threaded
server using mxODBC, but I could never use it because I always ran into a
memory leak, so I had to switch to a forked model (not a big pain with
SocketServer). However, M.-A. has now found the leak so I can think about
doing this again with mxODBC.

C extensions in Python are basically thread-safe by default, due to the
global interpreter lock: While the C extension is running, it has the
lock, so no other thread can run. This, by itself, would make a pretty
useless threading implementation, so Guido has some macros in the Python
API to give up and reacquire the lock around sections of code that would
block for arbitrarily long periods of time: Py_BEGIN_ALLOW_THREADS and
Py_END_ALLOW_THREADS. Judicious use of this construct makes a C extension
module what I call thread-friendly. I guess you can manipulate some Python
objects to some extent, but nobody else can be in a position to notice
(like modifying tuples: You can do it, as long as nobody else knows it
exists yet).

I believe that in practice, if the database's client library is
thread-safe, then the Python interface module for it needs to be
thread-friendly, or else there will be poor performance in threads: Even
if each thread has it's own connection, only one can be using the database
module at any given time, and database accesses will block all threads for
arbitrarily long periods of time.

Generally I don't think threads should try to share connections mainly
because even if it is thread-safe, it has some very strange implications
for transactions. It may mess up client-server protocols as well. So
sharing cursors becomes a moot point.

BTW, I have developed a patch against MySQLmodule-1.4 which makes it
thread-friendly; however, I have not been able to test it very well. Any
volunteers.

> 4. Optional named cursors
> 
> The cursor constructor should be allowed to have an optional
> argument for the cursor name. This is useful for UPDATEs.

This is so easy to implement (at least for ODBC) that there's no good
reason not to do it. Where it doesn't make sense, the API can just ignore
the optional argument. 

-- 
Andy Dustman                    You should always say "spam" and "eggs"
ComStar Communications Corp.                 instead of "foo" and "bar"
(706) 549-7689 | PGP KeyID=0xC72F3F1D   in Python examples. (Mark Lutz)