[DB-SIG] Re: DB-SIG digest, Vol 1 #86 - 5 msgs
vinay
vinay@itintegrated.com
Mon, 18 Jan 1999 23:00:17 +0500
At 12:01 PM 1/19/99 -0500, you wrote:
>Send DB-SIG maillist submissions to
> db-sig@python.org
>
>To subscribe or unsubscribe via the web, visit
> http://www.python.org/mailman/listinfo/db-sig
>or, via email, send a message with subject or body 'help' to
> db-sig-request@python.org
>You can reach the person managing the list at
> db-sig-admin@python.org
>
>(When replying, please edit your Subject line so it is more specific than
>"Re: Contents of DB-SIG digest...")
>
>Today's Topics:
>
> 1. API Enhancements (M.-A. Lemburg)
> 2. Re: oracledb memory leak (Jim Fulton)
> 3. Re: API Enhancements (Andy Dustman)
> 4. Re: API Enhancements (M.-A. Lemburg)
> 5. ODBC VC++ 5 Compilation (Donald A. Daugherty)
>Message: 1
>Sender: lemburg@vossnet.de
>Message-ID: <36A32169.2ECE2BD@lemburg.com>
>Date: Mon, 18 Jan 1999 12:56:25 +0100
>From: "M.-A. Lemburg" <mal@lemburg.com>
>Organization: IKDS
>MIME-Version: 1.0
>To: "DB-SIG @ Python.org" <db-sig@python.org>
>Subject: [DB-SIG] API Enhancements
>Content-Type: text/plain; charset=us-ascii
>Content-Transfer-Encoding: 7bit
>
>Hi everybody,
>
>It's been a long time since we've last had some discussion
>about a novelation of the API. As you probably remember I have
>a proposed spec available at:
>
> http://starship.skyport.net/~lemburg/DatabaseAPI-1.1.html
>
>The old one ist at:
>
> http://www.python.org/sigs/db-sig/DatabaseAPI.html
>
>I have some more points that I would like to clarify:
>
>1. Threading
>
>The API spec should make some notes about the scope of thread
>safety imposed on the interfaces. Thread safety could be given
>at module level (threads all use the same module, but maintain
>their own connections), connection level (threads share modules
>and connections, cursors are not shared between threads) and
>cursor level (threads share module, connections and cursors).
>
>The last level is probably not feasable, but I think connection
>level could be reached.
>
>2. Stored procedures
>
>Basically I want to revisit the discussion. The 1.1 proposal
>defines this interface:
>
>callproc(procname,list_of_parameters)
> This method is optional since not all databases
> provide stored procedures.
>
> Call a stored database procedure with the given
> name. The list of parameters must contain one
> entry for each argument that the procedure
> expects. The result of the call is returned by
> modifying the list contents in place. Input
> parameters are left untouched, output and
> input/output parameters replaced with the new
> values.
>
> The procedure may also provide a result set as
> output. This must then be made available through
> the standard fetch-methods.
>
>Is this general enough to fit everybody's needs ? I know that
>Jim Fulton would rather like an interface which returns a callable
>type... but it seems overkill to ask module writers to implement
>this just to be DB API conform.
>
>3. A standard catalog interface
>
>There is often a need to connect to a database without knowing
>in advance what tables it contains and how the table columns
>are named.
>
>The API should define a catalog method for this purpose,
>returning a sequence of all known objects of a certain
>type in the database, e.g. tables and views, in a predefined
>way, e.g.
>
>db.catalog('tables')
>
>ODBC has a wide variety of such functions, but they all
>return result sets with different schemas. Therefore I'd suggest
>having the interface not return a result set, but instead a Python
>list using a single format for all types of objects (passing None
>for non applicable or unkown entries):
>
>(qualifier,name,type,owner[,optional additional entries])
>
>To find out the column names a function could then use
>empty SELECTs and the cursor description attribute, e.g.
>c = db.cursor()
>c.execute('SELECT * FROM %s WHERE 1=0' % tablename)
># look at c.description
>
>4. Optional named cursors
>
>The cursor constructor should be allowed to have an optional
>argument for the cursor name. This is useful for UPDATEs.
>
>Comments ?
>
>--
>Marc-Andre Lemburg Y2000: 347 days left
>---------------------------------------------------------------------
> : Python Pages >>> http://starship.skyport.net/~lemburg/ :
> ---------------------------------------------------------
>
>Message: 2
>Message-ID: <36A37275.375B05C7@digicool.com>
>Date: Mon, 18 Jan 1999 12:42:13 -0500
>From: Jim Fulton <jim.fulton@digicool.com>
>Organization: Digital Creations, Inc.
>MIME-Version: 1.0
>To: arw@ifu.net
>CC: db-sig@python.org
>Subject: Re: [DB-SIG] oracledb memory leak
>References: <199901162217.RAA01375@python.org>
>Content-Type: text/plain; charset=us-ascii
>Content-Transfer-Encoding: 7bit
>
>arw@ifu.net wrote:
>>
>> <369ebdf4.919ec6d-@email.sps.mot.com> wrote:
>> Original Article: http://www.egroups.com/list/db-sig/?start=596
>> > Hi All,
>> >
>> > I am using the oracledb extension on HP-UX 10.20 for an intensive and
>> > persistent process and am seeing memory leaks that consumes over 100 Mgs
>> >
>> > before the process finally crashes. Has anyone seen a memory leak
>> > problem with the oracledb extension?
>> >
>> > Does anyone know if there is a Python database extension that is
>> > comparable (i.e similar functionallty, API's) to oracledb that will run
>> > on HP-UX, AIX and NT 4.0?
>> >
>> > Thanks for the help,
>> >
>> > Larry
>>
>> You didn't specify whether you wanted to stick with Oracle or not.
>> If you do, maybe have a look at the odbc or mxodbc modules.
>
>Oracle's ODBC drivers leak bad.
>
>Jim
>
>--
>Jim Fulton mailto:jim@digicool.com
>Technical Director (888) 344-4332 Python Powered!
>Digital Creations http://www.digicool.com http://www.python.org
>
>Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
>address may not be added to any commercial mail list with out my
>permission. Violation of my privacy with advertising or SPAM will
>result in a suit for a MINIMUM of $500 damages/incident, $1500 for
>repeats.
>Message: 3
>Date: Mon, 18 Jan 1999 12:59:08 -0500 (EST)
>From: Andy Dustman <adustman@comstar.net>
>To: "M.-A. Lemburg" <mal@lemburg.com>
>cc: "DB-SIG @ Python.org" <db-sig@python.org>
>Subject: Re: [DB-SIG] API Enhancements
>In-Reply-To: <36A32169.2ECE2BD@lemburg.com>
>Message-ID: <Pine.LNX.4.04.9901181232140.2056-100000@kenny.comstar.net>
>Copyright: 1999 by Andy Dustman
>MIME-Version: 1.0
>Content-Type: TEXT/PLAIN; charset=US-ASCII
>
>On Mon, 18 Jan 1999, M.-A. Lemburg wrote:
>
>> 1. Threading
>>
>> The API spec should make some notes about the scope of thread
>> safety imposed on the interfaces. Thread safety could be given
>> at module level (threads all use the same module, but maintain
>> their own connections), connection level (threads share modules
>> and connections, cursors are not shared between threads) and
>> cursor level (threads share module, connections and cursors).
>>
>> The last level is probably not feasable, but I think connection
>> level could be reached.
>
>This, of course, depends on what sort of threading support is built into
>the database. Part of the reason this question is being asked is because
>of a little e-mail discussion between me and Marc...
>
>Basically, I have a server app where I have wanted to have a threaded
>server using mxODBC, but I could never use it because I always ran into a
>memory leak, so I had to switch to a forked model (not a big pain with
>SocketServer). However, M.-A. has now found the leak so I can think about
>doing this again with mxODBC.
>
>C extensions in Python are basically thread-safe by default, due to the
>global interpreter lock: While the C extension is running, it has the
>lock, so no other thread can run. This, by itself, would make a pretty
>useless threading implementation, so Guido has some macros in the Python
>API to give up and reacquire the lock around sections of code that would
>block for arbitrarily long periods of time: Py_BEGIN_ALLOW_THREADS and
>Py_END_ALLOW_THREADS. Judicious use of this construct makes a C extension
>module what I call thread-friendly. I guess you can manipulate some Python
>objects to some extent, but nobody else can be in a position to notice
>(like modifying tuples: You can do it, as long as nobody else knows it
>exists yet).
>
>I believe that in practice, if the database's client library is
>thread-safe, then the Python interface module for it needs to be
>thread-friendly, or else there will be poor performance in threads: Even
>if each thread has it's own connection, only one can be using the database
>module at any given time, and database accesses will block all threads for
>arbitrarily long periods of time.
>
>Generally I don't think threads should try to share connections mainly
>because even if it is thread-safe, it has some very strange implications
>for transactions. It may mess up client-server protocols as well. So
>sharing cursors becomes a moot point.
>
>BTW, I have developed a patch against MySQLmodule-1.4 which makes it
>thread-friendly; however, I have not been able to test it very well. Any
>volunteers.
>
>> 4. Optional named cursors
>>
>> The cursor constructor should be allowed to have an optional
>> argument for the cursor name. This is useful for UPDATEs.
>
>This is so easy to implement (at least for ODBC) that there's no good
>reason not to do it. Where it doesn't make sense, the API can just ignore
>the optional argument.
>
>--
>Andy Dustman You should always say "spam" and "eggs"
>ComStar Communications Corp. instead of "foo" and "bar"
>(706) 549-7689 | PGP KeyID=0xC72F3F1D in Python examples. (Mark Lutz)
>
>
>Message: 4
>Sender: lemburg@vossnet.de
>Message-ID: <36A38BD2.67E09E67@lemburg.com>
>Date: Mon, 18 Jan 1999 20:30:26 +0100
>From: "M.-A. Lemburg" <mal@lemburg.com>
>Organization: IKDS
>MIME-Version: 1.0
>To: Andy Dustman <adustman@comstar.net>
>CC: "DB-SIG @ Python.org" <db-sig@python.org>
>Subject: Re: [DB-SIG] API Enhancements
>References: <Pine.LNX.4.04.9901181232140.2056-100000@kenny.comstar.net>
>Content-Type: text/plain; charset=us-ascii
>Content-Transfer-Encoding: 7bit
>
>Andy Dustman wrote:
>>
>> On Mon, 18 Jan 1999, M.-A. Lemburg wrote:
>>
>> > 1. Threading
>> >
>> > The API spec should make some notes about the scope of thread
>> > safety imposed on the interfaces. Thread safety could be given
>> > at module level (threads all use the same module, but maintain
>> > their own connections), connection level (threads share modules
>> > and connections, cursors are not shared between threads) and
>> > cursor level (threads share module, connections and cursors).
>> >
>> > The last level is probably not feasable, but I think connection
>> > level could be reached.
>>
>> This, of course, depends on what sort of threading support is built into
>> the database. Part of the reason this question is being asked is because
>> of a little e-mail discussion between me and Marc...
>>
>> Basically, I have a server app where I have wanted to have a threaded
>> server using mxODBC, but I could never use it because I always ran into a
>> memory leak, so I had to switch to a forked model (not a big pain with
>> SocketServer). However, M.-A. has now found the leak so I can think about
>> doing this again with mxODBC.
>
>Actually, I found a different leakage than the one announced on
>c.l.p. They will both be fixed in the next release which is due later
>this week.
>
>> C extensions in Python are basically thread-safe by default, due to the
>> global interpreter lock: While the C extension is running, it has the
>> lock, so no other thread can run. This, by itself, would make a pretty
>> useless threading implementation, so Guido has some macros in the Python
>> API to give up and reacquire the lock around sections of code that would
>> block for arbitrarily long periods of time: Py_BEGIN_ALLOW_THREADS and
>> Py_END_ALLOW_THREADS. Judicious use of this construct makes a C extension
>> module what I call thread-friendly. I guess you can manipulate some Python
>> objects to some extent, but nobody else can be in a position to notice
>> (like modifying tuples: You can do it, as long as nobody else knows it
>> exists yet).
>>
>> I believe that in practice, if the database's client library is
>> thread-safe, then the Python interface module for it needs to be
>> thread-friendly, or else there will be poor performance in threads: Even
>> if each thread has it's own connection, only one can be using the database
>> module at any given time, and database accesses will block all threads for
>> arbitrarily long periods of time.
>>
>> Generally I don't think threads should try to share connections mainly
>> because even if it is thread-safe, it has some very strange implications
>> for transactions. It may mess up client-server protocols as well. So
>> sharing cursors becomes a moot point.
>
>I've added those thread macros around most long running calls, e.g.
>those preparing, executing and fetching data and connects, in mxODBC.
>Since I'm not running threaded applications with it, someone else
>will have to check whether those macros do no harm to the data integrity
>of the various supported databases.
>
>--
>Marc-Andre Lemburg Y2000: 347 days left
>---------------------------------------------------------------------
> : Python Pages >>> http://starship.skyport.net/~lemburg/ :
> ---------------------------------------------------------
>
>Message: 5
>From: "Donald A. Daugherty" <bigd@europa.com>
>To: <db-sig@python.org>
>Subject: [DB-SIG] ODBC VC++ 5 Compilation
>Date: Tue, 19 Jan 1999 01:09:13 -0800
>Message-ID: <000001be438b$621c75a0$cc36cacc@bigd>
>MIME-Version: 1.0
>Content-Type: text/plain;
> charset="iso-8859-1"
>Content-Transfer-Encoding: 7bit
>Importance: Normal
>
>I get numerous errors from wchar.h and then a whole host of other errors
>when I try to compile (I downloaded this just today).
>
>I followed the instructions to the letter (I am only learning VC++ and my
>C/C++ experience is limited). Can you shed any light on this? I am
>downloading the current M$ universal data access SDK, it was the only thing
>I saw when I went looking for the ODBC SDK. It implies that it includes the
>ODBC stuff as well. Let me know, I am very interested in getting this
>compiled. Thanks!
>
>Donald A. Daugherty
>--------------------Configuration: ODBC - Win32 Debug--------------------
>Compiling...
>mxODBC.cpp
>G:\DevStudio\VC\INCLUDE\wchar.h(820) : error C2733: second C linkage of
>overloaded function 'wmemchr' not allowed
>G:\DevStudio\VC\INCLUDE\wchar.h(822) : error C2733: second C linkage of
>overloaded function 'wcschr' not allowed
>G:\DevStudio\VC\INCLUDE\wchar.h(824) : error C2733: second C linkage of
>overloaded function 'wcspbrk' not allowed
>G:\DevStudio\VC\INCLUDE\wchar.h(826) : error C2733: second C linkage of
>overloaded function 'wcsrchr' not allowed
>G:\DevStudio\VC\INCLUDE\wchar.h(828) : error C2733: second C linkage of
>overloaded function 'wcsstr' not allowed
>G:\Python\Odbc\Windows\mxODBC.cpp(308) : error C2059: syntax error :
>'__stdcall'
>G:\Python\Odbc\Windows\mxODBC.cpp(2251) : error C2664:
>'PyDict_GetItemString' : cannot convert parameter 2 from 'unsigned char
>[10]' to 'char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(2779) : warning C4273: 'strtod' :
>inconsistent dll linkage. dllexport assumed.
>G:\Python\Odbc\Windows\mxODBC.cpp(2795) : warning C4018: '==' :
>signed/unsigned mismatch
>G:\Python\Odbc\Windows\mxODBC.cpp(2851) : warning C4018: '==' :
>signed/unsigned mismatch
>G:\Python\Odbc\Windows\mxODBC.cpp(3936) : error C2664: 'SQLDescribeCol' :
>cannot convert parameter 3 from 'char [20]' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4611) : error C2664: 'SQLTables' : cannot
>convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4655) : error C2664: 'SQLTablePrivileges'
>: cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4700) : error C2664: 'SQLColumns' : cannot
>convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4745) : error C2664: 'SQLColumnPrivileges'
>: cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4793) : error C2664: 'SQLForeignKeys' :
>cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4837) : error C2664: 'SQLPrimaryKeys' :
>cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4881) : error C2664: 'SQLProcedures' :
>cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4927) : error C2664: 'SQLProcedureColumns'
>: cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(4977) : error C2664: 'SQLSpecialColumns' :
>cannot convert parameter 3 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(5024) : error C2664: 'SQLStatistics' :
>cannot convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(5162) : error C2086: 'mxODBCursor_Type' :
>redefinition
>G:\Python\Odbc\Windows\mxODBC.cpp(5265) : error C2664: 'SQLConnect' : cannot
>convert parameter 2 from 'char *' to 'unsigned char *'
>G:\Python\Odbc\Windows\mxODBC.cpp(5565) : error C2086: 'mxODBC_Type' :
>redefinition
>G:\Python\Odbc\Windows\mxODBC.cpp(5797) : error C2070: illegal sizeof
>operand
>Error executing cl.exe.
>
>ODBC.dll - 22 error(s), 3 warning(s)
>
>
>
>