From info at egenix.com Tue Apr 9 10:25:27 2013 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Tue, 09 Apr 2013 10:25:27 +0200 Subject: [DB-SIG] ANN: eGenix mxODBC Connect - Python ODBC Database Interface 2.0.3 Message-ID: <5163D077.6060905@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC Connect Python ODBC Database Interface Version 2.0.3 mxODBC Connect is our commercially supported client-server product for connecting Python applications to relational databases in a truly platform independent way. This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Connect-2.0.3-GA.html ________________________________________________________________________ INTRODUCTION The mxODBC Connect Database Interface for Python allows users to easily connect Python applications to all major databases on the market today in a highly portable, convenient and secure way. Python Database Connectivity the Easy Way ----------------------------------------- Unlike our mxODBC Python extension, mxODBC Connect is designed as client-server application, so you no longer need to find production quality ODBC drivers for all the platforms you target with your Python application. Instead you use an easy to install royalty-free Python client library which connects directly to the mxODBC Connect database server over the network. This makes mxODBC Connect a great basis for writing cross-platform multi-tier database applications and utilities in Python, especially if you run applications that need to communicate with databases such as MS SQL Server and MS Access, Oracle Database, IBM DB2 and Informix, Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more, that run on Windows or Linux machines. Ideal for Database Driven Client Applications --------------------------------------------- By removing the need to install and configure ODBC drivers on the client side and dealing with complicated network setups for each set of drivers, mxODBC Connect greatly simplifies deployment of database driven client applications, while at the same time making the network communication between client and database server more efficient and more secure. For more information, please have a look at the mxODBC Connect product page, in particular, the full list of available features. For more information, please see the product page: http://www.egenix.com/products/python/mxODBCConnect/ ________________________________________________________________________ NEWS The 2.0.3 release of mxODBC Connect includes the following enhancements and fixes: Security Enhancements --------------------- * Upgraded client and server to the most recent eGenix pyOpenSSL 0.13.1.1.0.1.5. Server Enhancements ------------------- * Updated the server to use mxODBC 3.2.2 for database connectivity. Client Enhancements ------------------- * Added support for the new .cursortype attribute in mxODBC 3.2.2 which is available on both connections and cursors to adjust the used ODBC cursor type. Performance Enhancements ------------------------ * MS SQL Server performance can now be much enhanced, and increased to levels beyond that of mxODBC Connect 2.0.2 and previous releases, by adjusting the default cursor type to forward-only cursors instead of static cursors: # Connect to the remote database from mx.ODBCConnect.Client import ServerSession session = ServerSession(...) ODBC = session.open() connection = ODBC.DriverConnect(...) # Use the faster forward-only cursors connection.cursortype = ODBC.SQL.CURSOR_FORWARD_ONLY # Cursors created on this connection will then default # to forward only cursors cursor = connection.cursor() * The performance increase compared to mxODBC Connect 2.0.2 is enormous: from 2-3x faster executes/fetches for average queries, up to 300x faster for simple cases. In mxODBC Connect 2.1, we will switch to using forward-only cursors per default for all database backends. * IBM DB2 can benefit from the same performance enhancements using forward-only cursors. The effect is a lot smaller compared to MS SQL Server, but still noticeable: up to 2x faster executes/fetches with forward-only cursors, compared to mxODBC Connect 2.0.2. * Added documentation to explain the different cursor types, compatibility with different database backends and effects on performance to the mxODBC documentation. New License Structure --------------------- Due to popular demand, we have added an unlimited connections server license, so now you can choose between: * the entry-level 20-connections server license, * an average use 50-connections server license. * and the unlimited connections server license for our high-end users. For the full set of changes, please check the mxODBC Connect change log. http://www.egenix.com/products/python/mxODBCConnect/changelog.html mxODBC Connect 2.0 Highlights ----------------------------- mxODBC Connect 2.0 was released on 2012-08-20. These are the most important highlights: * mxODBC Connect Server now uses mxODBC 3.2 internally and makes its API available in the mxODBC Connect Client. This is a major step forward from the mxODBC 3.0 version used in mxODBC Connect Server 1.0. * We've added native Windows x64 builds. * mxODBC Connect Client now integrates directly with gevent, allowing client applications to run asynchronous tasks while performing remote database queries. Please see the release announcement for full details: http://www.egenix.com/company/news/eGenix-mxODBC-Connect-2.0.0-GA.html ________________________________________________________________________ UPGRADING You are encouraged to upgrade to this latest mxODBC Connect release. When upgrading, please always upgrade both the server and the client installations to the same version - even for patch level releases. Customers who have purchased mxODBC Connect 2.0 licenses can continue to use their licenses with this patch level release. Customers who have purchased mxODBC Connect 1.x licenses can request 20% discount coupons for upgrade purchases. Please contact the eGenix.com Sales Team (sales at egenix.com) with your existing license serials for details. Users of our stand-alone mxODBC product will have to purchase new licenses from our online shop in order to use mxODBC Connect. You can request 30-day evaluation licenses by visiting our web-site or writing to sales at egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. http://www.egenix.com/products/python/mxODBCConnect/#Evaluation ________________________________________________________________________ DOWNLOADS The download archives as well as instructions for installation and configuration of the product can be found on the product page: http://www.egenix.com/products/python/mxODBCConnect/ If you want to try the package, jump straight to the download instructions: https://cms.egenix.com/products/python/mxODBCConnect/#Download Fully functional evaluation licenses for the mxODBC Connect Server are available free of charge: http://www.egenix.com/products/python/mxODBCConnect/#Evaluation mxODBC Connect Client is always free of charge. _______________________________________________________________________ SUPPORT Commercial support for this product is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. _______________________________________________________________________ INFORMATION About Python (http://www.python.org/): Python is an object-oriented Open Source programming language which runs on all modern platforms. By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for today's IT challenges. About eGenix (http://www.egenix.com/): eGenix is a software project, consulting and product company focusing on expert project services and professional quality products for companies, Python users and developers. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 09 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-02: Released mxODBC Zope DA 2.1.1 ... http://egenix.com/go41 2013-04-10: Python Meeting Duesseldorf ... tomorrow ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From dlenski at gmail.com Fri Apr 19 19:56:43 2013 From: dlenski at gmail.com (Dan Lenski) Date: Fri, 19 Apr 2013 17:56:43 +0000 (UTC) Subject: [DB-SIG] checking column types from cursor object in a database-independent way? Message-ID: Hi, I have to dump some data from various databases (Oracle and Postgre, at the moment) into CSV files, while preserving the column type information so the data can be correctly loaded into another application. Basically, I need to write a function that can distinguish string/numeric data in the columnar data from a cursor object in a fashion that's independent of the particular DBAPI module in use. According to PEP-0249, each DBAPI module has to define type objects (e.g. module.STRING) which "must compare equal to" the type objects in the cursor object for each column. The problem I'm having is... how can I figure out what are the appropriate type objects for this comparison if I *only* have access to the cursor object? I've been using an ugly kludge so far, to get a handle on the right DBAPI module: mod = sys.modules[cur.connection.__class__.__module__] #FIXME! if cur.description[0][1] == mod.STRING: print "first column is STRING data type" Is there a more robust and elegant solution to this? It seems like there ought to be a module-independent way to introspect the column data types in the cursor object. Thanks, Dan Lenski From vernondcole at gmail.com Fri Apr 19 20:31:33 2013 From: vernondcole at gmail.com (Vernon D. Cole) Date: Fri, 19 Apr 2013 12:31:33 -0600 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: Message-ID: Give the restriction you mention (that you have only the cursor object) the solution you propose looks pretty good. The cursor.connection attribute is an extension, but I think most api modules provide it. On Fri, Apr 19, 2013 at 11:56 AM, Dan Lenski wrote: > Hi, > I have to dump some data from various databases (Oracle and Postgre, at the > moment) into CSV files, while preserving the column type information so the > data can be correctly loaded into another application. Basically, I need > to > write a function that can distinguish string/numeric data in the columnar > data > from a cursor object in a fashion that's independent of the particular > DBAPI > module in use. > > According to PEP-0249, each DBAPI module has to define type objects (e.g. > module.STRING) which "must compare equal to" the type objects in the cursor > object for each column. > > The problem I'm having is... how can I figure out what are the appropriate > type objects for this comparison if I *only* have access to the cursor > object? > I've been using an ugly kludge so far, to get a handle on the right DBAPI > module: > > mod = sys.modules[cur.connection.__class__.__module__] #FIXME! > if cur.description[0][1] == mod.STRING: > print "first column is STRING data type" > > Is there a more robust and elegant solution to this? It seems like there > ought to be a module-independent way to introspect the column data types in > the cursor object. > > Thanks, > Dan Lenski > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike_mp at zzzcomputing.com Fri Apr 19 20:46:06 2013 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Fri, 19 Apr 2013 14:46:06 -0400 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: Message-ID: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> On Apr 19, 2013, at 1:56 PM, Dan Lenski wrote: > Hi, > I have to dump some data from various databases (Oracle and Postgre, at the > moment) into CSV files, while preserving the column type information so the > data can be correctly loaded into another application. Basically, I need to > write a function that can distinguish string/numeric data in the columnar data > from a cursor object in a fashion that's independent of the particular DBAPI > module in use. > > According to PEP-0249, each DBAPI module has to define type objects (e.g. > module.STRING) which "must compare equal to" the type objects in the cursor > object for each column. > > The problem I'm having is... how can I figure out what are the appropriate > type objects for this comparison if I *only* have access to the cursor object? Basically I'd question that precondition. I'd look into how the system is architected such that only a cursor is passed around, and try to substitute it/augment it with some kind of contextual object; either an object that can refer back to the DBAPI types, or even the DBAPI module itself. You can't really assume that ".connection" is present, nor that the Connection class of the DBAPI is in the same module space as where the type objects are defined. I'd rework the system and refuse the temptation to guess. > I've been using an ugly kludge so far, to get a handle on the right DBAPI > module: > > mod = sys.modules[cur.connection.__class__.__module__] #FIXME! > if cur.description[0][1] == mod.STRING: > print "first column is STRING data type" > > Is there a more robust and elegant solution to this? It seems like there > ought to be a module-independent way to introspect the column data types in > the cursor object. > > Thanks, > Dan Lenski > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From mal at python.org Fri Apr 19 21:06:36 2013 From: mal at python.org (M.-A. Lemburg) Date: Fri, 19 Apr 2013 21:06:36 +0200 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: <517195BC.8010103@python.org> On 19.04.2013 20:46, Michael Bayer wrote: > > On Apr 19, 2013, at 1:56 PM, Dan Lenski wrote: > >> Hi, >> I have to dump some data from various databases (Oracle and Postgre, at the >> moment) into CSV files, while preserving the column type information so the >> data can be correctly loaded into another application. Basically, I need to >> write a function that can distinguish string/numeric data in the columnar data >> from a cursor object in a fashion that's independent of the particular DBAPI >> module in use. >> >> According to PEP-0249, each DBAPI module has to define type objects (e.g. >> module.STRING) which "must compare equal to" the type objects in the cursor >> object for each column. >> >> The problem I'm having is... how can I figure out what are the appropriate >> type objects for this comparison if I *only* have access to the cursor object? > > > Basically I'd question that precondition. I'd look into how the system is architected such that only a cursor is passed around, and try to substitute it/augment it with some kind of contextual object; either an object that can refer back to the DBAPI types, or even the DBAPI module itself. You can't really assume that ".connection" is present, nor that the Connection class of the DBAPI is in the same module space as where the type objects are defined. I'd rework the system and refuse the temptation to guess. Yep, I think that's the way to go: simply pass the module object with the needed symbols around with the connection and cursor object. Note that adding something like connection.module won't work, since it may cause circular references which could then lead to connections being closed later than expected or modules not getting properly cleaned up at interpreter finalization time. At best, would have connection.modulename reference the module name as registered in sys.modules. >> I've been using an ugly kludge so far, to get a handle on the right DBAPI >> module: >> >> mod = sys.modules[cur.connection.__class__.__module__] #FIXME! >> if cur.description[0][1] == mod.STRING: >> print "first column is STRING data type" >> >> Is there a more robust and elegant solution to this? It seems like there >> ought to be a module-independent way to introspect the column data types in >> the cursor object. >> >> Thanks, >> Dan Lenski >> >> _______________________________________________ >> DB-SIG maillist - DB-SIG at python.org >> http://mail.python.org/mailman/listinfo/db-sig > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -- Marc-Andre Lemburg Vice Chairman Python Software Foundation http://www.python.org/psf/ From dlenski at gmail.com Mon Apr 22 03:50:17 2013 From: dlenski at gmail.com (Dan Lenski) Date: Mon, 22 Apr 2013 01:50:17 +0000 (UTC) Subject: [DB-SIG] checking column types from cursor object in a database-independent way? References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: On Fri, Apr 19, 2013 at 11:46 AM, Michael Bayer wrote: > > On Apr 19, 2013, at 1:56 PM, Dan Lenski wrote: > >> The problem I'm having is... how can I figure out what are the appropriate >> type objects for this comparison if I *only* have access to the cursor object? > > > Basically I'd question that precondition. I'd look into how the system is architected such that only a cursor is passed around, and try to substitute it/augment it with some kind of contextual object; either an object that can refer back to the DBAPI types, or even the DBAPI module itself. You can't really assume that ".connection" is present, nor that the Connection class of the DBAPI is in the same module space as where the type objects are defined. I'd rework the system and refuse the temptation to guess. I agree that my solution is not a good one... just a quick hack which seems to work for the modules that I've tried so far. However, it seems to me that the precondition is a reasonable one: the cursor object provides a DB-independent means of obtaining the data from a query, so why not the metadata as well? Actually, the DBAPI is pretty good in that regard, because it does establish the different type objects which a module must provide... but doesn't give a clear way to *find* those type objects from the cursor object. This makes it difficult to write truly DB- independent code to manipulate cursor objects. I *could* pass around a handle to the DB module along with the cursor itself, as you've suggested, but that seems redundant and error-prone to me. To my mind, this is a small gap in the DBAPI design: (1) DBAPI does specify a set of module-dependent type objects against which column type objects are to be compared (2) DBAPI does specify a cursor object which will produce column type objects after a query is executed (3) DBAPI *doesn't* provide any way to get from the cursor object to the module-defined type objects, at least not without passing a around module- dependent object. From vernondcole at gmail.com Mon Apr 22 11:14:34 2013 From: vernondcole at gmail.com (Vernon D. Cole) Date: Mon, 22 Apr 2013 03:14:34 -0600 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: Dan: You have a very good point. Perhaps we should put that on the wish list for API version 3. The .connection attribute of the cursor is optional. IMHO it should be required.. I think most existing api packages implement it. The it is suggested, but not required by the PEP, that the exception classes "should be exposed on the Connectionobjects as attributes (in addition to being available at module scope)." Adding them to the class definition for Connection is not difficult, and is quite convenient to use. I don't like the extra symbols in my debugger, but I can live with it. It would be trivial to include the type objects and constructors there, too. Again, perhaps this should be required. -- Vernon On Sun, Apr 21, 2013 at 7:50 PM, Dan Lenski wrote: > On Fri, Apr 19, 2013 at 11:46 AM, Michael Bayer > wrote: > > > > On Apr 19, 2013, at 1:56 PM, Dan Lenski wrote: > > > >> The problem I'm having is... how can I figure out what are the > appropriate > >> type objects for this comparison if I *only* have access to the cursor > object? > > > > > > Basically I'd question that precondition. I'd look into how the system > is architected such that only a cursor is passed around, and try to > substitute it/augment it with some kind of contextual object; either an > object that can refer back to the DBAPI types, or even the DBAPI module > itself. You can't really assume that ".connection" is present, nor that > the Connection class of the DBAPI is in the same module space as where the > type objects are defined. I'd rework the system and refuse the temptation > to guess. > > I agree that my solution is not a good one... just a quick hack which seems > to work for the modules that I've tried so far. > > However, it seems to me that the precondition is a reasonable one: the > cursor object provides a DB-independent means of obtaining the data from a > query, so why not the metadata as well? Actually, the DBAPI is pretty good > in that regard, because it does establish the different type objects which > a > module must provide... but doesn't give a clear way to *find* those type > objects from the cursor object. This makes it difficult to write truly DB- > independent code to manipulate cursor objects. > > I *could* pass around a handle to the DB module along with the cursor > itself, as you've suggested, but that seems redundant and error-prone to > me. > To my mind, this is a small gap in the DBAPI design: > (1) DBAPI does specify a set of module-dependent type objects against > which column type objects are to be compared > (2) DBAPI does specify a cursor object which will produce column type > objects after a query is executed > (3) DBAPI *doesn't* provide any way to get from the cursor object to the > module-defined type objects, at least not without passing a around module- > dependent object. > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dlenski at gmail.com Mon Apr 22 03:39:01 2013 From: dlenski at gmail.com (Daniel Lenski) Date: Sun, 21 Apr 2013 18:39:01 -0700 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: On Fri, Apr 19, 2013 at 11:46 AM, Michael Bayer wrote: > > On Apr 19, 2013, at 1:56 PM, Dan Lenski wrote: > >> The problem I'm having is... how can I figure out what are the appropriate >> type objects for this comparison if I *only* have access to the cursor object? > > > Basically I'd question that precondition. I'd look into how the system is architected such that only a cursor is passed around, and try to substitute it/augment it with some kind of contextual object; either an object that can refer back to the DBAPI types, or even the DBAPI module itself. You can't really assume that ".connection" is present, nor that the Connection class of the DBAPI is in the same module space as where the type objects are defined. I'd rework the system and refuse the temptation to guess. I agree that my solution is not a good one... just a quick hack which seems to work for the modules that I've tried so far. However, it seems to me that the precondition is a reasonable one: the cursor object provides a DB-independent means of obtaining the data from a query, so why not the metadata as well? Actually, the DBAPI is pretty good in that regard, because it does establish the different type objects which a module must provide... but doesn't give a clear way to *find* those type objects from the cursor object. This makes it difficult to write truly DB-independent code to manipulate cursor objects. I *could* pass around a handle to the DB module along with the cursor itself, as you've suggested, but that seems redundant and error-prone to me. To my mind, this is a small gap in the DBAPI design: (1) DBAPI does specify a set of module-dependent type objects against which column type objects are to be compared (2) DBAPI does specify a cursor object which will produce column type objects after a query is executed (3) DBAPI *doesn't* provide any way to get from the cursor object to the module-defined type objects, at least not without passing around module-dependent object. Dan From mike_mp at zzzcomputing.com Mon Apr 22 16:24:54 2013 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 22 Apr 2013 10:24:54 -0400 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> On Apr 21, 2013, at 9:39 PM, Daniel Lenski wrote: > > I *could* pass around a handle to the DB module along with the cursor > itself, as you've suggested, but that seems redundant and error-prone > to me. To my mind, this is a small gap in the DBAPI design: > (1) DBAPI does specify a set of module-dependent type objects > against which column type objects are to be compared > (2) DBAPI does specify a cursor object which will produce column > type objects after a query is executed > (3) DBAPI *doesn't* provide any way to get from the cursor object to > the module-defined type objects, at least not without passing around > module-dependent object. IMHO, the DBAPI is not meant to be used as a direct API within higher level application code; it only aims to provide a consistent low-level API to a wide variety of databases. It should always be framed within some kind of abstraction layer within real-world application. Therefore it should not concern/complicate itself worrying about convenience accessors, this only makes it more difficult for DBAPI implementors, leads to greater inconsistency between implementations, and makes it harder to test for compliance. From dlenski at gmail.com Mon Apr 22 18:40:09 2013 From: dlenski at gmail.com (Dan Lenski) Date: Mon, 22 Apr 2013 16:40:09 +0000 (UTC) Subject: [DB-SIG] checking column types from cursor object in a database-independent way? References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> Message-ID: Vernon D. Cole gmail.com> writes: > Dan: > You have a very good point.? Perhaps we should put that on the wish list for API version 3. > The .connection attribute of the cursor is optional.? IMHO it should be required..? I think most existing api packages implement it. > The it is suggested, but not required by the PEP, that the exception classes "should be > exposed on the Connection objects as attributes (in addition to > being available at module scope)." Adding them to the class definition for Connection is not difficult, and is quite convenient to use.? I don't like the extra symbols in my debugger, but I can live with it.? It would be trivial to include the type objects and constructors there, too.? Again, perhaps this should be required. Great, I do think it'd be a useful feature for v3! I don't know enough about the design goals of DBAPI to say whether the connection object is the *right* place to expose the column type objects, but it does seem like that would be a straightforward solution to the problem, as long as .connection is guaranteed to exist. if cur.description[n][1] == cur.connection.STRING: pass From dlenski at gmail.com Mon Apr 22 18:44:13 2013 From: dlenski at gmail.com (Dan Lenski) Date: Mon, 22 Apr 2013 16:44:13 +0000 (UTC) Subject: [DB-SIG] =?utf-8?q?checking_column_types_from_cursor_object_in_a?= =?utf-8?q?=09database-independent_way=3F?= References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> Message-ID: Michael Bayer zzzcomputing.com> writes: > On Apr 21, 2013, at 9:39 PM, Daniel Lenski gmail.com> wrote: > > > > > I *could* pass around a handle to the DB module along with the cursor > > itself, as you've suggested, but that seems redundant and error-prone > > to me. To my mind, this is a small gap in the DBAPI design: > > (1) DBAPI does specify a set of module-dependent type objects > > against which column type objects are to be compared > > (2) DBAPI does specify a cursor object which will produce column > > type objects after a query is executed > > (3) DBAPI *doesn't* provide any way to get from the cursor object to > > the module-defined type objects, at least not without passing around > > module-dependent object. > > IMHO, the DBAPI is not meant to be used as a direct API within higher level application code; it only aims to > provide a consistent low-level API to a wide variety of databases. It should always be framed within some > kind of abstraction layer within real-world application. Therefore it should not concern/complicate > itself worrying about convenience accessors, this only makes it more difficult for DBAPI implementors, > leads to greater inconsistency between implementations, and makes it harder to test for compliance. > I get your point about not crufting up DBAPI with a bunch of high-level features that will need to be reimplemented for each module... But this seems to me precisely the kind of feature that *should* exist at this level, because it makes it easier for higher-level interfaces to manipulate the underlying database objects in a generic way without carrying around extra module-dependent state on their own. From mike_mp at zzzcomputing.com Mon Apr 22 18:59:31 2013 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 22 Apr 2013 12:59:31 -0400 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> Message-ID: <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> On Apr 22, 2013, at 12:44 PM, Dan Lenski wrote: > Michael Bayer zzzcomputing.com> writes: > >> On Apr 21, 2013, at 9:39 PM, Daniel Lenski gmail.com> wrote: >> >>> >>> I *could* pass around a handle to the DB module along with the cursor >>> itself, as you've suggested, but that seems redundant and error-prone >>> to me. To my mind, this is a small gap in the DBAPI design: >>> (1) DBAPI does specify a set of module-dependent type objects >>> against which column type objects are to be compared >>> (2) DBAPI does specify a cursor object which will produce column >>> type objects after a query is executed >>> (3) DBAPI *doesn't* provide any way to get from the cursor object to >>> the module-defined type objects, at least not without passing around >>> module-dependent object. >> >> IMHO, the DBAPI is not meant to be used as a direct API within higher > level application code; it only aims to >> provide a consistent low-level API to a wide variety of databases. It > should always be framed within some >> kind of abstraction layer within real-world application. Therefore it > should not concern/complicate >> itself worrying about convenience accessors, this only makes it more > difficult for DBAPI implementors, >> leads to greater inconsistency between implementations, and makes it > harder to test for compliance. >> > > I get your point about not crufting up DBAPI with a bunch of high-level > features that will need to be reimplemented for each module... > > But this seems to me precisely the kind of feature that *should* exist at > this level, because it makes it easier for higher-level interfaces to > manipulate the underlying database objects in a generic way without carrying > around extra module-dependent state on their own. well I will say that there is precedent for this specific request - the cursor.connection attribute is part of the spec, and is pretty harmless, and the spec also includes the option for the DBAPI exception classes to be attached onto the Connection, which you can see here: http://www.python.org/dev/peps/pep-0249/#optional-db-api-extensions. If we're sticking the module-level exception classes directly onto the connection (which I find kind of distasteful, but there it is), might as well put *all* module-level constants onto it. > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From dlenski at gmail.com Tue Apr 23 00:24:16 2013 From: dlenski at gmail.com (Dan Lenski) Date: Mon, 22 Apr 2013 22:24:16 +0000 (UTC) Subject: [DB-SIG] =?utf-8?q?checking_column_types_from_cursor_object_in=09?= =?utf-8?q?a=09database-independent_way=3F?= References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> Message-ID: Michael Bayer zzzcomputing.com> writes: > On Apr 22, 2013, at 12:44 PM, Dan Lenski gmail.com> wrote: > > > I get your point about not crufting up DBAPI with a bunch of high-level > > features that will need to be reimplemented for each module... > > > > But this seems to me precisely the kind of feature that *should* exist at > > this level, because it makes it easier for higher-level interfaces to > > manipulate the underlying database objects in a generic way without carrying > > around extra module-dependent state on their own. > > well I will say that there is precedent for this specific request - the cursor.connection attribute is part > of the spec, and is pretty harmless, and the spec also includes the option for the DBAPI exception classes > to be attached onto the Connection, which you can see here: > http://www.python.org/dev/peps/pep-0249/#optional-db-api-extensions. If we're sticking the > module-level exception classes directly onto the connection (which I find kind of distasteful, but > there it is), might as well put *all* module-level constants onto it. Ah yes, that is quite a similar case. Since DBAPI specifies all these constants at the module-level, why not just include connection.module and cursor.module attributes to make them available? From vernondcole at gmail.com Tue Apr 23 00:51:00 2013 From: vernondcole at gmail.com (Vernon D. Cole) Date: Mon, 22 Apr 2013 16:51:00 -0600 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> Message-ID: One of those attributes needs to move anyway. In order to make the paramstyle switchable, it needs to be an attribute of the connection, so that a client who is supporting multiple connections to multiple engines (like, for example the database connection server I am writing right now) does not have trouble with the different streams stepping on each other. Most of the cruft is already hanging on the connection, and taking it out would break too much old code -- so we might as well put to remaining cruft in. On Mon, Apr 22, 2013 at 4:24 PM, Dan Lenski wrote: > Michael Bayer zzzcomputing.com> writes: > > On Apr 22, 2013, at 12:44 PM, Dan Lenski gmail.com> wrote: > > > > > I get your point about not crufting up DBAPI with a bunch of high-level > > > features that will need to be reimplemented for each module... > > > > > > But this seems to me precisely the kind of feature that *should* exist > at > > > this level, because it makes it easier for higher-level interfaces to > > > manipulate the underlying database objects in a generic way without > carrying > > > around extra module-dependent state on their own. > > > > well I will say that there is precedent for this specific request - the > cursor.connection attribute is part > > of the spec, and is pretty harmless, and the spec also includes the > option > for the DBAPI exception classes > > to be attached onto the Connection, which you can see here: > > http://www.python.org/dev/peps/pep-0249/#optional-db-api-extensions. If > we're sticking the > > module-level exception classes directly onto the connection (which I find > kind of distasteful, but > > there it is), might as well put *all* module-level constants onto it. > > Ah yes, that is quite a similar case. > > Since DBAPI specifies all these constants at the module-level, why not just > include connection.module and cursor.module attributes to make them > available? > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Tue Apr 23 09:29:53 2013 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 23 Apr 2013 09:29:53 +0200 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> Message-ID: <51763871.1040705@egenix.com> On 22.04.2013 18:59, Michael Bayer wrote: > > On Apr 22, 2013, at 12:44 PM, Dan Lenski wrote: > >> Michael Bayer zzzcomputing.com> writes: >> >>> On Apr 21, 2013, at 9:39 PM, Daniel Lenski gmail.com> wrote: >>> >>>> >>>> I *could* pass around a handle to the DB module along with the cursor >>>> itself, as you've suggested, but that seems redundant and error-prone >>>> to me. To my mind, this is a small gap in the DBAPI design: >>>> (1) DBAPI does specify a set of module-dependent type objects >>>> against which column type objects are to be compared >>>> (2) DBAPI does specify a cursor object which will produce column >>>> type objects after a query is executed >>>> (3) DBAPI *doesn't* provide any way to get from the cursor object to >>>> the module-defined type objects, at least not without passing around >>>> module-dependent object. >>> >>> IMHO, the DBAPI is not meant to be used as a direct API within higher >> level application code; it only aims to >>> provide a consistent low-level API to a wide variety of databases. It >> should always be framed within some >>> kind of abstraction layer within real-world application. Therefore it >> should not concern/complicate >>> itself worrying about convenience accessors, this only makes it more >> difficult for DBAPI implementors, >>> leads to greater inconsistency between implementations, and makes it >> harder to test for compliance. >>> >> >> I get your point about not crufting up DBAPI with a bunch of high-level >> features that will need to be reimplemented for each module... >> >> But this seems to me precisely the kind of feature that *should* exist at >> this level, because it makes it easier for higher-level interfaces to >> manipulate the underlying database objects in a generic way without carrying >> around extra module-dependent state on their own. > > > well I will say that there is precedent for this specific request - the cursor.connection attribute is part of the spec, and is pretty harmless, and the spec also includes the option for the DBAPI exception classes to be attached onto the Connection, which you can see here: http://www.python.org/dev/peps/pep-0249/#optional-db-api-extensions. If we're sticking the module-level exception classes directly onto the connection (which I find kind of distasteful, but there it is), might as well put *all* module-level constants onto it. I don't think we should clutter up the connection objects with module scope attributes that hardly ever get used. The exceptions are used a lot, so it makes sense to have them easily available via the connection object - even though I'm not sure whether that particular DB-API extension was such a good idea w/r to API design (it's one of those practicality beats purity things). Adding access to the database module via the connection object would allow to resolve all this. The problem with doing so is that you typically don't want the module object to be referenced directly by hundreds of objects in your application and you can also run into problems when reloading modules or (depending on how this is implemented) circular references. A method doing the lookup via the sys.modules dictionary could resolve those issues: database = connection.database() try: cursor = connection.cursor() cursor.execute(...) except database.DataError: ... Thoughts ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 23 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From vernondcole at gmail.com Tue Apr 23 10:10:38 2013 From: vernondcole at gmail.com (Vernon D. Cole) Date: Tue, 23 Apr 2013 02:10:38 -0600 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <51763871.1040705@egenix.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> <51763871.1040705@egenix.com> Message-ID: Yes, I _like_ that idea. I'm not so sure about "database" being the correct name for the attribute, but the idea itself is the correct "obvious way to do it", I think. I have just finished tearing all of the cruft stuff (exceptions, translations, etc) out of adodbapi into a separate module so that it would be re-useable. It also makes the package easier to understand. Now my test code contains a lot of... import adodbapi import adodbapi.apibase as api ... try: cursor = connection.cursor() cursor.execute(...) except api.DataError: Which, excluding the awful extra import statement, looks a lot like what you just suggested. Thinking while I type... if I were to package all of that as ... say perhaps a class that no one ever makes an instance of ... I could get exactly what you propose in short order. +1 the idea. -- Vernon On Tue, Apr 23, 2013 at 1:29 AM, M.-A. Lemburg wrote: > On 22.04.2013 18:59, Michael Bayer wrote: > > > > On Apr 22, 2013, at 12:44 PM, Dan Lenski wrote: > > > >> Michael Bayer zzzcomputing.com> writes: > >> > >>> On Apr 21, 2013, at 9:39 PM, Daniel Lenski gmail.com> > wrote: > >>> > >>>> > >>>> I *could* pass around a handle to the DB module along with the cursor > >>>> itself, as you've suggested, but that seems redundant and error-prone > >>>> to me. To my mind, this is a small gap in the DBAPI design: > >>>> (1) DBAPI does specify a set of module-dependent type objects > >>>> against which column type objects are to be compared > >>>> (2) DBAPI does specify a cursor object which will produce column > >>>> type objects after a query is executed > >>>> (3) DBAPI *doesn't* provide any way to get from the cursor object to > >>>> the module-defined type objects, at least not without passing around > >>>> module-dependent object. > >>> > >>> IMHO, the DBAPI is not meant to be used as a direct API within higher > >> level application code; it only aims to > >>> provide a consistent low-level API to a wide variety of databases. It > >> should always be framed within some > >>> kind of abstraction layer within real-world application. Therefore it > >> should not concern/complicate > >>> itself worrying about convenience accessors, this only makes it more > >> difficult for DBAPI implementors, > >>> leads to greater inconsistency between implementations, and makes it > >> harder to test for compliance. > >>> > >> > >> I get your point about not crufting up DBAPI with a bunch of high-level > >> features that will need to be reimplemented for each module... > >> > >> But this seems to me precisely the kind of feature that *should* exist > at > >> this level, because it makes it easier for higher-level interfaces to > >> manipulate the underlying database objects in a generic way without > carrying > >> around extra module-dependent state on their own. > > > > > > well I will say that there is precedent for this specific request - the > cursor.connection attribute is part of the spec, and is pretty harmless, > and the spec also includes the option for the DBAPI exception classes to be > attached onto the Connection, which you can see here: > http://www.python.org/dev/peps/pep-0249/#optional-db-api-extensions. If > we're sticking the module-level exception classes directly onto the > connection (which I find kind of distasteful, but there it is), might as > well put *all* module-level constants onto it. > > I don't think we should clutter up the connection objects with > module scope attributes that hardly ever get used. > > The exceptions are used a lot, so it makes sense to have them > easily available via the connection object - even though I'm not > sure whether that particular DB-API extension was such a > good idea w/r to API design (it's one of those practicality beats > purity things). > > Adding access to the database module via the connection object > would allow to resolve all this. The problem > with doing so is that you typically don't want the module > object to be referenced directly by hundreds of objects in your > application and you can also run into problems when reloading > modules or (depending on how this is implemented) circular > references. > > A method doing the lookup via the sys.modules dictionary > could resolve those issues: > > database = connection.database() > try: > cursor = connection.cursor() > cursor.execute(...) > except database.DataError: > ... > > Thoughts ? > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Apr 23 2013) > >>> Python Projects, Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope/Plone.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > 2013-04-17: Released eGenix mx Base 3.2.6 ... http://egenix.com/go43 > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike_mp at zzzcomputing.com Tue Apr 23 16:41:00 2013 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Tue, 23 Apr 2013 10:41:00 -0400 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <51763871.1040705@egenix.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> <51763871.1040705@egenix.com> Message-ID: <0DF2B047-0ACF-4286-A00D-7D8BF42D824D@zzzcomputing.com> On Apr 23, 2013, at 3:29 AM, M.-A. Lemburg wrote: > > > A method doing the lookup via the sys.modules dictionary > could resolve those issues: > > database = connection.database() > try: > cursor = connection.cursor() > cursor.execute(...) > except database.DataError: > ... > > Thoughts ? we probably want to call it "module" or "namespace" or "api" or something like that. The method itself can be a Python @property which would cause it to behave like an attribute in any case, or if we want to keep it as a method call to make it clear that code is being executed, that's fine as well. But the spec would need to say in either case, "don't assign the module itself to the connection". From dlenski at gmail.com Tue Apr 23 17:00:18 2013 From: dlenski at gmail.com (Daniel Lenski) Date: Tue, 23 Apr 2013 08:00:18 -0700 Subject: [DB-SIG] checking column types from cursor object in a database-independent way? In-Reply-To: <51763871.1040705@egenix.com> References: <337E5B06-72F0-47A0-ACCA-BB5AB4256798@zzzcomputing.com> <2AD29189-745E-4127-91BD-85775D980E73@zzzcomputing.com> <09CFDDF1-B874-447E-A4CA-1B07F1179CF0@zzzcomputing.com> <51763871.1040705@egenix.com> Message-ID: On Tue, Apr 23, 2013 at 12:29 AM, M.-A. Lemburg wrote: > > I don't think we should clutter up the connection objects with > module scope attributes that hardly ever get used. > > The exceptions are used a lot, so it makes sense to have them > easily available via the connection object - even though I'm not > sure whether that particular DB-API extension was such a > good idea w/r to API design (it's one of those practicality beats > purity things). Again, I understand the desire not to cruft up the cursor or connection namespaces, but I am sort of surprised that you consider the type objects to be little-used. It seems to me that they are a necessity for any abstract DB-independent layer. > Adding access to the database module via the connection object > would allow to resolve all this. The problem > with doing so is that you typically don't want the module > object to be referenced directly by hundreds of objects in your > application and you can also run into problems when reloading > modules or (depending on how this is implemented) circular > references. > > A method doing the lookup via the sys.modules dictionary > could resolve those issues: > > database = connection.database() > try: > cursor = connection.cursor() > cursor.execute(...) > except database.DataError: > ... > > Thoughts ? This seems like a reasonable solution to me, that would solve both this problem as well as similar ones. Alternatively, a cursor.coltypes attribute (in term containing .STRING, .DATETIME, etc.) would solve the present problem. Dan From vernondcole at gmail.com Sun Apr 28 23:38:09 2013 From: vernondcole at gmail.com (Vernon D. Cole) Date: Sun, 28 Apr 2013 22:38:09 +0100 Subject: [DB-SIG] Announcing Beta test of adodbapi v2.5 with Linux remote module Message-ID: Beta test is now available for adodbapi version 2.5 -- !!! NOW WORKS FROM LINUX !!! (as a remote client) -- ADO is Microsoft's update for ODBC, and can be used to open almost any tabular data source in existance. I have personally used adodbapi to read tables from: SQL Server, PostgreSQL, MySQL, Jet (a.k.a. ACCESS), IBM DB2, Active Directory, and Excel. Dozens of other sources are possible. Since ADO is proprietary, this requires the user to be running Windows. I have added a new "remote" module which will run on a Linux computer, and allow a Windows computer to act as a "proxy" to make these data sources available using a db-api version 2 interface on Linux. The remote client has almost the same interface as the local adodbapi -- except that custom error handlers and custom variant conversions are not supported from the remote. In order to make this work, I had to do major surgery on the way adodbapi processes connection strings. (The complexity of a connection string is the price you pay for the power of the ADO connection engine.) I am happy to say that you can now follow the PEP-249 suggestion using the USER, PASSWORD, HOST, and DATABASE keywords (along with any others you wish to define) as arguments to the .connect() method. There are also connection keyword macros. Since the remote computer may not be aware of whether the server is running 64 bit or 32 bit Python (which require different ADO "Providers") the remote may send a macro to the server, allowing the server to choose the correct one. There are also connection keywords for PARAMSTYLE, AUTOCOMMIT, and TIMEOUT, which allow you to override those defaults at connection time. There is a new method for the connection object: .get_table_names() which returns a list of all the table names in your database. This works pretty much everywhere, and will even list the worksheets in a remote Excel spreadsheet. The beta will be available in a source zip file from https://sourceforge.net/projects/adodbapi/files/adodbapi/2.5.0/ A rough draft of documentation is supplied as an .odt file. Please, several people try this and look it over. If all goes well, in a few weeks it will be part of django-mssql. -------------- next part -------------- An HTML attachment was scrubbed... URL: