From peter_e at gmx.net Wed Apr 11 20:35:54 2012 From: peter_e at gmx.net (Peter Eisentraut) Date: Wed, 11 Apr 2012 21:35:54 +0300 Subject: [DB-SIG] need some database API design advice Message-ID: <1334169354.25392.29.camel@vanquo.pezone.net> I'm looking for some API design advice. The PostgreSQL developers are currently considering an extension of the PL/Python database access API [0]. This is, for better or worse, a lower-level custom Python database access API, but you can build DB-API on top of that. We have added a few functions that extract metadata such as column names and types from a result set object (rv.colnames(), rv.coltypes(); think cursor.description). The question was what to do when there is no resulting row set, because the command was a utility command such as CREATE TABLE. Option 1 was that the functions should throw an exception, because the request is invalid. Option 2 was that the functions should return None. (This is what cursor.description is specified to do.) This was objected to because it would require extra checking for None. In addition to that, the question relative to option 1 in particular was how to detect whether a result row set exists, to avoid the exception-throwing calls. With option 2 you could check for is None, of course. There is an ongoing discussion [1] about which ones of these would be better style. So, in terms of code, which one of these is "better"? 1a. rv = plpy.execute("some SQL command") try: output(rv.colnames()) except SomeException: output("it's a utility command") 1b. rv = plpy.execute("some SQL command") if rv.has_rows(): # some currently nonexisting function to be added output(rv.colnames()) else: output("it's a utility command") 2. rv = plpy.execute("some SQL command") if rv.colnames() is not None: output(rv.colnames()) else: output("it's a utility command") [0] http://www.postgresql.org/docs/devel/static/plpython-database.html [1] http://archives.postgresql.org/message-id/CAK6bCay4yrFJD3po_bCke4ukjjsPLkbf+ad07jZiAU3N6cwUiA at mail.gmail.com From mal at egenix.com Wed Apr 11 21:47:18 2012 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 11 Apr 2012 21:47:18 +0200 Subject: [DB-SIG] need some database API design advice In-Reply-To: <1334169354.25392.29.camel@vanquo.pezone.net> References: <1334169354.25392.29.camel@vanquo.pezone.net> Message-ID: <4F85DFC6.9080803@egenix.com> Peter Eisentraut wrote: > I'm looking for some API design advice. > > The PostgreSQL developers are currently considering an extension of the > PL/Python database access API [0]. This is, for better or worse, a > lower-level custom Python database access API, but you can build DB-API > on top of that. > > We have added a few functions that extract metadata such as column names > and types from a result set object (rv.colnames(), rv.coltypes(); think > cursor.description). The question was what to do when there is no > resulting row set, because the command was a utility command such as > CREATE TABLE. Option 1 was that the functions should throw an > exception, because the request is invalid. Option 2 was that the > functions should return None. (This is what cursor.description is > specified to do.) This was objected to because it would require extra > checking for None. In addition to that, the question relative to option > 1 in particular was how to detect whether a result row set exists, to > avoid the exception-throwing calls. With option 2 you could check for > is None, of course. There is an ongoing discussion [1] about which ones > of these would be better style. > > So, in terms of code, which one of these is "better"? > > 1a. > > rv = plpy.execute("some SQL command") > try: > output(rv.colnames()) > except SomeException: > output("it's a utility command") > > 1b. > > rv = plpy.execute("some SQL command") > if rv.has_rows(): # some currently nonexisting function to be added > output(rv.colnames()) > else: > output("it's a utility command") > > 2. > > rv = plpy.execute("some SQL command") > if rv.colnames() is not None: > output(rv.colnames()) > else: > output("it's a utility command") When writing code, you typically know whether you are expecting a result set or not, so you don't run into such problems often. That said, calling a function is different than accessing an attribute. You normally expect a function to raise an exception in case it cannot process your request (due to a missing result set). The .fetch...() APIs also raise an exception if no result is present. Attribute access does not tend to generate exceptions if the values cannot be set. They simply revert to a default value (which is None in case of cursor.description). > [0] http://www.postgresql.org/docs/devel/static/plpython-database.html > [1] http://archives.postgresql.org/message-id/CAK6bCay4yrFJD3po_bCke4ukjjsPLkbf+ad07jZiAU3N6cwUiA at mail.gmail.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 11 2012) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2012-04-28: PythonCamp 2012, Cologne, Germany 17 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From Chris.Clark at actian.com Wed Apr 11 21:48:22 2012 From: Chris.Clark at actian.com (Chris Clark) Date: Wed, 11 Apr 2012 12:48:22 -0700 Subject: [DB-SIG] need some database API design advice In-Reply-To: <1334169354.25392.29.camel@vanquo.pezone.net> References: <1334169354.25392.29.camel@vanquo.pezone.net> Message-ID: <4F85E006.3010504@actian.com> On Wednesday 2012-04-11 12:44 (-0700), Peter Eisentraut wrote: > .... > So, in terms of code, which one of these is "better"? > > 1a. > > rv = plpy.execute("some SQL command") > try: > output(rv.colnames()) > except SomeException: > output("it's a utility command") > > 1b. > > rv = plpy.execute("some SQL command") > if rv.has_rows(): # some currently nonexisting function to be added > output(rv.colnames()) > else: > output("it's a utility command") > > 2. > > rv = plpy.execute("some SQL command") > if rv.colnames() is not None: > output(rv.colnames()) > else: > output("it's a utility command") My 2 cents, I don't like any of them :-p I prefer the last one modified slightly: rv = plpy.execute("some SQL command") if rv.colnames(): output(rv.colnames()) else: output("it's a utility command") I.e. loose the None check. I'm not sure I like making an explicit function call for the colnames. I.e. I think sticking with the pep249 description attribute is a better solution, the API already exist so clone that (when possible) in your new api. ....Of course a lot of this is is down to style preferences. Chris From peter_e at gmx.net Fri Apr 13 23:11:35 2012 From: peter_e at gmx.net (Peter Eisentraut) Date: Sat, 14 Apr 2012 00:11:35 +0300 Subject: [DB-SIG] need some database API design advice In-Reply-To: <4F85E006.3010504@actian.com> References: <1334169354.25392.29.camel@vanquo.pezone.net> <4F85E006.3010504@actian.com> Message-ID: <1334351495.9019.26.camel@vanquo.pezone.net> On ons, 2012-04-11 at 12:48 -0700, Chris Clark wrote: > > rv = plpy.execute("some SQL command") > > if rv.colnames() is not None: > > output(rv.colnames()) > > else: > > output("it's a utility command") > > My 2 cents, I don't like any of them :-p > > I prefer the last one modified slightly: > > rv = plpy.execute("some SQL command") > if rv.colnames(): > output(rv.colnames()) > else: > output("it's a utility command") > > I.e. loose the None check. That would misbehave if a result set exists but contains zero columns. (Sounds strange, but it's possible.) Which actually now makes me lean toward throwing an exception. > I'm not sure I like making an explicit > function call for the colnames. I.e. I think sticking with the pep249 > description attribute is a better solution, the API already exist so > clone that (when possible) in your new api. That would require populating this structure on every call, which would be expensive for such a low-level API, or turning the attribute into a fake function, which would be evil. I think this is best left to the plpydbapi layer on top of it. From mal at egenix.com Fri Apr 13 23:17:48 2012 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 13 Apr 2012 23:17:48 +0200 Subject: [DB-SIG] need some database API design advice In-Reply-To: <1334351495.9019.26.camel@vanquo.pezone.net> References: <1334169354.25392.29.camel@vanquo.pezone.net> <4F85E006.3010504@actian.com> <1334351495.9019.26.camel@vanquo.pezone.net> Message-ID: <4F8897FC.2020409@egenix.com> Peter Eisentraut wrote: >> I'm not sure I like making an explicit >> function call for the colnames. I.e. I think sticking with the pep249 >> description attribute is a better solution, the API already exist so >> clone that (when possible) in your new api. I agree with Chris Clark here... doing so saves you complexity in the dbapi layer you intend to write on top of the low level API. > That would require populating this structure on every call, which would > be expensive for such a low-level API, or turning the attribute into a > fake function, which would be evil. I think this is best left to the > plpydbapi layer on top of it. Using properties you can do this on the fly and only when needed. If you're writing the API in C, you can also create the .description tuple on demand and only when needed. You can also cache it in case you don't want to take the small extra hit of having to recreate it every time. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 13 2012) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2012-04-28: PythonCamp 2012, Cologne, Germany 15 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From Chris.Clark at actian.com Fri Apr 13 23:32:05 2012 From: Chris.Clark at actian.com (Chris Clark) Date: Fri, 13 Apr 2012 14:32:05 -0700 Subject: [DB-SIG] need some database API design advice In-Reply-To: <4F8897FC.2020409@egenix.com> References: <1334169354.25392.29.camel@vanquo.pezone.net> <4F85E006.3010504@actian.com> <1334351495.9019.26.camel@vanquo.pezone.net> <4F8897FC.2020409@egenix.com> Message-ID: <4F889B55.4070200@actian.com> On Friday 2012-04-13 14:25 (-0700), M.-A. Lemburg wrote: > Peter Eisentraut wrote: >>> I'm not sure I like making an explicit >>> function call for the colnames. I.e. I think sticking with the pep249 >>> description attribute is a better solution, the API already exist so >>> clone that (when possible) in your new api. > I agree with Chris Clark here... doing so saves you complexity in the > dbapi layer you intend to write on top of the low level API. > >> That would require populating this structure on every call, which would >> be expensive for such a low-level API, or turning the attribute into a >> fake function, which would be evil. I think this is best left to the >> plpydbapi layer on top of it. > Using properties you can do this on the fly and only when > needed. If you're writing the API in C, you can also create the > .description tuple on demand and only when needed. You can also > cache it in case you don't want to take the small extra hit > of having to recreate it every time. > That was what I had in mind. This is slightly off topic for C but it illustrates why an "if true" is usually considered more Pythonic than an explicit "is [not] None" check (for attributes/objects at any rate, not function call results). Consider: class MyClass(): def __nonzero__(self): return False x = MyClass() if x: print 'x is true' else: print 'x is False' the "if x" check is usually a inexpensive check. For more python magic methods, check out the excellent article http://www.rafekettler.com/magicmethods.html I think I confused the issue by leaving the function call in the example when my comment suggested not doing that. Chris From info at egenix.com Thu Apr 26 10:14:29 2012 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Thu, 26 Apr 2012 10:14:29 +0200 Subject: [DB-SIG] ANN: eGenix mxODBC - Python ODBC Database Interface 3.1.2 Message-ID: <4F9903E5.1060502@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC - Python ODBC Database Interface Version 3.1.2 mxODBC is our commercially supported Python extension providing ODBC database connectivity to Python applications on Windows, Mac OS X, Unix and BSD platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-3.1.2-GA.html ________________________________________________________________________ INTRODUCTION mxODBC provides an easy-to-use, high-performance, reliable and robust Python interface to ODBC compatible databases such as MS SQL Server, MS Access, Oracle Database, IBM DB2 and Informix , Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more: http://www.egenix.com/products/python/mxODBC/ The "eGenix mxODBC - Python ODBC Database Interface" product is a commercial extension to our open-source eGenix mx Base Distribution: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ NEWS mxODBC 3.1.2 Update ------------------- The 3.1.2 release of our mxODBC is the latest release of our popular Python ODBC Interface. The new patch-level version includes a few important fixes: * Fixed a compatibility problem with Python 2.7's distutils that was introduced in Python 2.7.3 * Improved compatibility of the mxODBC native Unicode string format handling with Unix ODBC drivers when running UCS4 builds of Python. Licenses for mxODBC 3.1.x remain valid for mxODBC 3.1.2 as well. Support for MS SQL Server Native Client for Linux ------------------------------------------------- The above changes allows using the NVARCHAR and NCHAR types of SQL Server with the *new MS SQL Server Native Client for Linux* with UCS4 Python builds. UCS2 Python builds don't exhibit this issue. The new driver has proven to be much more feature complete than the FreeTDS ODBC driver, so it's definitely worth a try. Note that the MS driver currently has an issue with Unicode connection strings which causes stack corruption. Please don't use Unicode connection strings with the MS SQL Server Native Client for Linux driver, since there's no way we can work around this problem: we only know the type of driver after connection and then it's already too late. You can download the new MS SQL Server Native Client for Linux from: http://www.microsoft.com/download/en/details.aspx?id=28160 Windows x64 and eGenix mx Base 3.2 ---------------------------------- With mxODBC 3.1.1 we had already introduced compatibility with our current mx Base 3.2 release in order to be able to support the Windows x64 platform. If you are currently using the combinations mxODBC 3.1.0 + mx Base 3.1, please consider upgrading to our latest releases mxODBC 3.1 + mx Base 3.2. eGenix mx Base 3.2 is available from our product page: http://www.egenix.com/products/python/mxBase/ mxODBC 3.1 Release Highlights ----------------------------- * We've added Python 2.7 support and builds for all platforms. * mxODBC 3.1 adds native support for the Windows 64-bit platforms as well as the Mac OS X 10.6 (Snow Leopard) 64-bit builds of Python. * mxODBC now fully supports the Oracle Instant Client ODBC driver. * We have updated the support for the latest IBM DB2 9.7 ODBC drivers and enhanced compatibility of mxODBC with the MS SQL Server Native Client ODBC driver on Windows and the Sybase ASE 15 ODBC drivers on Unix. * mxODBC 3.1 adds support for large-scale data warehouse databases Netezza and Teradata. * In addition to the Windows, Mac OS X, iODBC and unixODBC ODBC driver managers, we now also include support for the DataDirect ODBC manager. * The 64-bit support on Unix platforms was updated to support the new unixODBC 2.3.0 version. * We've improved the documentation on how to connect to various popular databases and now include many tips & tricks for each database/driver. * The Python 2.7 memoryview object is now supported as binary data container. * We have simplified handling of database warnings using a new customizable .warningformat attribute. * The catalog methods now accept both Unicode and 8-bit strings as parameters. * You can now select whether to use ANSI (8-bit) or Unicode ODBC APIs in the ODBC drivers, removing unnecessary data conversions and enhancing ODBC driver compatibility. For the full set of changes please check the mxODBC change log: http://www.egenix.com/products/python/mxODBC/changelog.html Feature Highlights ------------------ * Python Database API 2.0 Compliance: the mxODBC API is fully Python DB-API 2.0 compatible and implements a large number of powerful extensions. * Support for all popular ODBC Drivers: mxODBC includes adjustments and work-arounds to support MS SQL Server Native Client, MS SQL Server ODBC Driver, FreeTDS ODBC Driver, Oracle Instant Client ODBC Driver, IBM DB2 ODBC Driver, Sybase ASE ODBC Driver, Netezza ODBC Driver, Teradata ODBC Driver, PostgreSQL ODBC Driver, MySQL ODBC Driver, .MaxDB ODBC Driver as well as the ODBC driver sets of EasySoft, DataDirect, OpenLink, Actual Technologies. * Support for all popular ODBC Driver Managers: mxODBC comes with subpackages for the native Windows and Mac OS X ODBC managers, as well as the ODBC managers unixODBC, iODBC and DataDirect, which are commonly used on Unix systems. * Stable, robust and reliable:the mxODBC API has been in active production use for more than 10 years. * Full Python Support: mxODBC works with Python 2.4, 2.5, 2.6 and 2.7. * Full 64-bit Support: mxODBC runs on the following 64-bit platforms: Windows, Linux, FreeBSD and Mac OS X. For the full set of features mxODBC has to offer, please see: http://www.egenix.com/products/python/mxODBC/#Features New mxODBC Editions ------------------- Due to popular demand, we have extended the set of available mxODBC editions and included a new low-cost standard edition. mxODBC is now available in these three editions: * The low-cost Standard Edition which provides data connectivity to a selected set of database backends. * The Professional Edition, which gives full access to all mxODBC features. * The Product Development Edition, which allows including mxODBC in applications you develop. At the same time we have simplified our license terms to clarify the situation on multi-core and virtual machines. In most cases, you no longer need to purchase more than one license per processor or virtual machine, scaling down the overall license costs significantly compared to earlier mxODBC releases. For a complete overview of the new editions, please see the product page. http://www.egenix.com/products/python/mxODBC/#mxODBCEditions ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/mxODBC/ In order to use the eGenix mxODBC package you will first need to install the eGenix mx Base package: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ UPGRADING Users are encouraged to upgrade to this latest mxODBC release to benefit from the new features and updated ODBC driver support. We have taken special care, not to introduce backwards incompatible changes, making the upgrade experience as smooth as possible. Customers who have purchased mxODBC 3.1 license can continue to use their licenses with this patch level release. Customers who have purchased mxODBC 2.0, 2.1 or 3.0 licenses can upgrade their licenses using the mxODBC Professional Edition Upgrade License. If you want to try the new release before purchace, you can request 30-day evaluation licenses by visiting our web-site http://www.egenix.com/products/python/mxODBC/#Evaluation or by writing to sales at egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. _______________________________________________________________________ SUPPORT Commercial support for this product is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. _______________________________________________________________________ INFORMATION About Python (http://www.python.org/): Python is an object-oriented Open Source programming language which runs on all modern platforms. By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for today's IT challenges. About eGenix (http://www.egenix.com/): eGenix is a software project, consulting and product company focusing on expert services and professional quality products for companies, Python users and developers. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 26 2012) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2012-04-28: PythonCamp 2012, Cologne, Germany 2 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/