From wilk at flibuste.net Sun Feb 13 14:04:22 2011 From: wilk at flibuste.net (William Dode) Date: Sun, 13 Feb 2011 13:04:22 +0000 (UTC) Subject: [DB-SIG] mogrify/query for adodbapi Message-ID: Hi, With psycopg2 I use "mogrify" or the "query" attribute to retrieve the exact sql query. It's very usefull for debugging. Could it be possible to have the same with adodbapi ? I mean, is it possible with ado to retrieve the sql query ? How do you do with other databases ? bye thanks to read my bad english... -- William Dod? - http://flibuste.net Informaticien Ind?pendant From vernondcole at gmail.com Sun Feb 13 18:04:00 2011 From: vernondcole at gmail.com (Vernon Cole) Date: Sun, 13 Feb 2011 10:04:00 -0700 Subject: [DB-SIG] mogrify/query for adodbapi In-Reply-To: References: Message-ID: This sounds like a handy feature. Recent releases of adodbapi allow the programmer to specify the paramstyle she wishes to use. If she uses 'qmark' (the default) her query is used unchanged. If she uses 'named' or 'format', then her query is converted to 'qmark' before being used. The cursor stores an ADODB.Command structure as its .cmd attribute. The (converted) query text is stored in the .cmd.CommandText attribute. There is no storage of the original query, although it might be an optimization to keep it around to avoid re-parsing. Q) Should a reference to cursor.query return the original query text, or the reformatted version? Q) What is "mogrify"? Is that a reformatted version? -- Vernon On Sun, Feb 13, 2011 at 6:04 AM, William Dode wrote: > Hi, > > With psycopg2 I use "mogrify" or the "query" attribute to retrieve the > exact sql query. It's very usefull for debugging. > > Could it be possible to have the same with adodbapi ? I mean, is it > possible with ado to retrieve the sql query ? > > How do you do with other databases ? > > bye > > thanks to read my bad english... > > -- > William Dod? - http://flibuste.net > Informaticien Ind?pendant > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilk at flibuste.net Sun Feb 13 19:19:08 2011 From: wilk at flibuste.net (William Dode) Date: Sun, 13 Feb 2011 18:19:08 +0000 (UTC) Subject: [DB-SIG] mogrify/query for adodbapi References: Message-ID: On 13-02-2011, Vernon Cole wrote: > --===============0804799855== > Content-Type: multipart/alternative; boundary=0016e644df66158a5b049c2cec40 > > --0016e644df66158a5b049c2cec40 > Content-Type: text/plain; charset=ISO-8859-1 > Content-Transfer-Encoding: quoted-printable > > This sounds like a handy feature. Recent releases of adodbapi allow the > programmer to specify the paramstyle she wishes to use. If she uses 'qmark' > (the default) her query is used unchanged. If she uses 'named' or 'format', > then her query is converted to 'qmark' before being used. > The cursor stores an ADODB.Command structure as its .cmd attribute. The > (converted) query text is stored in the .cmd.CommandText attribute. There i= > s > no storage of the original query, although it might be an optimization to > keep it around to avoid re-parsing. > > Q) Should a reference to cursor.query return the original query text, or th= > e > reformatted version? Same as mogrify >>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >>> cur.query "INSERT INTO test (num, data) VALUES (42, E'bar')" > > Q) What is "mogrify"? Is that a reformatted version? mogrify(operation[, parameters])? Return a query string after arguments binding. The string returned is exactly the one that would be sent to the database running the execute() method or similar. >>> cur.mogrify("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) "INSERT INTO test (num, data) VALUES (42, E'bar')" I use to log it and some times use it to copy and paste. -- William Dod? http://flibuste.net Informaticien Ind?pendant From mal at egenix.com Mon Feb 14 11:49:33 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 14 Feb 2011 11:49:33 +0100 Subject: [DB-SIG] mogrify/query for adodbapi In-Reply-To: References: Message-ID: <4D5908BD.1080502@egenix.com> William Dode wrote: > On 13-02-2011, Vernon Cole wrote: >> --===============0804799855== >> Content-Type: multipart/alternative; boundary=0016e644df66158a5b049c2cec40 >> >> --0016e644df66158a5b049c2cec40 >> Content-Type: text/plain; charset=ISO-8859-1 >> Content-Transfer-Encoding: quoted-printable >> >> This sounds like a handy feature. Recent releases of adodbapi allow the >> programmer to specify the paramstyle she wishes to use. If she uses 'qmark' >> (the default) her query is used unchanged. If she uses 'named' or 'format', >> then her query is converted to 'qmark' before being used. >> The cursor stores an ADODB.Command structure as its .cmd attribute. The >> (converted) query text is stored in the .cmd.CommandText attribute. There i= >> s >> no storage of the original query, although it might be an optimization to >> keep it around to avoid re-parsing. >> >> Q) Should a reference to cursor.query return the original query text, or th= >> e >> reformatted version? > > Same as mogrify > >>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >>>> cur.query > "INSERT INTO test (num, data) VALUES (42, E'bar')" > > >> >> Q) What is "mogrify"? Is that a reformatted version? > > mogrify(operation[, parameters])? > > Return a query string after arguments binding. The string returned is exactly the one that would be sent to the database running the execute() method or similar. > > >>> cur.mogrify("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) > "INSERT INTO test (num, data) VALUES (42, E'bar')" > > I use to log it and some times use it to copy and paste. Since database adapater usually try to avoid merging the parameters into the query string before sending them off to the server and instead send the query and parameters separately (which is more efficient and also allows caching of access plans on the server side based on the query string), wouldn't it be better to log both the query string and a Python repr() of the parameters instead ? That's how we have do such logging in our code and it works nicely. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 14 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From wilk at flibuste.net Mon Feb 14 15:49:33 2011 From: wilk at flibuste.net (William Dode) Date: Mon, 14 Feb 2011 14:49:33 +0000 (UTC) Subject: [DB-SIG] mogrify/query for adodbapi References: <4D5908BD.1080502@egenix.com> Message-ID: On 14-02-2011, M.-A. Lemburg wrote: > William Dode wrote: >> On 13-02-2011, Vernon Cole wrote: >>> --===============0804799855== >>> Content-Type: multipart/alternative; boundary=0016e644df66158a5b049c2cec40 >>> >>> --0016e644df66158a5b049c2cec40 >>> Content-Type: text/plain; charset=ISO-8859-1 >>> Content-Transfer-Encoding: quoted-printable >>> >>> This sounds like a handy feature. Recent releases of adodbapi allow the >>> programmer to specify the paramstyle she wishes to use. If she uses 'qmark' >>> (the default) her query is used unchanged. If she uses 'named' or 'format', >>> then her query is converted to 'qmark' before being used. >>> The cursor stores an ADODB.Command structure as its .cmd attribute. The >>> (converted) query text is stored in the .cmd.CommandText attribute. There i= >>> s >>> no storage of the original query, although it might be an optimization to >>> keep it around to avoid re-parsing. >>> >>> Q) Should a reference to cursor.query return the original query text, or th= >>> e >>> reformatted version? >> >> Same as mogrify >> >>>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >>>>> cur.query >> "INSERT INTO test (num, data) VALUES (42, E'bar')" >> >> >>> >>> Q) What is "mogrify"? Is that a reformatted version? >> >> mogrify(operation[, parameters])? >> >> Return a query string after arguments binding. The string returned is exactly the one that would be sent to the database running the execute() method or similar. >> >> >>> cur.mogrify("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >> "INSERT INTO test (num, data) VALUES (42, E'bar')" >> >> I use to log it and some times use it to copy and paste. > > Since database adapater usually try to avoid merging the parameters > into the query string before sending them off to the server and > instead send the query and parameters separately (which is more > efficient and also allows caching of access plans on the server > side based on the query string), wouldn't it be better to log > both the query string and a Python repr() of the parameters > instead ? I would like to see the query after the binding. For example with msaccess "insert into t (a), (?)", (True,) will insert -1 with postgresql it will insert 1 Also i would like to can copy and paste the query for debugging. For example with msaccess, if a field is misswritted the error will not show wich field is it. If i copy-paste it on msaccess it will say me wich field is it. I thought maybe ado could show me this. If not i will do it by hand for debugging and of course use the params for the real query. > > That's how we have do such logging in our code and it works nicely. > -- William Dod? - http://flibuste.net Informaticien Ind?pendant From vernondcole at gmail.com Mon Feb 14 16:25:57 2011 From: vernondcole at gmail.com (Vernon Cole) Date: Mon, 14 Feb 2011 08:25:57 -0700 Subject: [DB-SIG] mogrify/query for adodbapi In-Reply-To: References: <4D5908BD.1080502@egenix.com> Message-ID: There may be some help for you. ADO will read and write ACCESS data tables, so you can migrate your more challenging data communications tasks to Python and eventually replace your ACCESS application with something a bit more trustworthy. Vernon Cole (sent from my 'droid phone) On Feb 14, 2011 7:49 AM, "William Dode" wrote: > On 14-02-2011, M.-A. Lemburg wrote: >> William Dode wrote: >>> On 13-02-2011, Vernon Cole wrote: >>>> --===============0804799855== >>>> Content-Type: multipart/alternative; boundary=0016e644df66158a5b049c2cec40 >>>> >>>> --0016e644df66158a5b049c2cec40 >>>> Content-Type: text/plain; charset=ISO-8859-1 >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> This sounds like a handy feature. Recent releases of adodbapi allow the >>>> programmer to specify the paramstyle she wishes to use. If she uses 'qmark' >>>> (the default) her query is used unchanged. If she uses 'named' or 'format', >>>> then her query is converted to 'qmark' before being used. >>>> The cursor stores an ADODB.Command structure as its .cmd attribute. The >>>> (converted) query text is stored in the .cmd.CommandText attribute. There i= >>>> s >>>> no storage of the original query, although it might be an optimization to >>>> keep it around to avoid re-parsing. >>>> >>>> Q) Should a reference to cursor.query return the original query text, or th= >>>> e >>>> reformatted version? >>> >>> Same as mogrify >>> >>>>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >>>>>> cur.query >>> "INSERT INTO test (num, data) VALUES (42, E'bar')" >>> >>> >>>> >>>> Q) What is "mogrify"? Is that a reformatted version? >>> >>> mogrify(operation[, parameters])? >>> >>> Return a query string after arguments binding. The string returned is exactly the one that would be sent to the database running the execute() method or similar. >>> >>> >>> cur.mogrify("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar')) >>> "INSERT INTO test (num, data) VALUES (42, E'bar')" >>> >>> I use to log it and some times use it to copy and paste. >> >> Since database adapater usually try to avoid merging the parameters >> into the query string before sending them off to the server and >> instead send the query and parameters separately (which is more >> efficient and also allows caching of access plans on the server >> side based on the query string), wouldn't it be better to log >> both the query string and a Python repr() of the parameters >> instead ? > > I would like to see the query after the binding. For example with > msaccess "insert into t (a), (?)", (True,) will insert -1 with > postgresql it will insert 1 > > Also i would like to can copy and paste the query for debugging. For > example with msaccess, if a field is misswritted the error will not show > wich field is it. If i copy-paste it on msaccess it will say me wich > field is it. > > I thought maybe ado could show me this. If not i will do it by hand for > debugging and of course use the params for the real query. > >> >> That's how we have do such logging in our code and it works nicely. >> > > > -- > William Dod? - http://flibuste.net > Informaticien Ind?pendant > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From fog at initd.org Mon Feb 14 16:56:38 2011 From: fog at initd.org (Federico Di Gregorio) Date: Mon, 14 Feb 2011 16:56:38 +0100 Subject: [DB-SIG] mogrify/query for adodbapi In-Reply-To: References: <4D5908BD.1080502@egenix.com> Message-ID: <4D5950B6.1040606@initd.org> [snip] On 14/02/11 15:49, William Dode wrote: > I would like to see the query after the binding. For example with > msaccess "insert into t (a), (?)", (True,) will insert -1 with > postgresql it will insert 1 > > Also i would like to can copy and paste the query for debugging. For > example with msaccess, if a field is misswritted the error will not show > wich field is it. If i copy-paste it on msaccess it will say me wich > field is it. > > I thought maybe ado could show me this. If not i will do it by hand for > debugging and of course use the params for the real query. If the python library sends the query+parameters you can't because it is the backend that will build the final query. Currently psycopg does all argument escaping and quoting on the client so we have the final query ready for inspection. Anyway, if the client library sends query+parameters to the backend it is possible to build a "fake" query for logging/debugging purpuse. For example, we plan to move psycopg .executemany() to prepared queries and even in that case .mogrify() will return the full query, but in that case it will be not the one we sent to the db. federico -- Federico Di Gregorio fog at initd.org There's no greys, only white that's got grubby. I'm surprised you don't know that. And sin, young man, is when you treat people as things. Including yourself. -- Granny Weatherwax -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From wilk at flibuste.net Tue Feb 15 14:00:11 2011 From: wilk at flibuste.net (William Dode) Date: Tue, 15 Feb 2011 13:00:11 +0000 (UTC) Subject: [DB-SIG] mogrify/query for adodbapi References: <4D5908BD.1080502@egenix.com> <4D5950B6.1040606@initd.org> Message-ID: On 14-02-2011, Federico Di Gregorio wrote: > This is an OpenPGP/MIME signed message (RFC 2440 and 3156) > --===============1638910873== > Content-Type: multipart/signed; micalg=pgp-sha1; > protocol="application/pgp-signature"; > boundary="------------enig8B948CD4F063A9F68C5802DF" > > This is an OpenPGP/MIME signed message (RFC 2440 and 3156) > --------------enig8B948CD4F063A9F68C5802DF > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: quoted-printable > > [snip] > On 14/02/11 15:49, William Dode wrote: >> I would like to see the query after the binding. For example with=20 >> msaccess "insert into t (a), (?)", (True,) will insert -1 with=20 >> postgresql it will insert 1 >>=20 >> Also i would like to can copy and paste the query for debugging. For=20 >> example with msaccess, if a field is misswritted the error will not sho= > w=20 >> wich field is it. If i copy-paste it on msaccess it will say me wich=20 >> field is it. >>=20 >> I thought maybe ado could show me this. If not i will do it by hand for= >=20 >> debugging and of course use the params for the real query. > > If the python library sends the query+parameters you can't because it is > the backend that will build the final query. Currently psycopg does all > argument escaping and quoting on the client so we have the final query > ready for inspection. > > Anyway, if the client library sends query+parameters to the backend it > is possible to build a "fake" query for logging/debugging purpuse. For > example, we plan to move psycopg .executemany() to prepared queries and > even in that case .mogrify() will return the full query, but in that > case it will be not the one we sent to the db. I understand, i will do myself a fake query for logging/debugging... -- William Dod? - http://flibuste.net Informaticien Ind?pendant From vernondcole at gmail.com Tue Feb 15 19:08:14 2011 From: vernondcole at gmail.com (Vernon Cole) Date: Tue, 15 Feb 2011 11:08:14 -0700 Subject: [DB-SIG] How does one determine which underlying database engine is in use? Message-ID: When one is using a generic database access tool, such as ADO or ODBC, it is possible that the programmer does not know in advance what database engine will eventually be used. Since different engines have vastly different requirements [1] it would be nice for the programmer to be able to determine which database engine his user has handed him. Is there any sort of standard or usual way that information may be obtained? (If egenix has an extension for this I will happily adopt it.) -- Vernon Cole [1] for example, do I use TOP or LIMIT to restrict the number of rows in a result set? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Tue Feb 15 19:18:11 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Feb 2011 19:18:11 +0100 Subject: [DB-SIG] How does one determine which underlying database engine is in use? In-Reply-To: References: Message-ID: <4D5AC363.1050909@egenix.com> Vernon Cole wrote: > When one is using a generic database access tool, such as ADO or ODBC, it is > possible that the programmer does not know in advance what database engine > will eventually be used. > > Since different engines have vastly different requirements [1] it would be > nice for the programmer to be able to determine which database engine his > user has handed him. > > Is there any sort of standard or usual way that information may be > obtained? (If egenix has an extension for this I will happily adopt it.) We use the connection attributes .dbms_name and .dbms_version for this in mxODBC and mxODBC Connect: http://www.egenix.com/products/python/mxODBC/doc/#_Toc269754563 > [1] for example, do I use TOP or LIMIT to restrict the number of rows in a > result set? The database engine version is usually more interesting than the engine name (which is typically fixed for an application). With the version you can easily determine whether advanced SQL features are available or not. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 15 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From vernondcole at gmail.com Wed Feb 16 08:29:43 2011 From: vernondcole at gmail.com (Vernon Cole) Date: Wed, 16 Feb 2011 00:29:43 -0700 Subject: [DB-SIG] announcing adodbapi version 2.4.2 with PostgreSQL support Message-ID: Recent communications on this list have encouraged me to get off my duff and finish the PostgreSQL support that I started ages ago. Thanks to William Dode and M.-A. Lemburg for the inspiration. Whats new in version 2.4.2 1. The cursor has a new .query attribute. It returns the (possibly converted) query sent to ADO. [Thanks to William Dode.] This may be useful for testing paramstyle 'format' and 'named' queries. .query is an extension borrowed from psycopg2. 2. Added .command and .parameters attributes, which are copies of the original command and parameters sent the the cursor. 3. Added tests using a PostgreSQL server. Tests are now run for ACCESS, MS SQL, MySQL and PostgreSQL. 4. Column name data access is now case insignificant (since PostgreSQL returns lower case column names). so (if a row object 'r' contains a first column 'spam') r[0], r.Spam, r.spam and r['SPAM'] are all equivalent. 5. The connection has new attributes .dbms_name and .dbms_version to display the underlying database engine. (like mxODBC) adodbapi is a Python DB-API 2.0 module that makes it easy to use Microsoft ADO for connecting with databases and other data sources using either CPython or IronPython. Home page: http://sourceforge.net/projects/adodbapi Features: * 100% DB-API 2.0 compliant. * Includes pyunit testcases that describe how to use the module. * Fully implemented in Python. -- runs in Python 2.3+ Python 3.0+ and IronPython 2.6+ * Licensed under the LGPL license, which means that it can be used freely even in commercial programs subject to certain restrictions. * Supports eGenix mxDateTime, Python 2.3 datetime module and Python time module. * Supports multiple paramstyles: 'qmark' 'named' 'format' * Supports data retrieval by column name e.g.: for row in myCurser.execute("select name,age from students"): print "Student", row.name, "is", row.age, "years old" * Supports user-definable system-to-Python data convertion functions (selected by ADO data type, or by column) * Money and Decimal column data is in decimal.Decimal format (unless you select another) -- Vernon Cole -------------- next part -------------- An HTML attachment was scrubbed... URL: