From robertb at trdlnk.com Mon Jun 11 22:26:38 2012 From: robertb at trdlnk.com (Robert Boehne) Date: Mon, 11 Jun 2012 15:26:38 -0500 Subject: [DB-SIG] Sybase module 0.40 released Message-ID: <4FD6547E.80809@trdlnk.com> WHAT IS IT: The Sybase module provides a Python interface to the Sybase relational database system. It supports all of the Python Database API, version 2.0 with extensions. The module is available here: http://downloads.sourceforge.net/python-sybase/python-sybase-0.40.tar.gz The module home page is here: http://python-sybase.sourceforge.net/ MAJOR CHANGES SINCE 0.39: Modify the DateTimeAsPython output conversion to return None when NULL is output support for Python without threads Ignore additional non-error codes from Sybase (1918 and 11932) Use outputmap in bulkcopy mode (thanks to patch by Cyrille Froehlich) Raise exception when opening a cursor on a closed connection Added unit tests Added new exception DeadLockError when Sybase is in a deadlock situation Add command properties CS_STICKY_BINDS and CS_HAVE_BINDS Added support for inputmap in bulkcopy reuse command and cursor when calling cursor.execute with same request Use ct_setparam to define ct_cursor parameters types instead of ct_param implicit conversion for CS_DATE_TYPE in CS_DATETIME_TYPE DataBuf Adding ct_cmd_props wrapper Increase DataBuf maxlength for params of a request when using CS_CHAR_TYPE params so that the buf can be reused BUGS CORRECTED SINCE 0.39: Corrected money type when using CS_MONEY4 (close bug 2615821) Corrected thread locking in ct_cmd_props (thanks to patch by Cyrille Froehlich) Corrected bug in type mapping in callproc (thanks to report by Skip Montanaro) Correct passing None in a DataBuf (thanks to patch by Bram Kuijvenhoven) The full ChangeLog is here: https://python-sybase.svn.sourceforge.net/svnroot/python-sybase/tags/r0_40/ChangeLog From mike_mp at zzzcomputing.com Tue Jun 12 16:07:41 2012 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Tue, 12 Jun 2012 10:07:41 -0400 Subject: [DB-SIG] definition of .rowcount Message-ID: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> Hey all - Had a user today confused about the word "affected", when we describe the definition of cursor.rowcount. Per pep-249: .rowcount This read-only attribute specifies the number of rows that the last .execute*() produced (for DQL statements like 'select') or affected (for DML statements like 'update' or 'insert'). So as many of you know, MySQL has actually two options for how rowcount can be reported. It can report the number of rows "found", that is, matched by the WHERE criterion of an UPDATE or DELETE statement, or it can report the number of rows that were actually updated or deleted, and here we're talking about an UPDATE that may or may not have had a net change in row value. The Python MySQL drivers tend to default to the latter. Every other database/DBAPI I've worked with only offers the former. Question is, is there an intent behind the term "affected" here? If it is intentionally vague, could there be some more explicit phrasing to that effect, "by 'affected' we mean whatever the underlying driver interprets this to mean" or something along those lines ? Or if it really means, "number of rows matched", should the word "affected" be altered here ? Thanks for reading. From mal at egenix.com Tue Jun 12 19:29:08 2012 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 12 Jun 2012 19:29:08 +0200 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> Message-ID: <4FD77C64.9070109@egenix.com> Michael Bayer wrote: > Hey all - > > Had a user today confused about the word "affected", when we describe the definition of cursor.rowcount. Per pep-249: > > .rowcount > > This read-only attribute specifies the number of rows that > the last .execute*() produced (for DQL statements like > 'select') or affected (for DML statements like 'update' or > 'insert'). > > So as many of you know, MySQL has actually two options for how rowcount can be reported. It can report the number of rows "found", that is, matched by the WHERE criterion of an UPDATE or DELETE statement, or it can report the number of rows that were actually updated or deleted, and here we're talking about an UPDATE that may or may not have had a net change in row value. The Python MySQL drivers tend to default to the latter. Every other database/DBAPI I've worked with only offers the former. > > Question is, is there an intent behind the term "affected" here? > > If it is intentionally vague, could there be some more explicit phrasing to that effect, "by 'affected' we mean whatever the underlying driver interprets this to mean" or something along those lines ? Or if it really means, "number of rows matched", should the word "affected" be altered here ? It is not really vague. For SELECT statements it refers to the number of rows matching the query and ready to be fetched. For UPDATE, DELETE and other statements manipulating rows it refers to the number of rows updated, deleted or inserted. Whether or not rows that actually changed or all rows touched by an UPDATE are counted is backend dependent. The latter is what most databases return (if they return any value at all). The term "affected" is a standard term used for describing the row count. See e.g. the ODBC API SQLRowCount: http://msdn.microsoft.com/en-us/library/windows/desktop/ms711835%28v=vs.85%29.aspx -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mike_mp at zzzcomputing.com Tue Jun 12 21:12:25 2012 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Tue, 12 Jun 2012 15:12:25 -0400 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <4FD77C64.9070109@egenix.com> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> <4FD77C64.9070109@egenix.com> Message-ID: <342003F8-0116-42E1-BC5E-D524396C1E11@zzzcomputing.com> On Jun 12, 2012, at 1:29 PM, M.-A. Lemburg wrote: > > It is not really vague. For SELECT statements it refers to the number > of rows matching the query and ready to be fetched. I'm not concerned about the SELECT case, most DBAPI's I've worked with don't support this usage (I suppose it assumes the cursor is buffering rows ? not sure) > For UPDATE, DELETE > and other statements manipulating rows it refers to the number of rows > updated, deleted or inserted. In that statement - "number of rows updated" - without the context that we on this list know what you mean, at face value it's not clear if "updated" means "matched" by the UPDATE or "actually changed with new data". Users of MySQL often have a notion of this term that is the opposite of what users of all the other database backends do. > > Whether or not rows that actually changed or all rows touched by > an UPDATE are counted is backend dependent. That was essentially my question - if "affected" is open to interpretation or if it means something specific. This says it is in fact open to interpretation. It would be nice if the spec added this phrase to the description. > > The term "affected" is a standard term used for describing the > row count. > > See e.g. the ODBC API SQLRowCount: > > http://msdn.microsoft.com/en-us/library/windows/desktop/ms711835%28v=vs.85%29.aspx I think in the vast majority of cases, nobody even debates what this term means. Only because MySQL has this odd behavior does the whole issue come into play. From peter_e at gmx.net Thu Jun 14 00:14:20 2012 From: peter_e at gmx.net (Peter Eisentraut) Date: Thu, 14 Jun 2012 01:14:20 +0300 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> Message-ID: <1339625660.11971.38.camel@vanquo.pezone.net> On tis, 2012-06-12 at 10:07 -0400, Michael Bayer wrote: > So as many of you know, MySQL has actually two options for how > rowcount can be reported. It can report the number of rows "found", > that is, matched by the WHERE criterion of an UPDATE or DELETE > statement, or it can report the number of rows that were actually > updated or deleted, and here we're talking about an UPDATE that may or > may not have had a net change in row value. The Python MySQL drivers > tend to default to the latter. Every other database/DBAPI I've > worked with only offers the former. Even if the value that is stored before and after an update is logically the same, this could fire triggers, so the row is still "affected". Also, some data types aren't comparable, so you can't always know this. So I think the problem with the alternative behavior of MySQL is that it isn't even generally definable, so it can't possibly be the correct answer for a general interface. There is also the question of how this definition for the UPDATE command extends to other commands. It probably doesn't, in a sensible way (well, maybe for REPLACE/INSERT ON DUPLICATE KEY UPDATE, but then you might get a second-level inconsistency between the regular UPDATE and the UPDATE run as part of the INSERT). I agree that the terms could be clarified, nevertheless. From mike_mp at zzzcomputing.com Thu Jun 14 00:43:23 2012 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Wed, 13 Jun 2012 18:43:23 -0400 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <1339625660.11971.38.camel@vanquo.pezone.net> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> <1339625660.11971.38.camel@vanquo.pezone.net> Message-ID: On Jun 13, 2012, at 6:14 PM, Peter Eisentraut wrote: > On tis, 2012-06-12 at 10:07 -0400, Michael Bayer wrote: >> So as many of you know, MySQL has actually two options for how >> rowcount can be reported. It can report the number of rows "found", >> that is, matched by the WHERE criterion of an UPDATE or DELETE >> statement, or it can report the number of rows that were actually >> updated or deleted, and here we're talking about an UPDATE that may or >> may not have had a net change in row value. The Python MySQL drivers >> tend to default to the latter. Every other database/DBAPI I've >> worked with only offers the former. > > Even if the value that is stored before and after an update is logically > the same, this could fire triggers, so the row is still "affected". > Also, some data types aren't comparable, so you can't always know this. > So I think the problem with the alternative behavior of MySQL is that it > isn't even generally definable, so it can't possibly be the correct > answer for a general interface. > > There is also the question of how this definition for the UPDATE command > extends to other commands. It probably doesn't, in a sensible way > (well, maybe for REPLACE/INSERT ON DUPLICATE KEY UPDATE, but then you > might get a second-level inconsistency between the regular UPDATE and > the UPDATE run as part of the INSERT). It goes without saying that this is one of a myriad number of areas where MySQL gets it wrong. > I agree that the terms could be clarified, nevertheless. Yup, I'd leave the word "affected" in, then just add a note that this can mean different things depending on backend. From mal at egenix.com Thu Jun 14 19:20:36 2012 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 14 Jun 2012 19:20:36 +0200 Subject: [DB-SIG] definition of .rowcount In-Reply-To: References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> <1339625660.11971.38.camel@vanquo.pezone.net> Message-ID: <4FDA1D64.1070605@egenix.com> Michael Bayer wrote: > > On Jun 13, 2012, at 6:14 PM, Peter Eisentraut wrote: > >> On tis, 2012-06-12 at 10:07 -0400, Michael Bayer wrote: >>> So as many of you know, MySQL has actually two options for how >>> rowcount can be reported. It can report the number of rows "found", >>> that is, matched by the WHERE criterion of an UPDATE or DELETE >>> statement, or it can report the number of rows that were actually >>> updated or deleted, and here we're talking about an UPDATE that may or >>> may not have had a net change in row value. The Python MySQL drivers >>> tend to default to the latter. Every other database/DBAPI I've >>> worked with only offers the former. >> >> Even if the value that is stored before and after an update is logically >> the same, this could fire triggers, so the row is still "affected". >> Also, some data types aren't comparable, so you can't always know this. >> So I think the problem with the alternative behavior of MySQL is that it >> isn't even generally definable, so it can't possibly be the correct >> answer for a general interface. >> >> There is also the question of how this definition for the UPDATE command >> extends to other commands. It probably doesn't, in a sensible way >> (well, maybe for REPLACE/INSERT ON DUPLICATE KEY UPDATE, but then you >> might get a second-level inconsistency between the regular UPDATE and >> the UPDATE run as part of the INSERT). > > It goes without saying that this is one of a myriad number of areas where MySQL gets it wrong. > >> I agree that the terms could be clarified, nevertheless. > > Yup, I'd leave the word "affected" in, then just add a note that this can mean different things depending on backend. I'll add a footnote next week, mentioning the possible different interpretation. BTW: I don't think the MySQL interpretation is particularly useful, since you typically use .rowcount to check whether the UPDATE actually found any rows. Even an update that doesn't change the current row values is an explicit update and should be counted. If you really don't want to count such rows, it's well possible to add a check the WHERE clause of the UPDATE to prevent (and not count) such updates. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mike_mp at zzzcomputing.com Thu Jun 14 19:24:48 2012 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Thu, 14 Jun 2012 13:24:48 -0400 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <4FDA1D64.1070605@egenix.com> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> <1339625660.11971.38.camel@vanquo.pezone.net> <4FDA1D64.1070605@egenix.com> Message-ID: <122B4577-E9D4-405C-9944-C70544D2D66F@zzzcomputing.com> On Jun 14, 2012, at 1:20 PM, M.-A. Lemburg wrote: > > I'll add a footnote next week, mentioning the possible different > interpretation. thanks ! > > BTW: I don't think the MySQL interpretation is particularly > useful, since you typically use .rowcount to check whether the > UPDATE actually found any rows. preaching to the choir ! From mal at egenix.com Mon Jun 25 17:15:28 2012 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 25 Jun 2012 17:15:28 +0200 Subject: [DB-SIG] definition of .rowcount In-Reply-To: <4FDA1D64.1070605@egenix.com> References: <56966308-1A37-4CB0-BC8A-361B796BF9BF@zzzcomputing.com> <1339625660.11971.38.camel@vanquo.pezone.net> <4FDA1D64.1070605@egenix.com> Message-ID: <4FE88090.3010001@egenix.com> M.-A. Lemburg wrote: > Michael Bayer wrote: >> >> On Jun 13, 2012, at 6:14 PM, Peter Eisentraut wrote: >> >>> On tis, 2012-06-12 at 10:07 -0400, Michael Bayer wrote: >>>> So as many of you know, MySQL has actually two options for how >>>> rowcount can be reported. It can report the number of rows "found", >>>> that is, matched by the WHERE criterion of an UPDATE or DELETE >>>> statement, or it can report the number of rows that were actually >>>> updated or deleted, and here we're talking about an UPDATE that may or >>>> may not have had a net change in row value. The Python MySQL drivers >>>> tend to default to the latter. Every other database/DBAPI I've >>>> worked with only offers the former. >>> >>> Even if the value that is stored before and after an update is logically >>> the same, this could fire triggers, so the row is still "affected". >>> Also, some data types aren't comparable, so you can't always know this. >>> So I think the problem with the alternative behavior of MySQL is that it >>> isn't even generally definable, so it can't possibly be the correct >>> answer for a general interface. >>> >>> There is also the question of how this definition for the UPDATE command >>> extends to other commands. It probably doesn't, in a sensible way >>> (well, maybe for REPLACE/INSERT ON DUPLICATE KEY UPDATE, but then you >>> might get a second-level inconsistency between the regular UPDATE and >>> the UPDATE run as part of the INSERT). >> >> It goes without saying that this is one of a myriad number of areas where MySQL gets it wrong. >> >>> I agree that the terms could be clarified, nevertheless. >> >> Yup, I'd leave the word "affected" in, then just add a note that this can mean different things depending on backend. > > I'll add a footnote next week, mentioning the possible different > interpretation. Done. The web site should update in a few hours. Here's the new footnote: .rowcount This read-only attribute specifies the number of rows that the last .execute*() produced (for DQL statements like 'select') or affected (for DML statements like 'update' or 'insert'). [9] ... [9] The term "number of affected rows" generally refers to the number of rows deleted, updated or inserted by the last statement run on the database cursor. Most databases will return the total number of rows that were found by the corresponding WHERE clause of the statement. Some databases use a different interpretation for UPDATEs and only return the number of rows that were changed by the UPDATE, even though the WHERE clause of the statement may have found more matching rows. Database module authors should try to implement the more common interpretation of returning the total number of rows found by the WHERE clause, or clearly document a different interpretation of the rowcount attribute. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 25 2012) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2012-07-17: Python Meeting Duesseldorf ... 22 days to go 2012-07-02: EuroPython 2012, Florence, Italy ... 7 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/