From miked at dewhirst.com.au Sun Apr 2 08:48:42 2006 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Sun, 02 Apr 2006 16:48:42 +1000 Subject: [DB-SIG] decimal datatypes In-Reply-To: References: Message-ID: <442F73CA.7090506@dewhirst.com.au> Chris > I hope I have the right list for this question... > > According to the homepage for pyodbc, pyodbc supports decimal data types, > but when I try to copy a row from one table to another, I'm getting: > > pyodbc.NotSupportedError: ('Python type Decimal not supported. param=26', > 'HY097') > try this ... from decimal import Decimal or from decimal import * I could be way off-beam with your problem but it sounds like the decimal module might be what you want. It contains the Decimal class. OTOH - it is possible or even more likely that you need a bespoke mapping from your actual data type to one supported by pyodbc (ie., decimal). OTTH - it is also possible that you spelled decimal incorrectly. It may actually start with a lower case d Good luck Mike From mkleehammer at gmail.com Mon Apr 3 22:17:21 2006 From: mkleehammer at gmail.com (Michael Kleehammer) Date: Mon, 3 Apr 2006 15:17:21 -0500 Subject: [DB-SIG] decimal datatypes (pyodbc) Message-ID: > According to the homepage for pyodbc, pyodbc supports decimal data types, > but when I try to copy a row from one table to another, I'm getting: > > pyodbc.NotSupportedError: ('Python type Decimal not supported. param=26', > 'HY097') Mike Dewhirst, Thanks for reporting this. Apparently the Decimal support wasn't complete. They were returned when selecting DECIMAL and NUMERIC columns but couldn't be used as parameters. I've implemented the necessary code in 2.0.18 which is ready for download. Thanks again, Michael Kleehammer PS, While we do monitor this newsgroup, if you create a bug report on sourceforge.net (http://sourceforge.net/tracker/?group_id=162557&atid=824254) it will send us an email right away. From szybalski at gmail.com Fri Apr 14 19:00:09 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Fri, 14 Apr 2006 12:00:09 -0500 Subject: [DB-SIG] mysql string length? Message-ID: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> Hello, I am trying to implement an database program but I am having these problems for which i don't seem to find an answer. 1. One of my columns in table_x is : body text not null when i use python to create sql : INSERT INTO table_x( body)VALUES( '%s')" % (body) this body is a string that varies in size. I keep getting an error if the size of body is longer then 255, and if its smaller everything goes smooth. Is this syntax correct? should '%s' be something else? 2. I get an error when i close() a connection and then reopen it in the same program. Is there a way around it. Here is more details on what i do. main : do function 1 do function 2 function 1: open connection to db get cursor close cursor close connection function 2: open connection to db get cursor close cursor close connection go to sleep for 6 hours go to function 1 I need to disconnect and then connect to db. Is there a reason i get (0, '') as an exception error? I read in docs that you can't disconnect and then connect, is that true? And if so is there a way around it? The easiest way to do it would be to keep the connection alive but that would mean the database will be filled with inactive connection, that might disconnect. Thank You -- Lukasz Szybalski From andychambers2002 at yahoo.co.uk Sat Apr 15 14:01:41 2006 From: andychambers2002 at yahoo.co.uk (Andrew Chambers) Date: Sat, 15 Apr 2006 12:01:41 +0000 Subject: [DB-SIG] mysql string length? In-Reply-To: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> Message-ID: <20060415120141.GA12186@yapgi.hopto.org> * Lukasz Szybalski [2006-04-14 12:00:09 -0500]: > INSERT INTO table_x( body)VALUES( '%s')" % (body) > > this body is a string that varies in size. I keep getting an error if > the size of body is longer then 255, and if its smaller everything > goes smooth. Is this syntax correct? should '%s' be something else? What is the datatype of body? It sounds like it is CHAR(255). Can you change this to be TEXT? Regards, Andy Send instant messages to your online friends http://uk.messenger.yahoo.com From szybalski at gmail.com Sun Apr 16 00:24:40 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Sat, 15 Apr 2006 17:24:40 -0500 Subject: [DB-SIG] mysql string length? In-Reply-To: <20060415120141.GA12186@yapgi.hopto.org> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> Message-ID: <1145139880.8861.9.camel@localhost.localdomain> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote: > * Lukasz Szybalski [2006-04-14 12:00:09 -0500]: > > > INSERT INTO table_x( body)VALUES( '%s')" % (body) > > > > this body is a string that varies in size. I keep getting an error if > > the size of body is longer then 255, and if its smaller everything > > goes smooth. Is this syntax correct? should '%s' be something else? > > What is the datatype of body? It sounds like it is CHAR(255). Can you > change this to be TEXT? > Yes it is text. mysql> describe table_x; +----------+---------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+---------+------+-----+---------+-------+ | id | int(11) | | PRI | 0 | | | body | text | YES | | NULL | | +----------+---------+------+-----+---------+-------+ Would some kind of characters that are passed in had something to do with the error, maybe EOF or something similar? The string that is passed in is from xml node. For debugging i made it return str(body), but that didn't change anything. Are there any other database/%s related requirements? Lukasz > Regards, > Andy > > Send instant messages to your online friends http://uk.messenger.yahoo.com > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From andy47 at halfcooked.com Sun Apr 16 12:42:09 2006 From: andy47 at halfcooked.com (Andy Todd) Date: Sun, 16 Apr 2006 20:42:09 +1000 Subject: [DB-SIG] mysql string length? In-Reply-To: <1145139880.8861.9.camel@localhost.localdomain> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> Message-ID: <44421F81.4080702@halfcooked.com> Lukasz Szybalski wrote: > On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote: >> * Lukasz Szybalski [2006-04-14 12:00:09 -0500]: >> >>> INSERT INTO table_x( body)VALUES( '%s')" % (body) >>> >>> this body is a string that varies in size. I keep getting an error if >>> the size of body is longer then 255, and if its smaller everything >>> goes smooth. Is this syntax correct? should '%s' be something else? >> What is the datatype of body? It sounds like it is CHAR(255). Can you >> change this to be TEXT? >> > Yes it is text. > mysql> describe table_x; > +----------+---------+------+-----+---------+-------+ > | Field | Type | Null | Key | Default | Extra | > +----------+---------+------+-----+---------+-------+ > | id | int(11) | | PRI | 0 | | > | body | text | YES | | NULL | | > +----------+---------+------+-----+---------+-------+ > > Would some kind of characters that are passed in had something to do > with the error, maybe EOF or something similar? > > The string that is passed in is from xml node. For debugging i made it > return str(body), but that didn't change anything. > > Are there any other database/%s related requirements? > > Lukasz > > >> Regards, >> Andy My observation is that with MySQLdb %s is a parameter substitution value not a string substitution indicator. What happens when you try something like this; >>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] >>> cursor.execute(stmt, (body,)) [1] note that there are no quote marks around the %s Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From engelbert.gruber at ssg.co.at Sun Apr 16 15:53:09 2006 From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at) Date: Sun, 16 Apr 2006 15:53:09 +0200 (CEST) Subject: [DB-SIG] mysql string length? In-Reply-To: <44421F81.4080702@halfcooked.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> Message-ID: On Sun, 16 Apr 2006, Andy Todd wrote: > Lukasz Szybalski wrote: >> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote: >>> * Lukasz Szybalski [2006-04-14 12:00:09 -0500]: >>> >>>> INSERT INTO table_x( body)VALUES( '%s')" % (body) >>>> >>>> this body is a string that varies in size. I keep getting an error if >>>> the size of body is longer then 255, and if its smaller everything >>>> goes smooth. Is this syntax correct? should '%s' be something else? >>> What is the datatype of body? It sounds like it is CHAR(255). Can you >>> change this to be TEXT? >>> >> Yes it is text. >> mysql> describe table_x; >> +----------+---------+------+-----+---------+-------+ >> | Field | Type | Null | Key | Default | Extra | >> +----------+---------+------+-----+---------+-------+ >> | id | int(11) | | PRI | 0 | | >> | body | text | YES | | NULL | | >> +----------+---------+------+-----+---------+-------+ >> >> Would some kind of characters that are passed in had something to do >> with the error, maybe EOF or something similar? >> >> The string that is passed in is from xml node. For debugging i made it >> return str(body), but that didn't change anything. >> >> Are there any other database/%s related requirements? not the way you pass it (as i understand it python first does "%" replacement and then passes the string. > My observation is that with MySQLdb %s is a parameter substitution value > not a string substitution indicator. > > What happens when you try something like this; > > >>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] > >>> cursor.execute(stmt, (body,)) > > [1] note that there are no quote marks around the %s this works here (debian testing, python 2.3.5, 4.0.21) :: import MySQLdb """ create table table_x ( id int auto_increment primary key, body text ) """ db = MySQLdb.connect(passwd="",db="test") c = db.cursor() body = "0....5...."*28 sql = "INSERT INTO table_x (body) VALUES('%s')" c.execute( sql % (body)) c.execute("select body from table_x") for row in c.fetchall(): print len(row[0]) i wont bet on fetching 1Mb. cheers -- From andy47 at halfcooked.com Mon Apr 17 11:45:46 2006 From: andy47 at halfcooked.com (Andy Todd) Date: Mon, 17 Apr 2006 19:45:46 +1000 Subject: [DB-SIG] mysql string length? In-Reply-To: References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> Message-ID: <444363CA.2030304@halfcooked.com> engelbert.gruber at ssg.co.at wrote: > On Sun, 16 Apr 2006, Andy Todd wrote: > >> Lukasz Szybalski wrote: >>> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote: >>>> * Lukasz Szybalski [2006-04-14 12:00:09 -0500]: >>>> >>>>> INSERT INTO table_x( body)VALUES( '%s')" % (body) >>>>> >>>>> this body is a string that varies in size. I keep getting an error if >>>>> the size of body is longer then 255, and if its smaller everything >>>>> goes smooth. Is this syntax correct? should '%s' be something else? >>>> What is the datatype of body? It sounds like it is CHAR(255). Can you >>>> change this to be TEXT? >>>> >>> Yes it is text. >>> mysql> describe table_x; >>> +----------+---------+------+-----+---------+-------+ >>> | Field | Type | Null | Key | Default | Extra | >>> +----------+---------+------+-----+---------+-------+ >>> | id | int(11) | | PRI | 0 | | >>> | body | text | YES | | NULL | | >>> +----------+---------+------+-----+---------+-------+ >>> >>> Would some kind of characters that are passed in had something to do >>> with the error, maybe EOF or something similar? >>> >>> The string that is passed in is from xml node. For debugging i made it >>> return str(body), but that didn't change anything. >>> >>> Are there any other database/%s related requirements? > > not the way you pass it (as i understand it python first does "%" > replacement and then passes the string. > >> My observation is that with MySQLdb %s is a parameter substitution value >> not a string substitution indicator. >> >> What happens when you try something like this; >> >>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] >>>>> cursor.execute(stmt, (body,)) >> [1] note that there are no quote marks around the %s > > this works here (debian testing, python 2.3.5, 4.0.21) > :: > > import MySQLdb > """ > create table table_x ( > id int auto_increment primary key, > body text > ) > """ > db = MySQLdb.connect(passwd="",db="test") > c = db.cursor() > body = "0....5...."*28 > sql = "INSERT INTO table_x (body) VALUES('%s')" > c.execute( sql % (body)) > c.execute("select body from table_x") > for row in c.fetchall(): > print len(row[0]) > > i wont bet on fetching 1Mb. > > cheers You've just repeated the original poster's code. As I suggested the problem is that he's using string substitution when he should be using parameter substitution. In these cases %s means two completely different things. Please read the sections on 'paramstyle' and the cursor '.execute' method in the DB-API 2.0 definition [1]. Regards, Andy [1] http://www.python.org/dev/peps/pep-0249/ -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From engelbert.gruber at ssg.co.at Mon Apr 17 14:56:11 2006 From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at) Date: Mon, 17 Apr 2006 14:56:11 +0200 (CEST) Subject: [DB-SIG] mysql string length? In-Reply-To: <444363CA.2030304@halfcooked.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> Message-ID: On Mon, 17 Apr 2006, Andy Todd wrote: >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] >>>>>> cursor.execute(stmt, (body,)) >>> [1] note that there are no quote marks around the %s >> >> this works here (debian testing, python 2.3.5, 4.0.21) :: >> >> import MySQLdb >> """ >> create table table_x ( >> id int auto_increment primary key, >> body text >> ) >> """ >> db = MySQLdb.connect(passwd="",db="test") >> c = db.cursor() >> body = "0....5...."*28 >> sql = "INSERT INTO table_x (body) VALUES('%s')" >> c.execute( sql % (body)) >> c.execute("select body from table_x") >> for row in c.fetchall(): >> print len(row[0]) >> >> i wont bet on fetching 1Mb. >> >> cheers > > You've just repeated the original poster's code. As I suggested the problem > is that he's using string substitution when he should be using parameter > substitution. In these cases %s means two completely different things. yes but it works here, even with ``body = "0....5...."*280`` so i cannot blame it on not using 'paramstyle', or what ? > Please read the sections on 'paramstyle' and the cursor '.execute' method in > the DB-API 2.0 definition [1]. of course you are right it is :: c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) or paramstyle without quotes :: c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) but both work here with bodylength up to 65535 bytes not 255. sorry for dupplication, but i fail to see the point. -- From farcepest at gmail.com Tue Apr 18 17:40:05 2006 From: farcepest at gmail.com (Andy Dustman) Date: Tue, 18 Apr 2006 11:40:05 -0400 Subject: [DB-SIG] mysql string length? In-Reply-To: References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> Message-ID: <9826f3800604180840p78b8c32cy7565d4dae8d8fda1@mail.gmail.com> On 4/17/06, engelbert.gruber at ssg.co.at wrote: > On Mon, 17 Apr 2006, Andy Todd wrote: > > >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] > >>>>>> cursor.execute(stmt, (body,)) > >>> [1] note that there are no quote marks around the %s > >> > >> this works here (debian testing, python 2.3.5, 4.0.21) :: > >> > >> import MySQLdb > >> """ > >> create table table_x ( > >> id int auto_increment primary key, > >> body text > >> ) > >> """ > >> db = MySQLdb.connect(passwd="",db="test") > >> c = db.cursor() > >> body = "0....5...."*28 > >> sql = "INSERT INTO table_x (body) VALUES('%s')" > >> c.execute( sql % (body)) > >> c.execute("select body from table_x") > >> for row in c.fetchall(): > >> print len(row[0]) > >> > >> i wont bet on fetching 1Mb. > >> > >> cheers > > > > You've just repeated the original poster's code. As I suggested the problem > > is that he's using string substitution when he should be using parameter > > substitution. In these cases %s means two completely different things. > > yes but it works here, even with ``body = "0....5...."*280`` so i cannot > blame it on not using 'paramstyle', or what ? > > > Please read the sections on 'paramstyle' and the cursor '.execute' method in > > the DB-API 2.0 definition [1]. > > of course you are right it is :: > > c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) Don't do that. It'll work right up until you have a string with quotes or \x00 bytes in it. Then bad things will happen. How bad depends on where body comes from (think SQL injection attacks). > or paramstyle without quotes :: > > c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) Do that instead. Special characters will be escaped. > but both work here with bodylength up to 65535 bytes not 255. TEXT columns are not the same as CHAR or VARCHAR. TEXT may be very large (they are non-binary BLOBs, but LOB sounds silly). CHAR/VARCHAR may be at most 255 characters. However, there are really four types of TEXT: TINYTEXT, TEXT, MEDIUMTEXT, and LONGTEXT. TEXT supports up to 65535 characters (16-bit length). If a 255-character limit is being hit, it's most likely either CHAR/VARCHAR or possibly TINYTEXT. -- The Pythonic Principle: Python works the way it does because if it didn't, it wouldn't be Python. From szybalski at gmail.com Tue Apr 18 17:46:45 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Tue, 18 Apr 2006 10:46:45 -0500 Subject: [DB-SIG] mysql string length? In-Reply-To: References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> Message-ID: <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> On 4/17/06, engelbert.gruber at ssg.co.at wrote: > On Mon, 17 Apr 2006, Andy Todd wrote: > > >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] > >>>>>> cursor.execute(stmt, (body,)) > >>> [1] note that there are no quote marks around the %s > >> > >> this works here (debian testing, python 2.3.5, 4.0.21) :: > >> > >> import MySQLdb > >> """ > >> create table table_x ( > >> id int auto_increment primary key, > >> body text > >> ) > >> """ > >> db = MySQLdb.connect(passwd="",db="test") > >> c = db.cursor() > >> body = "0....5...."*28 > >> sql = "INSERT INTO table_x (body) VALUES('%s')" > >> c.execute( sql % (body)) > >> c.execute("select body from table_x") > >> for row in c.fetchall(): > >> print len(row[0]) > >> > >> i wont bet on fetching 1Mb. > >> > >> cheers > > > > You've just repeated the original poster's code. As I suggested the problem > > is that he's using string substitution when he should be using parameter > > substitution. In these cases %s means two completely different things. > > yes but it works here, even with ``body = "0....5...."*280`` so i cannot > blame it on not using 'paramstyle', or what ? > > > Please read the sections on 'paramstyle' and the cursor '.execute' method in > > the DB-API 2.0 definition [1]. > > of course you are right it is :: > > c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) > > or paramstyle without quotes :: > > c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) > > but both work here with bodylength up to 65535 bytes not 255. > > sorry for dupplication, but i fail to see the point. > > Ok, I'm almost there. Thank you. So i finally realized that the length is not a problem but the actual body. The body includes characters such as " ' ", so ( I'm, I'll, etc) which cause it to give an error. >>>body="helo I'm here,I'll be there" >>> body=body *15 When i try : c.execute("Insert into table_x(body) VALUES(%s)",(body)) This will work but how do i make : sql="Insert into table_x(body) VALUES(%s)",(body) c.execute(sql) #complains that i give it a tuple, which i did when you look at sql One problem i see here is that it works in one way but no the other. So how do i add id and make it work both ways? c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) ??does not work I'm not familiar with ('pyformat' Python extended format codes, e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use "insert into table_x(id,body)VALUES('%d','%s')" % (id,body) -How to account for " I'll " in a body? -When would i HAVE to use pyformat? Can everything be done in 'format' ANSI C printf format codes, e.g. '...WHERE name=%s' Thanks, Lukasz From szybalski at gmail.com Tue Apr 18 20:00:34 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Tue, 18 Apr 2006 13:00:34 -0500 Subject: [DB-SIG] mysql string length? In-Reply-To: <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> Message-ID: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> On 4/18/06, Lukasz Szybalski wrote: > On 4/17/06, engelbert.gruber at ssg.co.at wrote: > > On Mon, 17 Apr 2006, Andy Todd wrote: > > > > >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] > > >>>>>> cursor.execute(stmt, (body,)) > > >>> [1] note that there are no quote marks around the %s > > >> > > >> this works here (debian testing, python 2.3.5, 4.0.21) :: > > >> > > >> import MySQLdb > > >> """ > > >> create table table_x ( > > >> id int auto_increment primary key, > > >> body text > > >> ) > > >> """ > > >> db = MySQLdb.connect(passwd="",db="test") > > >> c = db.cursor() > > >> body = "0....5...."*28 > > >> sql = "INSERT INTO table_x (body) VALUES('%s')" > > >> c.execute( sql % (body)) > > >> c.execute("select body from table_x") > > >> for row in c.fetchall(): > > >> print len(row[0]) > > >> > > >> i wont bet on fetching 1Mb. > > >> > > >> cheers > > > > > > You've just repeated the original poster's code. As I suggested the problem > > > is that he's using string substitution when he should be using parameter > > > substitution. In these cases %s means two completely different things. > > > > yes but it works here, even with ``body = "0....5...."*280`` so i cannot > > blame it on not using 'paramstyle', or what ? > > > > > Please read the sections on 'paramstyle' and the cursor '.execute' method in > > > the DB-API 2.0 definition [1]. > > > > of course you are right it is :: > > > > c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) > > > > or paramstyle without quotes :: > > > > c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) > > > > but both work here with bodylength up to 65535 bytes not 255. > > > > sorry for dupplication, but i fail to see the point. > > > > > > Ok, I'm almost there. Thank you. > So i finally realized that the length is not a problem but the actual > body. The body includes characters such as " ' ", so ( I'm, I'll, etc) > which cause it to give an error. > > >>>body="helo I'm here,I'll be there" > >>> body=body *15 > > When i try : > c.execute("Insert into table_x(body) VALUES(%s)",(body)) > This will work but how do i make : > > sql="Insert into table_x(body) VALUES(%s)",(body) > c.execute(sql) #complains that i give it a tuple, which i did when > you look at sql > > One problem i see here is that it works in one way but no the other. > So how do i add id and make it work both ways? > > c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) > ??does not work > > I'm not familiar with ('pyformat' Python extended format codes, > e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use > "insert into table_x(id,body)VALUES('%d','%s')" % (id,body) > > -How to account for " I'll " in a body? > -When would i HAVE to use pyformat? Can everything be done in > 'format' ANSI C printf format codes, e.g. '...WHERE name=%s' > Ok guys, thanks for help. I finally search for how to scape strings and i used: "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) where "'''" on each side of s = double quote + 3x single quote+ double quote Thanks again > Thanks, > Lukasz > -- Lukasz Szybalski www.lucasmanual.com From engelbert.gruber at ssg.co.at Tue Apr 18 20:09:56 2006 From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at) Date: Tue, 18 Apr 2006 20:09:56 +0200 (CEST) Subject: [DB-SIG] mysql string length? In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> Message-ID: On Tue, 18 Apr 2006, Lukasz Szybalski wrote: SNIP > Ok guys, thanks for help. I finally search for how to scape strings and i used: > > "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) > > where "'''" on each side of s = double quote + 3x single quote+ double quote as andy dustman said Don't do that. It'll work right up until you have a string with quotes or \x00 bytes in it. Then bad things will happen. How bad depends on where body comes from (think SQL injection attacks). > c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) Do that instead. Special characters will be escaped. cheers From python at venix.com Tue Apr 18 23:53:29 2006 From: python at venix.com (Python) Date: Tue, 18 Apr 2006 17:53:29 -0400 Subject: [DB-SIG] mysql string length? In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> Message-ID: <1145397209.6268.53.camel@www.venix.com> On Tue, 2006-04-18 at 13:00 -0500, Lukasz Szybalski wrote: > > When i try : > > c.execute("Insert into table_x(body) VALUES(%s)",(body)) > > This will work but how do i make : > > > > sql="Insert into table_x(body) VALUES(%s)",(body) > > c.execute(sql) #complains that i give it a tuple, which i did when c.execute( sql, (body,) ) (I'm pretty sure that with a single value this would also work: c.execute( sql, body ) but I do not have time to actually test that right now.) > > you look at sql > > > > One problem i see here is that it works in one way but no the other. > > So how do i add id and make it work both ways? > > > > c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) c.execute("Insert into table_x(id,body) VALUES(%s,%s)", (id,body) ) Notes: the placeholders are *ALWAYS* %s. This is not Python string formatting. There are two arguments, the SQL command string and the tuple of values to be substituted into the database. > > ??does not work > -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-653-8139 fax: 320-210-3409 -- Lloyd Kvam Venix Corp From andy47 at halfcooked.com Wed Apr 19 09:45:17 2006 From: andy47 at halfcooked.com (Andy Todd) Date: Wed, 19 Apr 2006 17:45:17 +1000 Subject: [DB-SIG] mysql string length? In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> Message-ID: <4445EA8D.6040806@halfcooked.com> Lukasz Szybalski wrote: > On 4/18/06, Lukasz Szybalski wrote: >> On 4/17/06, engelbert.gruber at ssg.co.at wrote: >>> On Mon, 17 Apr 2006, Andy Todd wrote: >>> >>>>>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] >>>>>>>>> cursor.execute(stmt, (body,)) >>>>>> [1] note that there are no quote marks around the %s >>>>> this works here (debian testing, python 2.3.5, 4.0.21) :: >>>>> >>>>> import MySQLdb >>>>> """ >>>>> create table table_x ( >>>>> id int auto_increment primary key, >>>>> body text >>>>> ) >>>>> """ >>>>> db = MySQLdb.connect(passwd="",db="test") >>>>> c = db.cursor() >>>>> body = "0....5...."*28 >>>>> sql = "INSERT INTO table_x (body) VALUES('%s')" >>>>> c.execute( sql % (body)) >>>>> c.execute("select body from table_x") >>>>> for row in c.fetchall(): >>>>> print len(row[0]) >>>>> >>>>> i wont bet on fetching 1Mb. >>>>> >>>>> cheers >>>> You've just repeated the original poster's code. As I suggested the problem >>>> is that he's using string substitution when he should be using parameter >>>> substitution. In these cases %s means two completely different things. >>> yes but it works here, even with ``body = "0....5...."*280`` so i cannot >>> blame it on not using 'paramstyle', or what ? >>> >>>> Please read the sections on 'paramstyle' and the cursor '.execute' method in >>>> the DB-API 2.0 definition [1]. >>> of course you are right it is :: >>> >>> c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) >>> >>> or paramstyle without quotes :: >>> >>> c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) >>> >>> but both work here with bodylength up to 65535 bytes not 255. >>> >>> sorry for dupplication, but i fail to see the point. >>> >>> >> Ok, I'm almost there. Thank you. >> So i finally realized that the length is not a problem but the actual >> body. The body includes characters such as " ' ", so ( I'm, I'll, etc) >> which cause it to give an error. >> >>>>> body="helo I'm here,I'll be there" >>>>> body=body *15 >> When i try : >> c.execute("Insert into table_x(body) VALUES(%s)",(body)) >> This will work but how do i make : >> >> sql="Insert into table_x(body) VALUES(%s)",(body) >> c.execute(sql) #complains that i give it a tuple, which i did when >> you look at sql >> >> One problem i see here is that it works in one way but no the other. >> So how do i add id and make it work both ways? >> >> c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) >> ??does not work >> >> I'm not familiar with ('pyformat' Python extended format codes, >> e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use >> "insert into table_x(id,body)VALUES('%d','%s')" % (id,body) >> >> -How to account for " I'll " in a body? >> -When would i HAVE to use pyformat? Can everything be done in >> 'format' ANSI C printf format codes, e.g. '...WHERE name=%s' >> > Ok guys, thanks for help. I finally search for how to scape strings and i used: > > "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) > > where "'''" on each side of s = double quote + 3x single quote+ double quote > > Thanks again >> Thanks, >> Lukasz >> > > > -- > Lukasz Szybalski > www.lucasmanual.com No, no, no, no. Do not use string substitution. I was trying to be subtle in my earlier messages but that didn't seem to work. In MySQLdb %s is a parameter substitution character NOT string substitution. Please repeat this thirty times and then come back and read the rest of this message. ... You've done it, right? OK. Your code examples re-written to use parameter substitution and not string substitution are; >>> c.execute("Insert into table_x(body) VALUES(%s)",(body)) This is good. >>> sql="Insert into table_x(body) VALUES(%s)",(body) >>> c.execute(sql) #complains that i give it a tuple, which i did when you look at sql this should be; >>> sql = "INSERT INTO table_x (body) VALUES (%s)" >>> c.execute(sql, (body,)) Note that you do parameter substitution when you execute your statement not when you assign it to a variable. Also note that the execute method expects a tuple hence the comma after 'body'. >>> c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) ??does not work And I'm very glad that it doesn't. You could do this as; >>> c.execute("Insert into table_x(id,body) VALUES(%s, %s)",(id, body)) Or >>> sql = "INSERT INTO table_x (id, body) VALUES (%s, %s)" >>> c.execute(sql, (id, body)) Note that in my versions of your code the substitution character is always %s. This does not mean that it's a string, MySQLdb is smart enough to figure out the underlying column's datatype and map whatever you provide (within reason of course) as part of the parameter substitution. >>> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) This is just not very good at all. If you carry on like this your code will almost entirely consist of single and double quotes and you will likely go stark raving mad. Once more, %s is simply a marker in the statement you provide to the execute method which says "substitute the next item from the tuple I've provided here please". It is not string substitution. Now hands up for changing the DB-API to remove the pyformat paramstyle? If we're going to have options I think we should restrict them to either named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC). Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From carsten at uniqsys.com Wed Apr 19 14:12:51 2006 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 19 Apr 2006 08:12:51 -0400 Subject: [DB-SIG] mysql string length? In-Reply-To: <4445EA8D.6040806@halfcooked.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> Message-ID: <1145448771.7406.10.camel@dot.uniqsys.com> On Wed, 2006-04-19 at 03:45, Andy Todd wrote: > Now hands up for changing the DB-API to remove the pyformat paramstyle? > If we're going to have options I think we should restrict them to either > named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC). I'm all for that with the clarification that 'pyformat' and 'format' should be deprecated, leaving 'qmark', 'numeric', and 'named'. Of course this change would have to go into a completely new version of DB-API, since it would break existing code. -Carsten From mal at egenix.com Wed Apr 19 14:24:10 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 14:24:10 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <4445EA8D.6040806@halfcooked.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> Message-ID: <44462BEA.6080006@egenix.com> Andy Todd wrote: > Once more, %s is simply a marker in the statement you provide to the > execute method which says "substitute the next item from the tuple I've > provided here please". It is not string substitution. > > Now hands up for changing the DB-API to remove the pyformat paramstyle? > If we're going to have options I think we should restrict them to either > named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC). Since this sort of confusion comes up again and again, I'm +1 on removing 'pyformat' and 'format' in the next version of the DB-API (and maybe marking it for deprecation in PEP 249 now). I've found the qmark style to be sufficient for everything, but then, I'm biased. For the few cases where binding variables are not allowed in SQL (e.g. table names), I use standard Python string formatting. Even then, things look very straight forward in the code: %s is a Python formatting marker, ? is a bound parameter. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From carsten at uniqsys.com Wed Apr 19 14:48:43 2006 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 19 Apr 2006 08:48:43 -0400 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44462BEA.6080006@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> Message-ID: <1145450923.7406.27.camel@dot.uniqsys.com> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: > I've found the qmark style to be sufficient for everything I agree. Considering that many modules already support qmark, and those that currently use 'format'/'pyformat' would have to move to something else if these are dropped, I'd like to suggest that supporting 'qmark' style should become a required feature. Modules would be free to also implement 'named' and/or 'numeric', but supporting 'qmark' should be mandatory. -Carsten From mal at egenix.com Wed Apr 19 16:20:53 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 16:20:53 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <1145450923.7406.27.camel@dot.uniqsys.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> Message-ID: <44464745.5020409@egenix.com> Carsten Haese wrote: > On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: >> I've found the qmark style to be sufficient for everything > > I agree. Considering that many modules already support qmark, and those > that currently use 'format'/'pyformat' would have to move to something > else if these are dropped, I'd like to suggest that supporting 'qmark' > style should become a required feature. Modules would be free to also > implement 'named' and/or 'numeric', but supporting 'qmark' should be > mandatory. I'm not sure whether that's easily possible for database modules which rely on different marker mechanisms, e.g. some modules probably use the implicit type information in the pyformat style to find out which method to use for binding variables. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From szybalski at gmail.com Wed Apr 19 16:43:11 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Wed, 19 Apr 2006 09:43:11 -0500 Subject: [DB-SIG] mysql string length? In-Reply-To: <4445EA8D.6040806@halfcooked.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> Message-ID: <804e5c70604190743y614cf9c1lb841a94a6575e10a@mail.gmail.com> On 4/19/06, Andy Todd wrote: > Lukasz Szybalski wrote: > > On 4/18/06, Lukasz Szybalski wrote: > >> On 4/17/06, engelbert.gruber at ssg.co.at wrote: > >>> On Mon, 17 Apr 2006, Andy Todd wrote: > >>> > >>>>>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1] > >>>>>>>>> cursor.execute(stmt, (body,)) > >>>>>> [1] note that there are no quote marks around the %s > >>>>> this works here (debian testing, python 2.3.5, 4.0.21) :: > >>>>> > >>>>> import MySQLdb > >>>>> """ > >>>>> create table table_x ( > >>>>> id int auto_increment primary key, > >>>>> body text > >>>>> ) > >>>>> """ > >>>>> db = MySQLdb.connect(passwd="",db="test") > >>>>> c = db.cursor() > >>>>> body = "0....5...."*28 > >>>>> sql = "INSERT INTO table_x (body) VALUES('%s')" > >>>>> c.execute( sql % (body)) > >>>>> c.execute("select body from table_x") > >>>>> for row in c.fetchall(): > >>>>> print len(row[0]) > >>>>> > >>>>> i wont bet on fetching 1Mb. > >>>>> > >>>>> cheers > >>>> You've just repeated the original poster's code. As I suggested the problem > >>>> is that he's using string substitution when he should be using parameter > >>>> substitution. In these cases %s means two completely different things. > >>> yes but it works here, even with ``body = "0....5...."*280`` so i cannot > >>> blame it on not using 'paramstyle', or what ? > >>> > >>>> Please read the sections on 'paramstyle' and the cursor '.execute' method in > >>>> the DB-API 2.0 definition [1]. > >>> of course you are right it is :: > >>> > >>> c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body)) > >>> > >>> or paramstyle without quotes :: > >>> > >>> c.execute(INSERT INTO table_x (body) VALUES(%s)", (body)) > >>> > >>> but both work here with bodylength up to 65535 bytes not 255. > >>> > >>> sorry for dupplication, but i fail to see the point. > >>> > >>> > >> Ok, I'm almost there. Thank you. > >> So i finally realized that the length is not a problem but the actual > >> body. The body includes characters such as " ' ", so ( I'm, I'll, etc) > >> which cause it to give an error. > >> > >>>>> body="helo I'm here,I'll be there" > >>>>> body=body *15 > >> When i try : > >> c.execute("Insert into table_x(body) VALUES(%s)",(body)) > >> This will work but how do i make : > >> > >> sql="Insert into table_x(body) VALUES(%s)",(body) > >> c.execute(sql) #complains that i give it a tuple, which i did when > >> you look at sql > >> > >> One problem i see here is that it works in one way but no the other. > >> So how do i add id and make it work both ways? > >> > >> c.execute("Insert into table_x(id,body) VALUES(%d,%s)",(id),(body)) > >> ??does not work > >> > >> I'm not familiar with ('pyformat' Python extended format codes, > >> e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use > >> "insert into table_x(id,body)VALUES('%d','%s')" % (id,body) > >> > >> -How to account for " I'll " in a body? > >> -When would i HAVE to use pyformat? Can everything be done in > >> 'format' ANSI C printf format codes, e.g. '...WHERE name=%s' > >> > > Ok guys, thanks for help. I finally search for how to scape strings and i used: > > > > "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) > > > > where "'''" on each side of s = double quote + 3x single quote+ double quote > > > > Thanks again > >> Thanks, > >> Lukasz > >> > > > > > > -- > > Lukasz Szybalski > > www.lucasmanual.com > > > No, no, no, no. Do not use string substitution. I was trying to be > subtle in my earlier messages but that didn't seem to work. In MySQLdb > %s is a parameter substitution character NOT string substitution. Please > repeat this thirty times and then come back and read the rest of this > message. > > ... > > You've done it, right? > > OK. Your code examples re-written to use parameter substitution and not > string substitution are; > > >>> c.execute("Insert into table_x(body) VALUES(%s)",(body)) > > This is good. > > >>> sql="Insert into table_x(body) VALUES(%s)",(body) > >>> c.execute(sql) #complains that i give it a tuple, which i did when > you look at sql > > this should be; > > >>> sql = "INSERT INTO table_x (body) VALUES (%s)" > >>> c.execute(sql, (body,)) > > Note that you do parameter substitution when you execute your statement > not when you assign it to a variable. Also note that the execute method > expects a tuple hence the comma after 'body'. > > >>> c.execute("Insert into table_x(id,body) > VALUES(%d,%s)",(id),(body)) ??does not work > > And I'm very glad that it doesn't. You could do this as; > > >>> c.execute("Insert into table_x(id,body) VALUES(%s, %s)",(id, body)) > > Or > > >>> sql = "INSERT INTO table_x (id, body) VALUES (%s, %s)" > >>> c.execute(sql, (id, body)) Sounds good. And just to make it even more clear, if i had 5 columns then : col3='hello' col4='bye' col5='example' sql = "insert into table_x( id, body, col3, col4, col5) values(%s,%s,%s,%s,%s)" c.execute(sql, (id,body,col3,col4,col5)) Great, this object/parameter substitution works like magic! Thanks again. Lukasz > Note that in my versions of your code the substitution character is > always %s. This does not mean that it's a string, MySQLdb is smart > enough to figure out the underlying column's datatype and map whatever > you provide (within reason of course) as part of the parameter substitution. > > >>> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body) > > This is just not very good at all. If you carry on like this your code > will almost entirely consist of single and double quotes and you will > likely go stark raving mad. > > Once more, %s is simply a marker in the statement you provide to the > execute method which says "substitute the next item from the tuple I've > provided here please". It is not string substitution. > > Now hands up for changing the DB-API to remove the pyformat paramstyle? > If we're going to have options I think we should restrict them to either > named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC). > > Regards, > Andy > -- > -------------------------------------------------------------------------------- > From the desk of Andrew J Todd esq - http://www.halfcooked.com/ > From gh at ghaering.de Wed Apr 19 16:46:04 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 19 Apr 2006 16:46:04 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44464745.5020409@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> Message-ID: <44464D2C.9020601@ghaering.de> M.-A. Lemburg wrote: > Carsten Haese wrote: > >>On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: >> >>>I've found the qmark style to be sufficient for everything >> >>I agree. Considering that many modules already support qmark, and those >>that currently use 'format'/'pyformat' would have to move to something >>else if these are dropped, I'd like to suggest that supporting 'qmark' >>style should become a required feature. Modules would be free to also >>implement 'named' and/or 'numeric', but supporting 'qmark' should be >>mandatory. > > I'm not sure whether that's easily possible for database modules > which rely on different marker mechanisms, e.g. some modules > probably use the implicit type information in the pyformat style > to find out which method to use for binding variables. The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only look at the Python values, not at the pyformat format character for deciding how to bind the variable. -- Gerhard From carsten at uniqsys.com Wed Apr 19 16:53:52 2006 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 19 Apr 2006 10:53:52 -0400 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44464745.5020409@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> Message-ID: <1145458432.7406.44.camel@dot.uniqsys.com> On Wed, 2006-04-19 at 10:20, M.-A. Lemburg wrote: > Carsten Haese wrote: > > On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: > >> I've found the qmark style to be sufficient for everything > > > > I agree. Considering that many modules already support qmark, and those > > that currently use 'format'/'pyformat' would have to move to something > > else if these are dropped, I'd like to suggest that supporting 'qmark' > > style should become a required feature. Modules would be free to also > > implement 'named' and/or 'numeric', but supporting 'qmark' should be > > mandatory. > > I'm not sure whether that's easily possible for database modules > which rely on different marker mechanisms, e.g. some modules > probably use the implicit type information in the pyformat style > to find out which method to use for binding variables. I don't follow. I thought we're voting on dropping 'pyformat' as a parameter style. Also, how does pyformat contain implicit type information? -Carsten From daniel.dittmar at sap.com Wed Apr 19 16:02:15 2006 From: daniel.dittmar at sap.com (Dittmar, Daniel) Date: Wed, 19 Apr 2006 16:02:15 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <1145450923.7406.27.camel@dot.uniqsys.com> Message-ID: >On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: >> I've found the qmark style to be sufficient for everything This is more about convenience (same as Python positional arguments vs. keyword arguments). Celko's book on SQL tree structures is full of examples using :names, with the same name appearing sevveral times in the same query. But I guess it's not that difficult to write an SQL tokenizer that can translate named to qmark and build a map from name to indices. > >I agree. Considering that many modules already support qmark, and those >that currently use 'format'/'pyformat' would have to move to something >else if these are dropped, I'd like to suggest that supporting 'qmark' >style should become a required feature. Modules would be free to also >implement 'named' and/or 'numeric', but supporting 'qmark' should be >mandatory. But how should a module support multiple notations? - by tokenizing any SQL statement to check which paramstyle was used - by having an option on the connection or the cursor - by having an additional method which requires qmark placeholders (this one would be backwards compatible) And while we're discussing annoying differences between drivers: aren't there some drivers that define def execute (stmt, *args) instead of def execute (stmt, args = []) Daniel -- Daniel Dittmar SAP Labs Berlin daniel.dittmar at sap.com From szybalski at gmail.com Wed Apr 19 16:56:30 2006 From: szybalski at gmail.com (Lukasz Szybalski) Date: Wed, 19 Apr 2006 09:56:30 -0500 Subject: [DB-SIG] Is there a fox pro substitute for linux? Message-ID: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> Hello again, Not sure if this is a right place to ask this but is there a "fox pro" substitute in Linux or a program that would act in a similar way but be platform independent (Linux,mac,win)? Fox pro aka, GUI way to design layout for database results, drug and drop, adjust size by mouse, create GUI for entering data in few minutes, and you just have to create sql. I assume something written in Java, or python. But I'm not sure if it exists or not. Does anybody came across something similar? Thx Lukasz From mal at egenix.com Wed Apr 19 17:04:49 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 17:04:49 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44464D2C.9020601@ghaering.de> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> Message-ID: <44465191.5090600@egenix.com> Gerhard H?ring wrote: > M.-A. Lemburg wrote: >> Carsten Haese wrote: >> >>> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: >>> >>>> I've found the qmark style to be sufficient for everything >>> I agree. Considering that many modules already support qmark, and those >>> that currently use 'format'/'pyformat' would have to move to something >>> else if these are dropped, I'd like to suggest that supporting 'qmark' >>> style should become a required feature. Modules would be free to also >>> implement 'named' and/or 'numeric', but supporting 'qmark' should be >>> mandatory. >> >> I'm not sure whether that's easily possible for database modules >> which rely on different marker mechanisms, e.g. some modules >> probably use the implicit type information in the pyformat style >> to find out which method to use for binding variables. > > The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only > look at the Python values, not at the pyformat format character for > deciding how to bind the variable. If that's the case, then I suppose supporting qmark style wouldn't be too much of a problem. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Apr 19 17:06:47 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 17:06:47 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <1145458432.7406.44.camel@dot.uniqsys.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <44464745.5020409@egenix.com> <1145458432.7406.44.camel@dot.uniqsys.com> Message-ID: <44465207.1000003@egenix.com> Carsten Haese wrote: > On Wed, 2006-04-19 at 10:20, M.-A. Lemburg wrote: >> Carsten Haese wrote: >>> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: >>>> I've found the qmark style to be sufficient for everything >>> I agree. Considering that many modules already support qmark, and those >>> that currently use 'format'/'pyformat' would have to move to something >>> else if these are dropped, I'd like to suggest that supporting 'qmark' >>> style should become a required feature. Modules would be free to also >>> implement 'named' and/or 'numeric', but supporting 'qmark' should be >>> mandatory. >> I'm not sure whether that's easily possible for database modules >> which rely on different marker mechanisms, e.g. some modules >> probably use the implicit type information in the pyformat style >> to find out which method to use for binding variables. > > I don't follow. I thought we're voting on dropping 'pyformat' as a > parameter style. Also, how does pyformat contain implicit type > information? Markers such as %i and %s could be treated as meaning: bind these variables as integer and string. This implicit type information might be vital to some modules. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Apr 19 17:13:36 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 17:13:36 +0200 Subject: [DB-SIG] Is there a fox pro substitute for linux? In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> Message-ID: <444653A0.5020508@egenix.com> Lukasz Szybalski wrote: > Hello again, > Not sure if this is a right place to ask this but is there a "fox pro" > substitute in Linux or a program that would act in a similar way but > be platform independent (Linux,mac,win)? > > Fox pro aka, GUI way to design layout for database results, drug and > drop, adjust size by mouse, create GUI for entering data in few > minutes, and you just have to create sql. > > I assume something written in Java, or python. But I'm not sure if it > exists or not. Does anybody came across something similar? You might be looking for something like theKompany's Rekall: http://www.thekompany.com/products/rekall/ You can script it using Python, so it's vaguely related to this mailing list :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From plblists at iotk.com Wed Apr 19 17:09:32 2006 From: plblists at iotk.com (Peter L. Buschman) Date: Wed, 19 Apr 2006 17:09:32 +0200 Subject: [DB-SIG] Is there a fox pro substitute for linux? In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com > References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> Message-ID: <7.0.1.0.2.20060419170856.03d57470@iotk.com> Check out Dabo :) http://dabodev.com/ --PLB At 16:56 19.4.06, Lukasz Szybalski wrote: >Hello again, >Not sure if this is a right place to ask this but is there a "fox pro" >substitute in Linux or a program that would act in a similar way but >be platform independent (Linux,mac,win)? > >Fox pro aka, GUI way to design layout for database results, drug and >drop, adjust size by mouse, create GUI for entering data in few >minutes, and you just have to create sql. > >I assume something written in Java, or python. But I'm not sure if it >exists or not. Does anybody came across something similar? > >Thx >Lukasz >_______________________________________________ >DB-SIG maillist - DB-SIG at python.org >http://mail.python.org/mailman/listinfo/db-sig From ianb at colorstudy.com Wed Apr 19 17:21:09 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 19 Apr 2006 10:21:09 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44465191.5090600@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> Message-ID: <44465565.8090801@colorstudy.com> M.-A. Lemburg wrote: >>The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only >>look at the Python values, not at the pyformat format character for >>deciding how to bind the variable. > > > If that's the case, then I suppose supporting qmark style > wouldn't be too much of a problem. AFAIK, they all use %, so you have to do execute("%s like 'foo%%'", (value,)); this is quite different from qmark. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From gh at ghaering.de Wed Apr 19 17:38:09 2006 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Wed, 19 Apr 2006 17:38:09 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44465191.5090600@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> Message-ID: <44465961.1090904@ghaering.de> M.-A. Lemburg wrote: > [...] > If that's the case, then I suppose supporting qmark style > wouldn't be too much of a problem. It's doable, but it requires parsing the SQL, i. e. find all the qmarks that are not inside SQL string literals. Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively support parameter binding, so the Python adapter will end up quoting parameters and stuffing them in the query string. Which is why these modules just use (py)format now, then they only need to quote the parameters appropriately. The rest they get for free from Python. Abolishing (py)format certainly means additional work for the module authors. If we do so - I haven't read the whole thread, so I don't know what the arguments are for it - then we should include example code in the DB-API for parsing qmarks out of ANSI SQL statements. -- Gerhard (*) Only in a stupid, nontransparent form. From carsten at uniqsys.com Wed Apr 19 18:13:44 2006 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 19 Apr 2006 12:13:44 -0400 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44465961.1090904@ghaering.de> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> Message-ID: <1145463224.7406.83.camel@dot.uniqsys.com> On Wed, 2006-04-19 at 11:38, Gerhard H?ring wrote: > Abolishing (py)format certainly means additional work for the module > authors. If we do so - I haven't read the whole thread, so I don't know > what the arguments are for it - then we should include example code in > the DB-API for parsing qmarks out of ANSI SQL statements. The main argument for abolishing (py)format is that it blurs the line between parameter passing, which is good, and hand-rolling a query via string substitution, which is bad because it invites SQL injections if not done carefully, and it's almost never done carefully. Especially newbies seem to have a problem with telling the two apart and understanding why parameter binding is better than string substitution. Abolishing %s should make it a lot easier to clearly separate the two concepts. Ian also brought up the point that implementations that use (py)format have a rather ugly wart: Literal % signs in queries have to be doubled up to prevent accidental parameter markers. This is ugly and makes writing portable code unnecessarily hard. I agree that if we decide to abolish (py)format, we should help out module authors for databases that don't natively support '?' by providing example code for performing the necessary parsing. -Carsten From leif at solumslekt.org Wed Apr 19 17:22:32 2006 From: leif at solumslekt.org (Leif B. Kristensen) Date: Wed, 19 Apr 2006 17:22:32 +0200 Subject: [DB-SIG] Is there a fox pro substitute for linux? In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> Message-ID: <200604191722.32386.leif@solumslekt.org> On Wednesday 19 April 2006 16:56, Lukasz Szybalski wrote: >Hello again, >Not sure if this is a right place to ask this but is there a "fox pro" >substitute in Linux or a program that would act in a similar way but >be platform independent (Linux,mac,win)? > >Fox pro aka, GUI way to design layout for database results, drug and >drop, adjust size by mouse, create GUI for entering data in few >minutes, and you just have to create sql. > >I assume something written in Java, or python. But I'm not sure if it >exists or not. Does anybody came across something similar? Dabo is probably the closest you'll get. I haven't tried it seriously, mostly because of the poor documentation, but they had a quite an active and helpful mailing list at the time I looked into it about half a year ago. Personally, I prefer plain old PHP. If you know something about HTML forms, you can churn out a Web interface with it in no time. The downside is of course the inherent statelessness of the http protocol. -- Leif Biberg Kristensen | Registered Linux User #338009 http://solumslekt.org/ | Cruising with Gentoo/KDE From ianb at colorstudy.com Wed Apr 19 18:29:03 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 19 Apr 2006 11:29:03 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <1145463224.7406.83.camel@dot.uniqsys.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> Message-ID: <4446654F.3020704@colorstudy.com> Carsten Haese wrote: > On Wed, 2006-04-19 at 11:38, Gerhard H?ring wrote: > >>Abolishing (py)format certainly means additional work for the module >>authors. If we do so - I haven't read the whole thread, so I don't know >>what the arguments are for it - then we should include example code in >>the DB-API for parsing qmarks out of ANSI SQL statements. > > > The main argument for abolishing (py)format is that it blurs the line > between parameter passing, which is good, and hand-rolling a query via > string substitution, which is bad because it invites SQL injections if > not done carefully, and it's almost never done carefully. It's not *so* bad, because this works: execute('... where x = %s', (x,)) And this will fail immediately: execute('... where x = %s' % (x,)) It's only repr() which really confuses things. Well, that and cases where you have integers. Anyway, it's significantly better than PHP. > Especially newbies seem to have a problem with telling the two apart and > understanding why parameter binding is better than string substitution. > Abolishing %s should make it a lot easier to clearly separate the two > concepts. > > Ian also brought up the point that implementations that use (py)format > have a rather ugly wart: Literal % signs in queries have to be doubled > up to prevent accidental parameter markers. This is ugly and makes > writing portable code unnecessarily hard. > > I agree that if we decide to abolish (py)format, we should help out > module authors for databases that don't natively support '?' by > providing example code for performing the necessary parsing. When using a higher-level SQL generator the parsing overhead would be unnecessary, as such a library can produce proper pyformat queries itself. Using a parser and whatnot might be more reasonable if the user could select the format in some manner; then libraries that didn't care could stick with the most efficient one for the driver. OK, crazy idea: use \x00 as a marker, which requires no quoting or parsing (as it can't be included in any SQL literal anyway). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From python at venix.com Wed Apr 19 18:50:22 2006 From: python at venix.com (Python) Date: Wed, 19 Apr 2006 12:50:22 -0400 Subject: [DB-SIG] Is there a fox pro substitute for linux? In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com> Message-ID: <1145465422.6268.90.camel@www.venix.com> On Wed, 2006-04-19 at 09:56 -0500, Lukasz Szybalski wrote: > Hello again, > Not sure if this is a right place to ask this but is there a "fox pro" > substitute in Linux or a program that would act in a similar way but > be platform independent (Linux,mac,win)? > > Fox pro aka, GUI way to design layout for database results, drug and > drop, adjust size by mouse, create GUI for entering data in few > minutes, and you just have to create sql. > > I assume something written in Java, or python. But I'm not sure if it > exists or not. Does anybody came across something similar? http://dabodev.com/ The key developers on this project are FoxPro veterans and are trying to incorporate FoxPro features that they felt were important. They've presented at the last 2 PyCons. > > Thx > Lukasz > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig -- Lloyd Kvam Venix Corp From ianb at colorstudy.com Wed Apr 19 19:00:44 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 19 Apr 2006 12:00:44 -0500 Subject: [DB-SIG] SQL-API Message-ID: <44466CBC.3000502@colorstudy.com> Hi guys. So, I've been meaning to bring this up here, and what with the param marker discussions... Anyway, while doing the SQLObject 2 rewrite I wanted to extract some of the machinery that has nothing to do with ORM stuff. The result is SQL-API: http://sqlobject.org/sqlapi/ Instead of fixing problems with DB-API, it tries more to wrap existing DB-API drivers and give them a more consistent, higher-level interface. It's also intended to be pretty neutral about everything else, and it is entirely procedural. I'd really like to move this forward so that it can serve the purposes of more than just SQLObject 2, so I'm interested in other consumers, particularly people using very different strategies from SQLObject. There's a lot of tricky aspects to using the DB-API that I think can really be solved to everyone's satisfaction once, instead of each system having to implement these things themselves -- things like configuration, logging, pooling, exceptions, and also a place to hang other more complex stuff like database portability information, or the occassional driver monkeypatch. Both the scope and implementation of SQL-API is fairly open at this point, so I'm very open to suggestions or contributions. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From mal at egenix.com Wed Apr 19 20:36:21 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 19 Apr 2006 20:36:21 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <4446654F.3020704@colorstudy.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> Message-ID: <44468325.3050007@egenix.com> Ian Bicking wrote: > Carsten Haese wrote: >> I agree that if we decide to abolish (py)format, we should help out >> module authors for databases that don't natively support '?' by >> providing example code for performing the necessary parsing. > > When using a higher-level SQL generator the parsing overhead would be > unnecessary, as such a library can produce proper pyformat queries > itself. Using a parser and whatnot might be more reasonable if the user > could select the format in some manner; then libraries that didn't care > could stick with the most efficient one for the driver. Actually, I don't think that parsing SQL is really necessary at all: in all the years I've used qmark style, I've never come across a situation where a SQL literal would include a question mark. In reality, it all boils down to doing a simple search for '?' in the string - after all, you usually pass strings in via bound parameters. > OK, crazy idea: use \x00 as a marker, which requires no quoting or > parsing (as it can't be included in any SQL literal anyway). Not sure about that one: some systems such as e.g. Zope use \x00 to delimit multi-line SQL statements. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Chris.Clark at ingres.com Thu Apr 20 22:53:36 2006 From: Chris.Clark at ingres.com (Chris Clark) Date: Thu, 20 Apr 2006 13:53:36 -0700 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44465207.1000003@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <44464745.5020409@egenix.com> <1145458432.7406.44.camel@dot.uniqsys.com> <44465207.1000003@egenix.com> Message-ID: <4447F4D0.1020303@ingres.com> My vote - +1 for making qmark mandatory and everything else optional My ramblings - M.-A. Lemburg wrote: > Markers such as %i and %s could be treated as meaning: bind > these variables as integer and string. This implicit type > information might be vital to some modules. > > I have to confess I couldn't figure out from the v2 spec if this was implied or not for pyformat - I've certainly seen dbi users (e.g. Django), using %s irrespective of the Python or DBMS type; relying on on the host DBMS to then coerce, e.g. into an integer column. The way I read the spec was that the constructors in the "Type Objects and Constructors" section in pep 249 are to be used to show the destination type. E.g.: z=23 x.execute("select ?", (dbmodule.NUMBER(z),) ) I'm definitely a +1 for making qmark mandatory and everything else optional. Question marks are standard in SQL across a few api's, e.g. ODBC, embedded (sql) C, JDBC, etc. Gerhard suggestion of example code for parsing is a good one, but someone would have pony up some code :-) Some drivers are pure C (so they would need pure C code) and others use some Python code and they would prefer Python so this (apparently) simple request may not get fulfilled :-( (in a later email) M.-A. Lemburg wrote: > Actually, I don't think that parsing SQL is really necessary > at all: in all the years I've used qmark style, I've never come > across a situation where a SQL literal would include a question > mark. > > In reality, it all boils down to doing a simple search for > '?' in the string - after all, you usually pass strings in via > bound parameters. For most real world applications I think this is a reasonable assumption, however there is always a an edge case and I wouldn't be surprised if some one on the list says, "Hang on! I've got some dynamic queries with string literals containing question marks". But for initial testing: sql_query = "select foo from bar where a = ? and b like '%slugs'" new_sql_query = sql_query.replace('%', '%%') # deal with embedded percent signs new_sql_query = new_sql_query.replace('?', '%s') # will NOT deal with embedded qmarks in string literals! ## what ever pyformat driver currently does for pyformat queries is probably a resonable short term testing solution until a more robust solution is found for drivers that don't natively support parameter binding. Chris From dieter at handshake.de Thu Apr 20 20:08:48 2006 From: dieter at handshake.de (Dieter Maurer) Date: Thu, 20 Apr 2006 20:08:48 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44468325.3050007@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> Message-ID: <17479.52784.244921.297972@gargle.gargle.HOWL> M.-A. Lemburg wrote at 2006-4-19 20:36 +0200: > ... >Actually, I don't think that parsing SQL is really necessary >at all: in all the years I've used qmark style, I've never come >across a situation where a SQL literal would include a question >mark. I do not think this is a good idea. '?' are perfectly legal in string literals and there they conventionally do *NOT* mean "insert a parameter". >In reality, it all boils down to doing a simple search for >'?' in the string - after all, you usually pass strings in via >bound parameters. Zope, at least, would be unhappy, as it does not (yet) support bound parameters. -- Dieter From ianb at colorstudy.com Thu Apr 20 23:15:32 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 20 Apr 2006 16:15:32 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44468325.3050007@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> Message-ID: <4447F9F4.30607@colorstudy.com> M.-A. Lemburg wrote: >>OK, crazy idea: use \x00 as a marker, which requires no quoting or >>parsing (as it can't be included in any SQL literal anyway). > > > Not sure about that one: some systems such as e.g. Zope use > \x00 to delimit multi-line SQL statements. \x01 then! -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Thu Apr 20 23:29:37 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 20 Apr 2006 16:29:37 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: References: Message-ID: <4447FD41.9080506@colorstudy.com> Paul DuBois wrote: > On 4/20/06 13:08, "Dieter Maurer" wrote: > > >>M.-A. Lemburg wrote at 2006-4-19 20:36 +0200: >> >>>... >>>Actually, I don't think that parsing SQL is really necessary >>>at all: in all the years I've used qmark style, I've never come >>>across a situation where a SQL literal would include a question >>>mark. >> >>I do not think this is a good idea. '?' are perfectly legal >>in string literals and there they conventionally >>do *NOT* mean "insert a parameter". > > > Umm ... thing is, the ? placeholder markers in a SQL string _don't_ occur > within string literals. That is, when you construct a SQL statement > containing placeholder markers, you don't write the placeholders within > quote marks. The parameter substitution mechanism adds quotes as necessary > when it substitutes a data value for a placeholder. He's referring to something like this: def get_androgenous_users(state): execute("SELECT * FROM users WHERE state = ? and gender = '?', (state,)) ... -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From paul at snake.net Thu Apr 20 23:27:49 2006 From: paul at snake.net (Paul DuBois) Date: Thu, 20 Apr 2006 16:27:49 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <17479.52784.244921.297972@gargle.gargle.HOWL> Message-ID: On 4/20/06 13:08, "Dieter Maurer" wrote: > M.-A. Lemburg wrote at 2006-4-19 20:36 +0200: >> ... >> Actually, I don't think that parsing SQL is really necessary >> at all: in all the years I've used qmark style, I've never come >> across a situation where a SQL literal would include a question >> mark. > > I do not think this is a good idea. '?' are perfectly legal > in string literals and there they conventionally > do *NOT* mean "insert a parameter". Umm ... thing is, the ? placeholder markers in a SQL string _don't_ occur within string literals. That is, when you construct a SQL statement containing placeholder markers, you don't write the placeholders within quote marks. The parameter substitution mechanism adds quotes as necessary when it substitutes a data value for a placeholder. If this were not true, then binding NULL to a placeholder would not be possible. If you wrote '?' rather than ?, NULL would incorrectly become 'NULL' and thus would itself become a literal string in the statement. >> In reality, it all boils down to doing a simple search for >> '?' in the string - after all, you usually pass strings in via >> bound parameters. > > Zope, at least, would be unhappy, as it does not (yet) support > bound parameters. > From farcepest at gmail.com Fri Apr 21 07:37:06 2006 From: farcepest at gmail.com (Andy Dustman) Date: Fri, 21 Apr 2006 01:37:06 -0400 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <44465961.1090904@ghaering.de> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> Message-ID: <9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com> On 4/19/06, Gerhard H?ring wrote: > M.-A. Lemburg wrote: > > [...] > > If that's the case, then I suppose supporting qmark style > > wouldn't be too much of a problem. > > It's doable, but it requires parsing the SQL, i. e. find all the qmarks > that are not inside SQL string literals. > > Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively > support parameter binding, so the Python adapter will end up quoting > parameters and stuffing them in the query string. -1 on making qmark mandatory, even though I am constantly having to correct newbies who see %s and think, "I must use a string formatting operator since I am too ignorant to read PEP-249". On the other hand, MySQL-4.1 and newer *does* support ? as a placeholder *iff* you are using the prepared statement API, which MySQLdb does not implement *yet*, but that is one of the next things to do. One complication is, not all SQL statements will work with the prepared statements API, and there is a lot of code out there that correctly uses %s. I am probably going to have to have some means for indicating what paramstyle is in use by the application, so the driver can transliterate it if it needs to, and that's one of the things I have to figure out for MySQLdb-1.3/1.4/2.0. -- The Pythonic Principle: Python works the way it does because if it didn't, it wouldn't be Python. From mal at egenix.com Fri Apr 21 13:30:20 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Apr 2006 13:30:20 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com> Message-ID: <4448C24C.7030700@egenix.com> Andy Dustman wrote: > On 4/19/06, Gerhard H?ring wrote: >> M.-A. Lemburg wrote: >>> [...] >>> If that's the case, then I suppose supporting qmark style >>> wouldn't be too much of a problem. >> It's doable, but it requires parsing the SQL, i. e. find all the qmarks >> that are not inside SQL string literals. >> >> Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively >> support parameter binding, so the Python adapter will end up quoting >> parameters and stuffing them in the query string. > > -1 on making qmark mandatory, even though I am constantly having to > correct newbies who see %s and think, "I must use a string formatting > operator since I am too ignorant to read PEP-249". > > On the other hand, MySQL-4.1 and newer *does* support ? as a > placeholder *iff* you are using the prepared statement API, which > MySQLdb does not implement *yet*, but that is one of the next things > to do. One complication is, not all SQL statements will work with the > prepared statements API, and there is a lot of code out there that > correctly uses %s. I am probably going to have to have some means for > indicating what paramstyle is in use by the application, so the driver > can transliterate it if it needs to, and that's one of the things I > have to figure out for MySQLdb-1.3/1.4/2.0. If we make qmark support mandatory and deprecate pyformat/format in DB API 2.1 would that be a workable approach ? I'd say that we schedule DB API 2.1 for later this year with the main aim of making some of the currently already documented standard extensions mandatory and maybe adding optional support for defining type mappings (both input and output). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 21 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From wilk-ml at flibuste.net Sat Apr 22 20:53:42 2006 From: wilk-ml at flibuste.net (William Dode) Date: Sat, 22 Apr 2006 18:53:42 +0000 (UTC) Subject: [DB-SIG] how to know the connexion is closed Message-ID: Hi, I search for a generic way (for psycopg, mysqldb, adodbapi) to know that the connexion is closed, for example because the database server was restarted. I mean for wich database exceptions should i ask again for a connexion ? thanks -- William Dod? - http://flibuste.net From mal at egenix.com Mon Apr 24 20:30:10 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 24 Apr 2006 20:30:10 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <17479.52784.244921.297972@gargle.gargle.HOWL> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> <17479.52784.244921.297972@gargle.gargle.HOWL> Message-ID: <444D1932.1020503@egenix.com> Dieter Maurer wrote: > M.-A. Lemburg wrote at 2006-4-19 20:36 +0200: >> ... >> Actually, I don't think that parsing SQL is really necessary >> at all: in all the years I've used qmark style, I've never come >> across a situation where a SQL literal would include a question >> mark. > > I do not think this is a good idea. '?' are perfectly legal > in string literals and there they conventionally > do *NOT* mean "insert a parameter". Right, but if you use bound parameters, you rarely ever have a need to use question marks in SQL literals. >> In reality, it all boils down to doing a simple search for >> '?' in the string - after all, you usually pass strings in via >> bound parameters. > > Zope, at least, would be unhappy, as it does not (yet) support > bound parameters. Would be a good incentive to change this :-) Of course, scanning the SQL statement string for SQL literals is safer, so this is what modules should do (at least if the user passes in bound parameters to the .execute() method). Does anyone know a good implementation of such a search&replace mechanism ? I tried coming up with something using re.sub() but it seems this is beyond what you can do with re. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From patriciap.gu at gmail.com Mon Apr 24 20:35:33 2006 From: patriciap.gu at gmail.com (Patty) Date: Mon, 24 Apr 2006 18:35:33 +0000 (UTC) Subject: [DB-SIG] MySQLdb - syntax question Message-ID: I have the following code: UPDATE mytable SET col1 = %s, col2 = %s, col3 = %s, col4 = %s, col5 = %s, col6 = %s WHERE name = %s """, (tuple(entries))) In this example, entries is a list of numbers and the last element is a name of a target. This works fine. But, there will be times when new columns will be added based on a dictionary I have. I've been asked not to hard code the column names (i.e. col1, col2, etc.). I need to know if it is possible to put placeholders instead of column names (i.e. %s=%s -> column name = column value). If so, what is the right syntax for that? Thanks, Patty From chris at cogdon.org Mon Apr 24 20:51:19 2006 From: chris at cogdon.org (Chris Cogdon) Date: Mon, 24 Apr 2006 11:51:19 -0700 Subject: [DB-SIG] MySQLdb - syntax question In-Reply-To: References: Message-ID: On Apr 24, 2006, at 11:35, Patty wrote: > I have the following code: > > UPDATE mytable SET col1 = %s, col2 = %s, col3 = %s, col4 = %s, > col5 = %s, col6 = %s WHERE name = %s > """, (tuple(entries))) > In this example, entries is a list of numbers and the last element is > a name of > a target. > > This works fine. But, there will be times when new columns will be > added based > on a dictionary I have. I've been asked not to hard code the column > names (i.e. > col1, col2, etc.). I need to know if it is possible to put > placeholders instead > of column names (i.e. %s=%s -> column name = column value). If so, > what is the > right syntax for that? Unfortunately, using the %s syntax is going to put the name of the column in quotes, and so MySQL will think you're trying to assign one string to another, rather than a string to a column. Instead, you'll need to piecemeal your string from parts manually, like this: d = { ... your dictionary of column names and values } d_items = d.items () # doing it this way to ensure that we dont get inconsistent ordering query_string = "update mytable set " + ",".join ( [ key + "=%s" for key, value in d_items ] ) cursor.execute ( query_string, tuple ( [ value for key, value in d_items ] ) ) -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From dieter at handshake.de Mon Apr 24 21:09:43 2006 From: dieter at handshake.de (Dieter Maurer) Date: Mon, 24 Apr 2006 21:09:43 +0200 Subject: [DB-SIG] how to know the connexion is closed In-Reply-To: References: Message-ID: <17485.8823.318107.617005@gargle.gargle.HOWL> William Dode wrote at 2006-4-22 18:53 +0000: >I search for a generic way (for psycopg, mysqldb, adodbapi) to >know that the connexion is closed I fear there is no generic way.... -- Dieter From patriciap.gu at gmail.com Mon Apr 24 22:37:37 2006 From: patriciap.gu at gmail.com (Patty) Date: Mon, 24 Apr 2006 20:37:37 +0000 (UTC) Subject: [DB-SIG] MySQLdb - syntax question References: Message-ID: Hi again, > query_string = "update mytable set " + ",".join ( [ key + "=%s" for > key, value in d_items ] ) > > cursor.execute ( query_string, tuple ( [ value for key, value in > d_items ] ) ) > Thanks for your prompt reply. I kinda got it, but the the part that I forgot to mention was that the values are not retrieved from the dictionary. What I retrieve from the dictionary are the column names and the target. The values are in a separate list, so how would it be in this case? I'm sorry if I sound silly, but I'm new to this. Thanks! Patty From chris at cogdon.org Mon Apr 24 22:58:26 2006 From: chris at cogdon.org (Chris Cogdon) Date: Mon, 24 Apr 2006 13:58:26 -0700 Subject: [DB-SIG] MySQLdb - syntax question In-Reply-To: References: Message-ID: On Apr 24, 2006, at 13:37, Patty wrote: > > Hi again, > >> query_string = "update mytable set " + ",".join ( [ key + "=%s" for >> key, value in d_items ] ) >> >> cursor.execute ( query_string, tuple ( [ value for key, value in >> d_items ] ) ) >> > > > Thanks for your prompt reply. I kinda got it, but the the part that I > forgot to > mention was that the values are not retrieved from the dictionary. > What I > retrieve from the dictionary are the column names and the target. The > values are > in a separate list, so how would it be in this case? I'm sorry if I > sound silly, > but I'm new to this. I don't know what you mean by 'target' :) But, that kind of fiddling is more in the python realm than the DB/mysql realm... ie, as far as the dbabi side is concerned, the values are passed in through a tuple, and connected to the query string with % marks, and the column names have to be manually inserted into the query string (which is what the + and join items did) because there's no dbapi support for manipulating non-literal-value entries in the query string. ie, you can't do this: colname = "column1" value = "John" cursor.execute ( "update mytable set %s=%s", ( colname, value ) ) Because that would result in this SQL: update mytable set "column1"="John"; # the mysql connector might use single quotes, I'm not sure And that's not what you wanted. Instead, if it was just ONE value, you'd do this: cursor.execute ( "update mytable set "+colname+"=%s", ( value, ) ) Would would send this SQL: update mytable set column1="John"; Now, my example above assumed all the column names, and values, came from one dictionary, and used join and list comprehensions to expand everything. They're just python tricks, not mysql or db-api tricks. -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From patriciap.gu at gmail.com Mon Apr 24 23:03:00 2006 From: patriciap.gu at gmail.com (Patty) Date: Mon, 24 Apr 2006 21:03:00 +0000 (UTC) Subject: [DB-SIG] MySQLdb - syntax question References: Message-ID: Got it!! Thank you!!!! From Chris.Clark at ingres.com Mon Apr 24 23:56:46 2006 From: Chris.Clark at ingres.com (Chris Clark) Date: Mon, 24 Apr 2006 14:56:46 -0700 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <444D1932.1020503@egenix.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> <17479.52784.244921.297972@gargle.gargle.HOWL> <444D1932.1020503@egenix.com> Message-ID: <444D499E.4030409@ingres.com> In reference to finding question marks in SQL queries and ignoring string literals M.-A. Lemburg wrote: > Does anyone know a good implementation of such a search&replace > mechanism ? > I've not seen any existing code to deal with this BUT there are a couple of places to check: * SnakeSQL * gadfly as they are pure Python database engines and so have some SQL parsing code in them already that maybe usable. But you can always cheat and iterate through the SQL string and just count (incrementing and decrementing) single quotes. You need to remember to deal with the special case of embedded single quotes (by keeping a mini back buffer) like this: 'it''s Monty Python' Chris From ianb at colorstudy.com Tue Apr 25 00:19:47 2006 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 24 Apr 2006 17:19:47 -0500 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <444D499E.4030409@ingres.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> <17479.52784.244921.297972@gargle.gargle.HOWL> <444D1932.1020503@egenix.com> <444D499E.4030409@ingres.com> Message-ID: <444D4F03.6070609@colorstudy.com> Chris Clark wrote: > In reference to finding question marks in SQL queries and ignoring > string literals > > M.-A. Lemburg wrote: >>Does anyone know a good implementation of such a search&replace >>mechanism ? >> > > I've not seen any existing code to deal with this BUT there are a couple > of places to check: > > * SnakeSQL > * gadfly > > > as they are pure Python database engines and so have some SQL parsing > code in them already that maybe usable. > > But you can always cheat and iterate through the SQL string and just > count (incrementing and decrementing) single quotes. > You need to remember to deal with the special case of embedded single > quotes (by keeping a mini back buffer) like this: > > 'it''s Monty Python' Postgres and MySQL allow 'it\'s Monty Python'. Parsing SQL for every query seems like a bit of a performance problem, especially because the parsing itself can be different for different databases. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From bill at celestial.net Tue Apr 25 01:12:32 2006 From: bill at celestial.net (Bill Campbell) Date: Mon, 24 Apr 2006 16:12:32 -0700 Subject: [DB-SIG] how to know the connexion is closed In-Reply-To: <17485.8823.318107.617005@gargle.gargle.HOWL> References: <17485.8823.318107.617005@gargle.gargle.HOWL> Message-ID: <20060424231232.GA76508@alexis.mi.celestial.com> On Mon, Apr 24, 2006, Dieter Maurer wrote: >William Dode wrote at 2006-4-22 18:53 +0000: >>I search for a generic way (for psycopg, mysqldb, adodbapi) to >>know that the connexion is closed > >I fear there is no generic way.... One could use something like: try: cur = conn.cursor() except: cur = None I would think that attempting to get a cursor from a closed connection would always raise an error. Bill -- INTERNET: bill at Celestial.COM Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX: (206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 ``Intellectually, teachers fall between education theorists and bright cocker spaniels. (Probably closer to the education theorists. The AKC has been doing wonders with spaniels.) If you think I'm kidding look at the GREs for education majors, whose scores are the lowest of all fields, and remember that these are the smart ones.'' -- http://www.FredOnEverything.net From wilk-ml at flibuste.net Tue Apr 25 09:24:56 2006 From: wilk-ml at flibuste.net (William Dode) Date: Tue, 25 Apr 2006 07:24:56 +0000 (UTC) Subject: [DB-SIG] how to know the connexion is closed References: <17485.8823.318107.617005@gargle.gargle.HOWL> Message-ID: On 24-04-2006, Dieter Maurer wrote: > William Dode wrote at 2006-4-22 18:53 +0000: >>I search for a generic way (for psycopg, mysqldb, adodbapi) to >>know that the connexion is closed > > I fear there is no generic way.... > ok... so, perhaps a solution for postgresql with psycopg ? -- William Dod? - http://flibuste.net From chris at cogdon.org Tue Apr 25 09:38:03 2006 From: chris at cogdon.org (Chris Cogdon) Date: Tue, 25 Apr 2006 00:38:03 -0700 Subject: [DB-SIG] how to know the connexion is closed In-Reply-To: References: <17485.8823.318107.617005@gargle.gargle.HOWL> Message-ID: On Apr 25, 2006, at 12:24 AM, William Dode wrote: > On 24-04-2006, Dieter Maurer wrote: >> William Dode wrote at 2006-4-22 18:53 +0000: >>> I search for a generic way (for psycopg, mysqldb, adodbapi) to >>> know that the connexion is closed >> >> I fear there is no generic way.... >> > > ok... so, perhaps a solution for postgresql with psycopg ? How about trying a statement that doesn't do anything, and if it fails with an exception, then it was closed? :) -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From wilk-ml at flibuste.net Tue Apr 25 11:00:17 2006 From: wilk-ml at flibuste.net (William Dode) Date: Tue, 25 Apr 2006 09:00:17 +0000 (UTC) Subject: [DB-SIG] how to know the connexion is closed References: <17485.8823.318107.617005@gargle.gargle.HOWL> <20060424231232.GA76508@alexis.mi.celestial.com> Message-ID: On 24-04-2006, Bill Campbell wrote: > On Mon, Apr 24, 2006, Dieter Maurer wrote: >>William Dode wrote at 2006-4-22 18:53 +0000: >>>I search for a generic way (for psycopg, mysqldb, adodbapi) to >>>know that the connexion is closed >> >>I fear there is no generic way.... > > One could use something like: > > try: cur = conn.cursor() > except: cur = None > > I would think that attempting to get a cursor from a closed > connection would always raise an error. Not with MySQLdb : >>> cx = MySQLdb.connect(db='scrabble') >>> cx.close() >>> cx.cursor() (and a query fail with InterfaceError) with sqlite it raise ProgrammingError >>> cx = sqlite.connect(':memory:') >>> cx.close() >>> cx.cursor() Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 552, in cursor self._checkNotClosed("cursor") File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 482, in _checkNotClosed raise _sqlite.ProgrammingError, \ _sqlite.ProgrammingError: cursor failed - Connection is closed. with psycopg it raise InterfaceError >>> import psycopg >>> cx = psycopg.connect('dbname=pytodo') >>> cx.close() >>> cx.cursor() Traceback (most recent call last): File "", line 1, in ? psycopg.InterfaceError: already closed Same problem if i try to close a connection already closed MySQLdb and sqlite raise ProgrammingError psycopg raise InterfaceError Don't you think it should be a spec in dbapi ? -- William Dod? - http://flibuste.net From mal at egenix.com Tue Apr 25 12:15:52 2006 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 25 Apr 2006 12:15:52 +0200 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: <444D4F03.6070609@colorstudy.com> References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com> <20060415120141.GA12186@yapgi.hopto.org> <1145139880.8861.9.camel@localhost.localdomain> <44421F81.4080702@halfcooked.com> <444363CA.2030304@halfcooked.com> <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com> <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com> <4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com> <1145450923.7406.27.camel@dot.uniqsys.com> <44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com> <44465961.1090904@ghaering.de> <1145463224.7406.83.camel@dot.uniqsys.com> <4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com> <17479.52784.244921.297972@gargle.gargle.HOWL> <444D1932.1020503@egenix.com> <444D499E.4030409@ingres.com> <444D4F03.6070609@colorstudy.com> Message-ID: <444DF6D8.7040608@egenix.com> Ian Bicking wrote: > Chris Clark wrote: >> In reference to finding question marks in SQL queries and ignoring >> string literals >> >> M.-A. Lemburg wrote: >>> Does anyone know a good implementation of such a search&replace >>> mechanism ? >>> >> I've not seen any existing code to deal with this BUT there are a couple >> of places to check: >> >> * SnakeSQL >> * gadfly >> >> >> as they are pure Python database engines and so have some SQL parsing >> code in them already that maybe usable. >> >> But you can always cheat and iterate through the SQL string and just >> count (incrementing and decrementing) single quotes. >> You need to remember to deal with the special case of embedded single >> quotes (by keeping a mini back buffer) like this: >> >> 'it''s Monty Python' > > Postgres and MySQL allow 'it\'s Monty Python'. Parsing SQL for every > query seems like a bit of a performance problem, especially because the > parsing itself can be different for different databases. I don't think that's much of a problem: each database module will have to implement whatever parsing is needed. In most cases, this will only have to be some counting of quotes to avoid replacing question marks inside SQL literals. You can fast track this by searching for a quote first: in many cases you won't need to bother with the more complicated SQL literal aware search&replace at all, if there are no literals in the SQL statement. Perhaps we could collect a few quote quoting rules for the various backends. So far we have: * '' - standard SQL quoted quote * \' - C style quoted quote We probably also need to look at different ways to write a SQL literal. Using single quotes is standard SQL, but there might also be some backends which use or support double quotes. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From andy47 at halfcooked.com Sun Apr 30 04:52:38 2006 From: andy47 at halfcooked.com (Andy Todd) Date: Sun, 30 Apr 2006 12:52:38 +1000 Subject: [DB-SIG] paramstyles (mysql string length?) In-Reply-To: References: Message-ID: <44542676.2060300@halfcooked.com> Dittmar, Daniel wrote: >> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote: > > But how should a module support multiple notations? > - by tokenizing any SQL statement to check which paramstyle was used > - by having an option on the connection or the cursor > - by having an additional method which requires qmark placeholders (this > one would be backwards compatible) > > > Daniel > As paramstyle is currently an attribute of the module perhaps it could be inherited by the connection. That way the user (if the module allows) could set the paramstyle on their connection to one of the available options. If the module only supports a single paramstyle then changing the attribute on the connection would have no effect. Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/