From nhodgson at ebsquared.com.au Mon Sep 1 12:03:57 2003 From: nhodgson at ebsquared.com.au (Neil Hodgson) Date: Sun Aug 31 21:04:07 2003 Subject: [DB-SIG] Switching databases Message-ID: <3F529AFD.5060008@ebsquared.com.au> We are currently using an Oracle 9i database through DCOracle2, directly importing DCOracle2 and sometimes referring to DCOracle2.dco2, the DCOracle2 time/date types, and DCOracle2.DatabaseError. In the future we may want to experiment with cx_Oracle or another database such as MySQL. Are there any good strategies for providing flexibility and avoiding mass code changes here? Neil From msanchez at grupoburke.com Mon Sep 1 08:21:29 2003 From: msanchez at grupoburke.com (Marcos =?ISO-8859-1?Q?S=E1nchez?= Provencio) Date: Mon Sep 1 03:21:30 2003 Subject: [DB-SIG] Switching databases In-Reply-To: <3F529AFD.5060008@ebsquared.com.au> References: <3F529AFD.5060008@ebsquared.com.au> Message-ID: <1062400930.657.4.camel@cynar.proteus> One must program his/her own db-api independent layer. Wait! We have debated this over and over! El lun, 01-09-2003 a las 03:03, Neil Hodgson escribi?: > We are currently using an Oracle 9i database through DCOracle2, > directly importing DCOracle2 and sometimes referring to DCOracle2.dco2, > the DCOracle2 time/date types, and DCOracle2.DatabaseError. In the > future we may want to experiment with cx_Oracle or another database such > as MySQL. > > Are there any good strategies for providing flexibility and avoiding > mass code changes here? > > Neil > > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- gpg --recv-keys --keyserver wwwkeys.pgp.net B9AD9B1B -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Esta parte del mensaje =?ISO-8859-1?Q?est=E1?= firmada digitalmente Url : http://mail.python.org/pipermail/db-sig/attachments/20030901/81b9baee/attachment.bin From ods at strana.ru Mon Sep 1 12:21:52 2003 From: ods at strana.ru (Denis S. Otkidach) Date: Mon Sep 1 03:22:27 2003 Subject: [DB-SIG] Switching databases In-Reply-To: <3F529AFD.5060008@ebsquared.com.au> Message-ID: On Mon, 1 Sep 2003, Neil Hodgson wrote: NH> We are currently using an Oracle 9i database through NH> DCOracle2, NH> directly importing DCOracle2 and sometimes referring to NH> DCOracle2.dco2, NH> the DCOracle2 time/date types, and DCOracle2.DatabaseError. NH> In the NH> future we may want to experiment with cx_Oracle or another NH> database such NH> as MySQL. NH> NH> Are there any good strategies for providing flexibility NH> and avoiding NH> mass code changes here? The only way I know: write your own adapters for desired databases and switch to them from DB API. There are two most known incompatibility issues with DB API: paramstyle and date/time conversion. -- Denis S. Otkidach http://www.python.ru/ [ru] From arjen.dijkstra at hccnet.nl Tue Sep 2 23:15:31 2003 From: arjen.dijkstra at hccnet.nl (Arjen Dijkstra) Date: Tue Sep 2 16:09:13 2003 Subject: [DB-SIG] Re: DB-SIG Digest, Vol 5, Issue 6 In-Reply-To: References: Message-ID: <20030902221531.0add172b.arjen.dijkstra@hccnet.nl> On Sun, 31 Aug 2003 21:04:12 -0400 db-sig-request@python.org wrote: > Send DB-SIG mailing list submissions to > db-sig@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/db-sig > or, via email, send a message with subject or body 'help' to > db-sig-request@python.org > > You can reach the person managing the list at > db-sig-owner@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of DB-SIG digest..." > From nhodgson at ebsquared.com.au Thu Sep 4 13:42:16 2003 From: nhodgson at ebsquared.com.au (Neil Hodgson) Date: Thu Sep 4 17:06:54 2003 Subject: [DB-SIG] sysdba through DCOracle2 Message-ID: <3F56A688.5090907@ebsquared.com.au> Hi, Thanks for the replies to my previous question. As well as coding database use in Python through DCOracle2, I would like to perform user account management this way. With sqlplus it is possible to log in from the command line as sysdba with sqlplus "sys/password as sysdba" There doesn't appear to be a way to do this through DCOracle2.connect as it attempts to parse the connection string that it expects to look like "scott/tiger@orac". Does anyone know a way to connect as sysdba through DCOracle2? Neil From greg at thinksql.co.uk Thu Sep 4 01:57:17 2003 From: greg at thinksql.co.uk (Greg Gaughan) Date: Thu Sep 4 18:34:45 2003 Subject: [DB-SIG] ANN: Pure Python DB-API module for ThinkSQL Message-ID: <003601c37277$1bba1340$107b7b7b@FARADAY> We have just released an initial version of a pure Python DB-API 2.0 compliant module for the ThinkSQL RDBMS. It can be downloaded from the News section of our site at http://www.thinksql.co.uk. ThinkSQL is a powerful, cross-platform, multi-threaded relational database management system. It supports Core ISO SQL, transactions, sub-selects, views, stored procedures, functions, comprehensive constraints, large objects, multi-version concurrency control, on-line backups, and a statistical optimiser that uses constraints and relationships to improve plans. The SQL server is simple to install and bloat-free. It runs under Windows and Linux and includes native ODBC, dbExpress (Delphi/Kylix), JDBC and Python drivers. Regards, Greg Gaughan ThinkSQL Ltd From james at digconn.co.uk Tue Sep 9 11:18:18 2003 From: james at digconn.co.uk (james@digconn.co.uk) Date: Tue Sep 9 15:09:06 2003 Subject: [DB-SIG] help Message-ID: Can someone please point me in the direction of where i can find the modules to make my postgresql database interact with my python script. I'm running python on windows so i need to find the right setup files as well. Can someone also poine me in the direction of some documentation for the interaction between python and postgresql databases thankyou -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20030909/34be37a6/attachment.htm From ianb at colorstudy.com Tue Sep 9 15:14:54 2003 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Sep 9 15:14:58 2003 Subject: [DB-SIG] help In-Reply-To: Message-ID: On Tuesday, September 9, 2003, at 10:18 AM, wrote: > Can someone please point me in the direction of where i can find the > modules to make my postgresql database interact with my python script. > I'm running python on windows so i need to find the right setup files > as well. Can someone also poine me in the direction of some > documentation for the interaction between python and postgresql > databases > You want to use a package called psycopg (there's also pygresql and popy which do equivalent things, but I think psycopg is probably your best bet) Ian From pythontutor at venix.com Tue Sep 9 16:16:25 2003 From: pythontutor at venix.com (Lloyd Kvam) Date: Tue Sep 9 15:16:33 2003 Subject: [DB-SIG] help In-Reply-To: References: Message-ID: <3F5E2709.6060600@venix.com> http://www.python.org/topics/database/ Database Topic Guide In particular the link to the DB-API spec v2.0 will explain how things work. The link to Database Modules will list available modules. I do NOT know the best choice for Windows. james@digconn.co.uk wrote: > Can someone please point me in the direction of where i can find the > modules to make my postgresql database interact with my python script. > I'm running python on windows so i need to find the right setup files as > well. Can someone also poine me in the direction of some documentation > for the interaction between python and postgresql databases > > thankyou > > > ------------------------------------------------------------------------ > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-443-6155 fax: 801-459-9582 From gupt_vive at rediffmail.com Wed Sep 10 06:23:49 2003 From: gupt_vive at rediffmail.com (vivek kumar gupta) Date: Wed Sep 10 01:23:51 2003 Subject: [DB-SIG] help Message-ID: <20030910052227.21276.qmail@webmail7.rediffmail.com> On Tue, 09 Sep 2003 james@digconn.co.uk wrote : >Can someone please point me in the direction of where i can find >the >modules to make my postgresql database interact with my python >script. >I'm running python on windows so i need to find the right setup >files >as well. Can someone also poine me in the direction of some >documentation for the interaction between python and postgresql >databases I am using pyPgSQL on Windows 2000 and it works very nice. Documentation can be viewsed using pydoc utility ( Module docs in your start menu ). Some FAQs are also there on the site. http://pypgsql.sourceforge.net regards Vivek Kumar ___________________________________________________ Meet your old school or college friends from 1 Million + database... Click here to reunite www.batchmates.com/rediff.asp From james at digconn.co.uk Wed Sep 10 09:20:17 2003 From: james at digconn.co.uk (james@digconn.co.uk) Date: Wed Sep 10 04:20:18 2003 Subject: [DB-SIG] Again Message-ID: can someone please tell me what i'm mean't to do when i get this message: My code leading up to this is: db = psycopg.connect('host=localhost dbname=dataname user=postgres') cursor=db.cursor() cursor.execute('SELECT * FROM "RingInfo"') cursor.fetchall All i'm trying to do at the moment is see if i can see data in my postgresql database and i still don't know if the actual database and python code are interacting. thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20030910/f01440c1/attachment.htm From eivindt-db-sig at multinet.no Wed Sep 10 11:25:51 2003 From: eivindt-db-sig at multinet.no (Eivind Tagseth) Date: Wed Sep 10 04:25:59 2003 Subject: [DB-SIG] Again In-Reply-To: References: Message-ID: <20030910082551.GB2024@tagseth-trd.consultit.no> * james@digconn.co.uk [030910 08:21]: > can someone please tell me what i'm mean't to do when i get this message: > > My code leading up to this is: > > db = psycopg.connect('host=localhost dbname=dataname user=postgres') > cursor=db.cursor() > cursor.execute('SELECT * FROM "RingInfo"') > cursor.fetchall You're printing the method, no calling it. Try cursor.fetchall() Eivind From andy47 at halfcooked.com Wed Sep 10 10:35:14 2003 From: andy47 at halfcooked.com (Andy Todd) Date: Wed Sep 10 04:38:24 2003 Subject: [DB-SIG] Again In-Reply-To: <20030910082551.GB2024@tagseth-trd.consultit.no> References: <20030910082551.GB2024@tagseth-trd.consultit.no> Message-ID: <3F5EE242.7000007@halfcooked.com> Eivind Tagseth wrote: > * james@digconn.co.uk [030910 08:21]: > >>can someone please tell me what i'm mean't to do when i get this message: >> >>My code leading up to this is: >> >>db = psycopg.connect('host=localhost dbname=dataname user=postgres') >>cursor=db.cursor() >>cursor.execute('SELECT * FROM "RingInfo"') >>cursor.fetchall > > > You're printing the method, no calling it. Try cursor.fetchall() > > > > Eivind > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig And whilst you are at it, assign the results of the call to something, otherwise you will lose them. A simple approach is; >>> cursor.execute('') >>> results=cursor.fetchall() >>> print "I have %d rows" % len(results) >>> for row in results: >>> print row Or just fetch a single row if you're simply testing, replace the call to 'fetchall' with 'fetchone'. Regardless, you should experiment in the Python interpreter as its a very forgiving environment - and perhaps brush up on your Python knowledge with one of the excellent tutorials that are available on-line (http://www.python.org/topics/learn/). Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From sibel14324 at gmx.de Sat Sep 13 20:00:46 2003 From: sibel14324 at gmx.de (Sibel Unlu) Date: Sat Sep 13 19:52:49 2003 Subject: [DB-SIG] Sibel aus =?iso-8859-1?q?G=F6ttingen?= Uni Message-ID: <200309132352.h8DNqdm04836@p15093686.pureserver.info> Hi ich bin die Sibel 19 Jahre alt . Selam ben Sibel 19 Yasindayim Komme aus dem Wunderbaren S?den der T?rkei. T?rkiyenin Bursa Sehirinden geliyorum Z.z studiere ich in G?ttingen Medizin und suche auf diesem Wege eine Beziehung die sich Rein sexuell dienen soll. Kisa bir zamandan beri G?ttingen,de Universit?t Okuyorum Tip b?l?m?nde , ve sadece Sexuell iliski icin Arkadaslik ariyorum (?ber mich: 19 Yasinda T?rkin T?rk Hobby : Lesen, Reiten, Kino, Chaten, Sex Treffs Hobilerim: Kitap okumak, Ata binmek, Conema, Catlesmek, Erotik Bulusmalar Wenn du mit Kontaktieren willst, Klicks du hier oder auf die Blau Markierte Schriften Benimle Bulusmak icin buraya ve mavi yazilara tikla Selamlar Atesli Kiz Bussi bis dann -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20030914/b78ddd5c/attachment.html From stedew2 at netzero.com Tue Sep 16 17:23:04 2003 From: stedew2 at netzero.com (Steve Dewey) Date: Tue Sep 16 17:14:52 2003 Subject: [DB-SIG] UPDATE from SELECT? Message-ID: <001a01c37c98$bde9df80$0600a8c0@vinnie> I am writing a class to provide the following services: - execution of a db query - local storage of result set - navigation of the result set - field level data access by column name - storage of record modifications with incremental rollback - application of changes back to database, initially per record but later possibly in batch mode - insertion into result set with subsequent insertion into database - deletion from result set with subsequent deletion from database Initially I am restricting the target database to MySQL. I am using MySQLdb. I am also using the db_row module for data storage. I have implemented the first five services on my list ( actually the rollback is not yet implemented but the necessary data structure is there, I just need to write the rollback function.) I am now working on the sixth item, "application of changes", and I realise I have some choices to make. To this point the only metadata my class has required is the query and the name of the database to be queried. I am retrieving the column names from the cursor.description attribute. Now I need an UPDATE statement and it appears to me that it must be built up. I have the column names and the column values, but the table name is embedded in the string representing the query. So my question is : Should I attempt to parse the query string within my class for the table name and any other tokens I need to build my UPDATE statement or should I pass the individual elements in and build my SELECT and UPDATE statements( and later on my INSERT and DELETE statements) inside my class? Is there any Python code out there that will derive an UPDATE or INSERT from a SELECT? If I pass the individual elements into my class (i.e. table name, where clause, order by columns, ...) what are the necessary elements? How far down do I have to break my syntax? Note that I am restricting my query to those which are updatable (i.e. one table, no aggregate functions, no DISTINCT, no GROUP BY or HAVING, ...) Finally, am I reinventing the wheel? I'm certain this has been done before, but I haven't found any open source Python code out there that does what I'm trying to do. Bits and pieces yes (e.g. db_row), but not the complete package. From arkamir at softhome.net Tue Sep 16 19:20:24 2003 From: arkamir at softhome.net (Conrad Koziol) Date: Tue Sep 16 19:20:30 2003 Subject: [DB-SIG] Newb Quiestion Message-ID: <1063754424.5539.3.camel@quercus.home.net> Hello i'm new to programming and was wondering if you can suggest a free database. Also any place to find a tutorial on using it with python and the modules you need to use it??? I'm trying to make a forum for you information. Thanks conrad ps. I sent this message once before but i'v been having problems with my email account and I'm not sure the message was sent. Anyways i havent recieved any db-sig digests in a while From bryan at bryangudorf.com Wed Sep 17 00:16:30 2003 From: bryan at bryangudorf.com (Bryan J Gudorf) Date: Wed Sep 17 00:17:57 2003 Subject: [DB-SIG] Python Database Objects (PDO) Message-ID: Hey Everyone, I just finished writing a tool called Python Database Objects (PDO) that acts like a wrapper class over DB-API 2.0 based modules, giving them a giving them a common object oriented api. It was inspired by ADO, JDBC and other object oriented APIs.In theory, it should make switching between databases and deployment of database driven applications easier. PDO is being released under a BSD License by my company, NeuroKode Labs, and is available at: http://pdo.neurokode.com. We've created a sourceforge project, and are opening development of this module to the python community. PDO has also been registered with the Python Packages Index. Right now PDO supports basic database functionality: Select, Update, Insert, Delete through two Objects: ResultSet Object and Simple Execute Heres a really quick example: import pdo DBConn = pdo.connect('Module=MySQLdb;User=SalesReader;Passwd=secretpassword;DB=Sales' ) Results = DBConn.openRS("SELECT * FROM Customers") while Results.next() print "Name: " + Results.fields['Name'].value ... It supports MySQL and SQLite currently, but adding support for other DB-API modules should be a snap. Its usable already, but we'll be adding more features (such as a complex execute with query paramters, and editable recordsets) as we get time. Thoughts, opinions? ~Bryan J Gudorf NeuroKode Labs, LLC From matt at pollenation.net Wed Sep 17 04:53:33 2003 From: matt at pollenation.net (Matt Goodall) Date: Wed Sep 17 04:36:05 2003 Subject: [DB-SIG] Newb Quiestion In-Reply-To: <1063754424.5539.3.camel@quercus.home.net> References: <1063754424.5539.3.camel@quercus.home.net> Message-ID: <3F68210D.1090303@pollenation.net> Conrad Koziol wrote: >Hello i'm new to programming and was wondering if you can suggest a free >database. > It really depends on your requirements but PostgreSQL (http://www.postgresql.org/) is my preferred solution. MySQL (http://www.mysql.com/) is another popular choice. >Also any place to find a tutorial on using it with python and >the modules you need to use it??? > You will almost certainly be accessing the database via the DB-API interface which is quite straightforward: http://python.org/peps/pep-0249.html I don't have any good tutorial links, have you tried Google? A full list of modules is available here: http://python.org/topics/database/modules.html I have personally used pyscopg and pyPgSQL (with PostgreSQL) and both work well. They are licensed differently which may influence your choice. Hope this helps. Cheers, Matt -- Matt Goodall, Pollenation Internet Ltd w: http://www.pollenationinternet.com e: matt@pollenation.net t: +44 (0)113 2252500 From msanchez at grupoburke.com Wed Sep 17 10:48:06 2003 From: msanchez at grupoburke.com (Marcos =?ISO-8859-1?Q?S=E1nchez?= Provencio) Date: Wed Sep 17 10:45:59 2003 Subject: [DB-SIG] Newb Quiestion In-Reply-To: <1063754424.5539.3.camel@quercus.home.net> References: <1063754424.5539.3.camel@quercus.home.net> Message-ID: <1063810085.2963.1.camel@cynar.proteus> For a minimal learning set, try sqlite or gadfly. They don't need a separate server, acting in a manner that is similar to MS Access or dBase. El mi?, 17-09-2003 a las 01:20, Conrad Koziol escribi?: > Hello i'm new to programming and was wondering if you can suggest a free > database. Also any place to find a tutorial on using it with python and > the modules you need to use it??? I'm trying to make a forum for you > information. > > Thanks > conrad > > ps. I sent this message once before but i'v been having problems with my > email account and I'm not sure the message was sent. Anyways i havent > recieved any db-sig digests in a while > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- gpg --recv-keys --keyserver wwwkeys.pgp.net B9AD9B1B -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Esta parte del mensaje =?ISO-8859-1?Q?est=E1?= firmada digitalmente Url : http://mail.python.org/pipermail/db-sig/attachments/20030917/889de77f/attachment.bin From nkomcgysn at bigfoot.com Mon Sep 22 15:39:13 2003 From: nkomcgysn at bigfoot.com (Kip Waters) Date: Sun Sep 21 22:35:46 2003 Subject: [DB-SIG] new Sexual attractant jqnyqyywchayyla Message-ID: <5ffr4pc-j0v8s-$nta--$t$s32k-19@lsf3qt1.go1> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20030922/22528443/attachment.html From wnvcole at peppermillcas.com Mon Sep 22 17:22:02 2003 From: wnvcole at peppermillcas.com (Vernon Cole) Date: Mon Sep 22 17:22:13 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? Message-ID: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> I have been away from the leading edge for a long time, and am trying to jump back in and learn the new stuff. Python seems like a GREAT tool. But..... From: ...PEP: 249 ...Title: Python Database API Specification v2.0 ...Version: $Revision: 1.9 $ I extract the following: vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv Question: How can I construct a dictionary out of the tuples returned by .fetchxxx(): Answer: There are several existing tools available which provide helpers for this task. Most of them use the approach of using the column names defined in the cursor attribute .description as basis for the keys in the row dictionary. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nice to hear that there are "several existing tools". I have been searching for a week now, but cannot find any of them. Would someone be so kind as to actually publish a LIST of such tools?? I find it difficult to believe that the "latest and greatest" tools available in the 21st century still cannot return a value from a named field (column) in one row of a database. Good grief! How can I NOT have a feature on a 256 MB Pentium in 2003 that I had on a 30 KB PDP-11 in 1983? Was RDM _that_ far ahead of its time? -------------- Vernon Cole once (a long time ago) a developer of RDM. wnvcole AT nospam.peppermillcas.com From TKeating at origin.ea.com Mon Sep 22 17:40:33 2003 From: TKeating at origin.ea.com (Keating, Tim) Date: Mon Sep 22 17:40:46 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can Iconstruct a dictionary??? Message-ID: <25BCE2B1149CDA4E9758CC08E70A20A9FD64F3@eahq-mb1> It's not terribly hard to do by hand, which is probably why existing modules that do it get little attention. Here are three that I found: http://opensource.theopalgroup.com/ (See Python Database Row Module, partway down the page) http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81252 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/163605 TK -----Original Message----- From: Vernon Cole [mailto:wnvcole@peppermillcas.com] Sent: Monday, September 22, 2003 4:22 PM To: 'db-sig@python.org' Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can Iconstruct a dictionary??? I have been away from the leading edge for a long time, and am trying to jump back in and learn the new stuff. Python seems like a GREAT tool. But..... From: ...PEP: 249 ...Title: Python Database API Specification v2.0 ...Version: $Revision: 1.9 $ I extract the following: vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv Question: How can I construct a dictionary out of the tuples returned by .fetchxxx(): Answer: There are several existing tools available which provide helpers for this task. Most of them use the approach of using the column names defined in the cursor attribute .description as basis for the keys in the row dictionary. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nice to hear that there are "several existing tools". I have been searching for a week now, but cannot find any of them. Would someone be so kind as to actually publish a LIST of such tools?? I find it difficult to believe that the "latest and greatest" tools available in the 21st century still cannot return a value from a named field (column) in one row of a database. Good grief! How can I NOT have a feature on a 256 MB Pentium in 2003 that I had on a 30 KB PDP-11 in 1983? Was RDM _that_ far ahead of its time? -------------- Vernon Cole once (a long time ago) a developer of RDM. wnvcole AT nospam.peppermillcas.com _______________________________________________ DB-SIG maillist - DB-SIG@python.org http://mail.python.org/mailman/listinfo/db-sig From morillas at posta.unizar.es Mon Sep 22 16:47:48 2003 From: morillas at posta.unizar.es (luis miguel morillas) Date: Mon Sep 22 17:41:10 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> References: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> Message-ID: <20030922204747.GA2616@marmota> Asunto: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? Fecha: lun, sep 22, 2003 at 02:22:02 -0700 Citando a Vernon Cole (wnvcole@peppermillcas.com): > I have been away from the leading edge for a long time, and am trying to > jump back > in and learn the new stuff. Python seems like a GREAT tool. > But..... > > From: > ...PEP: 249 > ...Title: Python Database API Specification v2.0 > ...Version: $Revision: 1.9 $ > I extract the following: > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > Question: > > How can I construct a dictionary out of the tuples returned by > .fetchxxx(): > > Answer: > > There are several existing tools available which provide > helpers for this task. Most of them use the approach of using > the column names defined in the cursor attribute .description > as basis for the keys in the row dictionary. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Nice to hear that there are "several existing tools". > > I have been searching for a week now, but cannot find any of them. > > Would someone be so kind as to actually publish a LIST of such tools?? > Perhaps Python Database Row Module Version 0.71 (work in progress -- check back often for updates) This Python 2.2 module defines light-weight objects which allow very flexible access to a fixed number of positional and named attributes via several interfaces. Or, more simply, these objects are a better way of returning the results of database queries, since they allow effcient access to fields by name or by index. It uses some of the new features of the Python 2.2 class system, and provide a nice demonstration of how to take advantage of them. http://opensource.theopalgroup.com/ > I find it difficult to believe that the "latest and greatest" tools > available > in the 21st century still cannot return a value from a named field (column) > in one row of a database. Good grief! How can I NOT have a feature on a 256 > MB Pentium in 2003 that I had > on a 30 KB PDP-11 in 1983? > Was RDM _that_ far ahead of its time? > -------------- > Vernon Cole > once (a long time ago) a developer of RDM. > wnvcole AT nospam.peppermillcas.com > > > > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- Luis Miguel # Por un mundo con conocimiento libre # No a las patentes de software http://www.zaralinux.org - http://www.hispalinux.es http://www.augustux.org From chris at cogdon.org Mon Sep 22 17:41:21 2003 From: chris at cogdon.org (Chris Cogdon) Date: Mon Sep 22 17:41:25 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> Message-ID: <81C39BE4-ED45-11D7-9A81-000A95E3823E@cogdon.org> On Monday, Sep 22, 2003, at 14:22 US/Pacific, Vernon Cole wrote: > I have been away from the leading edge for a long time, and am trying > to > jump back > in and learn the new stuff. Python seems like a GREAT tool. > But..... > > From: > ...PEP: 249 > ...Title: Python Database API Specification v2.0 > ...Version: $Revision: 1.9 $ > I extract the following: > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > Question: > > How can I construct a dictionary out of the tuples returned by > .fetchxxx(): > > Answer: > > There are several existing tools available which provide > helpers for this task. Most of them use the approach of using > the column names defined in the cursor attribute .description > as basis for the keys in the row dictionary. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Nice to hear that there are "several existing tools". > > I have been searching for a week now, but cannot find any of them. > > Would someone be so kind as to actually publish a LIST of such tools?? > > I find it difficult to believe that the "latest and greatest" tools > available > in the 21st century still cannot return a value from a named field > (column) > in one row of a database. Good grief! How can I NOT have a feature on > a 256 > MB Pentium in 2003 that I had > on a 30 KB PDP-11 in 1983? > Was RDM _that_ far ahead of its time? cursor = db.cursor () cursor. execute ( 'some query' ) a_row = cursor.fetchone () d = {} for descr, val in zip ( cur.description, a_row ): d[descr[0]] = val # dictionary 'd' now has the answers you want # the format of 'description' may vary from database type to type, unfortunately, but the above # works for pyPgSQL del sarcasm # :) -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From chris at cogdon.org Mon Sep 22 17:47:18 2003 From: chris at cogdon.org (Chris Cogdon) Date: Mon Sep 22 17:47:23 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> Message-ID: <569CF8C8-ED46-11D7-9A81-000A95E3823E@cogdon.org> Actually... some connectors do the assignment for you, such as pyPgSQL: row = cursor.fetchone() print row['somecolumnname'] The downside of this is that pyPgSQL takes a BIG performance hit because of all the, possibly unnecessary, dictionary assignments before it returns the values. Search the db-sig archives for a performance matrix I posted. -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From lkvam at venix.com Mon Sep 22 19:06:50 2003 From: lkvam at venix.com (Lloyd Kvam) Date: Mon Sep 22 19:06:55 2003 Subject: [DB-SIG] May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> References: <07AF3C77A0FBD311A99F00508B65203903272B75@sebastian.peppermillcas.com> Message-ID: <3F6F808A.1050503@venix.com> MySQLdb in the cursor.py module includes: class CursorDictRowsMixIn: """This is a MixIn class that causes all rows to be returned as dictionaries. This is a non-standard feature.""" _fetch_type = 1 The documentation explains how to make this mixin the default. I have always just created a dictionary when I needed it. For reporting and general searching by eye, I have found it is easier and quicker to deal with lists of tuple values rather than lists of dictionaries. Vernon Cole wrote: > I have been away from the leading edge for a long time, and am trying to > jump back > in and learn the new stuff. Python seems like a GREAT tool. > But..... > > From: > ...PEP: 249 > ...Title: Python Database API Specification v2.0 > ...Version: $Revision: 1.9 $ > I extract the following: > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > Question: > > How can I construct a dictionary out of the tuples returned by > .fetchxxx(): > > Answer: > > There are several existing tools available which provide > helpers for this task. Most of them use the approach of using > the column names defined in the cursor attribute .description > as basis for the keys in the row dictionary. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Nice to hear that there are "several existing tools". > > I have been searching for a week now, but cannot find any of them. > > Would someone be so kind as to actually publish a LIST of such tools?? > > I find it difficult to believe that the "latest and greatest" tools > available > in the 21st century still cannot return a value from a named field (column) > in one row of a database. Good grief! How can I NOT have a feature on a 256 > MB Pentium in 2003 that I had > on a 30 KB PDP-11 in 1983? > Was RDM _that_ far ahead of its time? > -------------- > Vernon Cole > once (a long time ago) a developer of RDM. > wnvcole AT nospam.peppermillcas.com > > > > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig > -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-443-6155 fax: 801-459-9582 From jfranz at neurokodeXREMOVEX.com Mon Sep 22 20:19:28 2003 From: jfranz at neurokodeXREMOVEX.com (Jon Franz_antispam) Date: Mon Sep 22 20:15:24 2003 Subject: [DB-SIG] Re: May we get an actual ANSWER to the F.A.Q...How can I construct a dictionary??? Message-ID: <002c01c38168$5ada3e60$7401a8c0@voidmk9> > Would someone be so kind as to actually publish a LIST of such tools?? Unfortunately, I think you've fallen victim to some of the hubris present in the SIG. Many of the people here seem perfectly fine with the status quo. Perhaps they worked on DB-API specifications, and thus feel protective of it? I know for a fact that many solutions to your problem exist, but the DB-SIG resource pages on python.org do not list them. > It's not terribly hard to do by hand, which is probably why existing > modules that do it get little attention. I think perhaps it has more to do with the attitudes of those involved in the SIG, to be honest. I've watched this list for quite a while, and although helpful, many people seem to shun the idea of providing higher level interfaces in the DB-API, such as access-by-column-name. As long as we're listing modules that provide this, I might as well throw my company's hat in the ring: http://pdo.neurokode.com It works, and is being updated to support more database modules, and support more features. ~Jon Franz NeuroKode Labs, LLC From daniel.dittmar at sap.com Tue Sep 23 05:07:39 2003 From: daniel.dittmar at sap.com (Dittmar, Daniel) Date: Tue Sep 23 05:37:29 2003 Subject: [DB-SIG] Re: May we get an actual ANSWER to the F.A.Q...How c an I construct a dictionary??? Message-ID: > > I think perhaps it has more to do with the attitudes of those > involved in > the > SIG, to be honest. I've watched this list for quite a while, > and although > helpful, many people seem to shun the idea of providing higher level > interfaces in the DB-API, such as access-by-column-name. - there are those driver authors who don't want to have to add something to their driver when perfectly good solutions exists as wrappers, especially when the driver is written in C - there are those driver authors who have implemented such a feature, but are afraid that the new API might be incompatible with their existing code - there are those driver users who have their own abstraction layer anyway - there are those nitpickers who say that column names aren't necessarily unique and that the uppercase/lowercase issues make it worse Daniel -- Daniel Dittmar SAP DB, SAP Labs Berlin daniel.dittmar@sap.com http://www.sapdb.org/ From msajec at tqs.com Tue Sep 23 11:32:55 2003 From: msajec at tqs.com (Sajec, Mike TQO) Date: Tue Sep 23 11:38:02 2003 Subject: [DB-SIG] informixdb module on python 2.2? Message-ID: <2E5FEA6E71B22B46BBD13D126F71920903760D72@STARBURST.tqs.com> Hello Python DB Experts! If anyone can help with this I will very much appreciate it! We're interested in using python at work to connect to one of our informix databases. Unfortunately, we're having trouble compiling the informixdb-1.3 module under python 2.2? It seems the absence of makefile.pre.in in python 2.2 causes the break under py2.2. Is there an easy way around this. I've heard rumors of a manual method, but have been unable to track down details. Again, any help is very much appreciated! Thanks in advance, Mike From bryan at bryangudorf.com Tue Sep 23 11:51:25 2003 From: bryan at bryangudorf.com (Bryan J Gudorf) Date: Tue Sep 23 11:51:30 2003 Subject: [DB-SIG] PDO 1.0.1 Released. Message-ID: Hey Guys, We just got PDO 1.0.1 finished and released on our Source Forge Project. It adds support for PostgreSQL via the pgdb module. Below is the news item from SourceForge: "Python Database Objects version 1.0.1 has been released. This incremental update adds support for the PostgreSQL via the pgdb module. PDO is a python module providing a common object oriented api across mutliple RDBMS. Similar in scope to ADO and JDBC, PDO provides column access by name, forward and backward movement within a resultset, and much more." Links: Project Homepage: http://pdo.neurokode.com Download Links: http://www.sourceforge.net/project/showfiles.php?group_id=86244 Bryan J Gudorf NeuroKode Labs, LLC http://www.neurokode.com From magnus at thinkware.se Thu Sep 25 09:52:35 2003 From: magnus at thinkware.se (Magnus Lycka) Date: Thu Sep 25 09:52:40 2003 Subject: =?ISO-8859-1?B?UkU6IFtEQi1TSUddIFJlOiBNYXkgd2UgZ2V0IGFuIGFjdHVhbCBBTlNXRVIgdG8gdGhlIEYuQS5RLi4uSG93IGNhbiBJIGNvbnN0cnVjdCBhIGRpY3Rpb25hcnk/Pz8=?= Message-ID: From: "Dittmar, Daniel" > - there are those nitpickers who say that column names aren't > necessarily unique and that the uppercase/lowercase issues make it worse Heh, that's funny. Never thought of that, but it is certainly true for DB2. I just tried Select CURRENT DATE AS A, 45 AS A from SYSIBM.SYSDUMMY1 I got A |A ----------+-- 2003-09-25|45 How odd! I had assumed that this would be an error. On the other hand, I don't really see why anyone would like to do that, and I could accept that a tuple -> dict tool would fail silently on something like this. I haven't tried this with db_row etc... -- Magnus Lycka, Thinkware AB Alvans vag 99, SE-907 50 UMEA, SWEDEN phone: int+46 70 582 80 65, fax: int+46 70 612 80 65 http://www.thinkware.se/ mailto:magnus@thinkware.se From jacobs at penguin.theopalgroup.com Thu Sep 25 10:27:33 2003 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Thu Sep 25 10:27:37 2003 Subject: [DB-SIG] Re: column names and uniqueness In-Reply-To: Message-ID: On Thu, 25 Sep 2003, Magnus Lycka wrote: > From: "Dittmar, Daniel" > > - there are those nitpickers who say that column names aren't > > necessarily unique and that the uppercase/lowercase issues make it worse > > Heh, that's funny. Never thought of that, but it is > certainly true for DB2. I just tried > > Select CURRENT DATE AS A, 45 AS A from SYSIBM.SYSDUMMY1 > > I got > > A |A > ----------+-- > 2003-09-25|45 > > How odd! I had assumed that this would be an error. On the > other hand, I don't really see why anyone would like to do > that, and I could accept that a tuple -> dict tool would > fail silently on something like this. I haven't tried this > with db_row etc... db_row has a strict uniqueness check, so it will not allow this. It will also catch this with "A" and "a" in case-insenstivie db_rows. It hasn't caused anyone problems to date, though I'm still considering adding a feature to optionally assign unique suffixes to make such names unique. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From daniel.dittmar at sap.com Thu Sep 25 10:53:09 2003 From: daniel.dittmar at sap.com (Dittmar, Daniel) Date: Thu Sep 25 10:53:53 2003 Subject: [DB-SIG] Re: May we get an actual ANSWER to the F.A.Q...How c an I construct a dictionary??? Message-ID: > > - there are those nitpickers who say that column names aren't > > necessarily unique and that the uppercase/lowercase issues > make it worse > > Heh, that's funny. Never thought of that, but it is > certainly true for DB2. I just tried [...] > How odd! I had assumed that this would be an error. On the > other hand, I don't really see why anyone would like to do that This mostly happens when joining tables and similar named columns appear in both tables. SAP DB did complain, but we had to become more lenient as applications that were ported from other databases were actually generating such result sets. > that, and I could accept that a tuple -> dict tool would > fail silently on something like this. I haven't tried this > with db_row etc... One more implementation defined feature (how to resolve duplicate column names) probably won't make that much of a difference. Daniel -- Daniel Dittmar SAP DB, SAP Labs Berlin daniel.dittmar@sap.com http://www.sapdb.org/ From jfranz at neurokode.com Thu Sep 25 12:12:34 2003 From: jfranz at neurokode.com (Jon Franz) Date: Thu Sep 25 12:08:43 2003 Subject: [DB-SIG] Re: May we get an actual ANSWER to the F.A.Q...How c an I construct a dictionary??? Message-ID: <031701c3837f$d93b8e30$7401a8c0@voidmk9> > How odd! I had assumed that this would be an error. On the > other hand, I don't really see why anyone would like to do > that, and I could accept that a tuple -> dict tool would > fail silently on something like this. I haven't tried this > with db_row etc... I know from experience that ADO silently fails on this - the last column with a particular name prevents access to the other same-name columns, except by index. This sort of thing can happen unexpectedly when you select * from multiple tables with some common column names - but that is bad practice anyway. I think a silent failure is a Bad Idea(TM) - but which method would be acceptable to the community: - throw an exception - allow access to the other same-name columns via a suffix/renaming scheme ('A:1', 'A:2', ...) At first blush, I like throwing an exception. The second option is semi-useful, but the name mangling would need to be Well documented and standardized. Unless a user had read deep into the docs, mangling would be unexpected and possibly painful. ~Jon Franz NeuroKode Labs, LLC From plb at iotk.com Thu Sep 25 12:52:24 2003 From: plb at iotk.com (Peter L. Buschman) Date: Thu Sep 25 12:52:34 2003 Subject: [DB-SIG] Inheriting from a DBAPI 2.0 Cursor Object Message-ID: <5.2.1.1.2.20030925183547.02dfabd0@localhost> Maybe I just haven't been programming in Python long enough, but this has me stumped... I am writing a DBAPI 2.0 compliant database abstraction layer that wraps most of the available DBAPI 2.0 drivers for Python and presents a consistent paramstyle, connection parameters, etc., to the end-user. So far, I've been successful in importing my driver object and connection method, but the connection and cursor objects are just passed-through from the underlying DBAPI driver. The next thing I want to do is override the execute() method of the cursor object so that it passes the sql statement and parameters through a function that converts from the abstraction layer's paramstyle to the underlying driver's paramstyle, but I'm not quite sure how to do this. Here is what an example looks like today: import iotk.dal as dal # Database Abstraction Layer driver = dal.driver.new('mxodbc') # Or mysqldb, psycopg, whatever... conn = driver.connect( dsn='foo' ) cursor = conn.cursor() cursor.execute('SELECT * FROM sometable' WHERE SOMETHING = ?', ( 1 ) ) My connect() function is simply a wrapper around the underlying driver's connect method that is smart enough to know what driver it is talking to and what parameters to use. The object returned is simply one of that driver's connection objects. How would I best define my own connection and cursor objects such that I can inherit from the underlying driver but also override those few methods like execute to make the driver access as transparent as possible? --PLB From chris at cogdon.org Thu Sep 25 13:56:54 2003 From: chris at cogdon.org (Chris Cogdon) Date: Thu Sep 25 13:56:59 2003 Subject: [DB-SIG] Inheriting from a DBAPI 2.0 Cursor Object In-Reply-To: <5.2.1.1.2.20030925183547.02dfabd0@localhost> Message-ID: On Thursday, Sep 25, 2003, at 09:52 US/Pacific, Peter L. Buschman wrote: > My connect() function is simply a wrapper around the underlying > driver's connect method that is smart enough to know > what driver it is talking to and what parameters to use. The object > returned is simply one of that driver's connection objects. > > How would I best define my own connection and cursor objects such that > I can inherit from the underlying driver but also > override those few methods like execute to make the driver access as > transparent as possible? The 'connect' function would need to return one of your OWN object types, which can either 'wrap' or 'inherit' from the real type. Once you do that, then you can define all the methods how you please, even if its just to call the wrapped object (or, if you're inheriting, you can just omit it). For example, here's a rough outline of how I'd do it myself: In 'driver' def new ( connection_type ): if connection_type = "mxodbc": return MxodbcDriver () class MxodbcDriver: def connect ( self, *argc, **argk ): return MxodbcWrapper ( db ): # although, I dont know why you'd want to do it the above way... you could always specifiy BOTH the database type, and the arguments, in the 'new' function. class MxodbcWrapper: def __init__ ( self, *argc, *argk ): self.db = mxodbc ( *argc, **argk ) def cursor ( self ): # do wrapper specific stuff return MxodbcCursor ( self ) class MxodbcCursor: def __init__ ( self, wrapper, cursor ): self.wrapper = wrapper self.cursor = self.wrapper.db.cursor () # This gets the real DB's cursor def execute ( self, args ): # self.wrapper can be used to get access to the wrapper, or underlying DB # self.cursor can be used to get to the real cursor # wrapper specific stuff self.cursor.execute ( args ) -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From daniel.dittmar at sap.com Thu Sep 25 14:36:31 2003 From: daniel.dittmar at sap.com (Daniel Dittmar) Date: Thu Sep 25 14:36:48 2003 Subject: [DB-SIG] Re: May we get an actual ANSWER to the F.A.Q...How c an I construct a dictionary??? In-Reply-To: <031701c3837f$d93b8e30$7401a8c0@voidmk9> References: <031701c3837f$d93b8e30$7401a8c0@voidmk9> Message-ID: Jon Franz wrote: > I think a silent failure is a Bad Idea(TM) - but which > method would be acceptable to the community: > - throw an exception > - allow access to the other same-name columns via a > suffix/renaming scheme ('A:1', 'A:2', ...) Or throw an exception only when such a column is accessed by name. - if the duplicate name is accidental, you get an exception - if it is intentional (perhaps you get the statement from somewhere else), you can still access the problematic columns by index Daniel Dittmar -- Daniel Dittmar SAP DB, SAP Labs Berlin daniel.dittmar@sap.com http://www.sapdb.org From davidrushby at yahoo.com Thu Sep 25 16:48:13 2003 From: davidrushby at yahoo.com (David Rushby) Date: Thu Sep 25 16:48:17 2003 Subject: [DB-SIG] Distributed Transactions Message-ID: <20030925204813.33878.qmail@web11004.mail.yahoo.com> In January 2002 there was some talk by Jim Fulton and others of creating a standard Python framework for distributed transaction management: http://mail.python.org/pipermail/db-sig/2002-January/002233.html Did anything ever come of this? I'm working on a project that requires distributed transactions, and although I have the necessary low-level elements (a database that supports distributed two-phase commit (Firebird) and a module that exposes that functionality (kinterbasdb)), I can't find much info about standard distributed transaction frameworks for Python, with the exception of: http://www.hare.demon.co.uk/pyxasw/ My interoperability and scalability requirements are minimal, so I don't want to pay the price in complexity of dealing with full-blown XA. I'll write a simplistic TransactionManager/ResourceManager system myself if necessary, but of course I'd rather deal with an existing Python-oriented framework. --- A related question: aside from Firebird and ZODB (via ZEO), are there any open source databases (relational or otherwise) that support distributed transactions? AFAIK none of PostgreSQL, MySQL, or SAPDB (now MaxDB) have this feature. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From plblists at iotk.com Thu Sep 25 21:19:21 2003 From: plblists at iotk.com (Peter L. Buschman) Date: Thu Sep 25 21:20:33 2003 Subject: [DB-SIG] Inheriting from a DBAPI 2.0 Cursor Object In-Reply-To: References: <5.2.1.1.2.20030925183547.02dfabd0@localhost> Message-ID: <5.2.1.1.2.20030926031705.043b0db0@127.0.0.1> Chis: Thanks so much. I had been so focused on finding an inheritance solution ( difficult since the Connection and Cursor objects are not exposed directly in DBAPI ) that I completely missed simply wrapping the relevant objects. DBAPI is not that complex, so wrapping the methods is not that much extra code. --Peter At 10:56 AM 9/25/2003 -0700, Chris Cogdon wrote: >On Thursday, Sep 25, 2003, at 09:52 US/Pacific, Peter L. Buschman wrote: > >>My connect() function is simply a wrapper around the underlying driver's >>connect method that is smart enough to know >>what driver it is talking to and what parameters to use. The object >>returned is simply one of that driver's connection objects. >> >>How would I best define my own connection and cursor objects such that I >>can inherit from the underlying driver but also >>override those few methods like execute to make the driver access as >>transparent as possible? > >The 'connect' function would need to return one of your OWN object types, >which can either 'wrap' or 'inherit' from the real type. Once you do that, >then you can define all the methods how you please, even if its just to >call the wrapped object (or, if you're inheriting, you can just omit it). > >For example, here's a rough outline of how I'd do it myself: > >In 'driver' > >def new ( connection_type ): > if connection_type = "mxodbc": > return MxodbcDriver () > >class MxodbcDriver: > > def connect ( self, *argc, **argk ): > return MxodbcWrapper ( db ): > ># although, I dont know why you'd want to do it the above way... you could >always specifiy BOTH the database type, and the arguments, in the 'new' >function. > > >class MxodbcWrapper: > > def __init__ ( self, *argc, *argk ): > self.db = mxodbc ( *argc, **argk ) > > def cursor ( self ): > # do wrapper specific stuff > return MxodbcCursor ( self ) > > >class MxodbcCursor: > > def __init__ ( self, wrapper, cursor ): > self.wrapper = wrapper > self.cursor = self.wrapper.db.cursor () # This gets the > real DB's cursor > > def execute ( self, args ): > # self.wrapper can be used to get access to the wrapper, > or underlying DB > # self.cursor can be used to get to the real cursor > > # wrapper specific stuff > > self.cursor.execute ( args ) > > > >-- > ("`-/")_.-'"``-._ Chris Cogdon > . . `; -._ )-;-,_`) > (v_,)' _ )`-.\ ``-' > _.- _..-_/ / ((.' >((,.-' ((,/ fL > > >_______________________________________________ >DB-SIG maillist - DB-SIG@python.org >http://mail.python.org/mailman/listinfo/db-sig From chris at cogdon.org Thu Sep 25 21:55:12 2003 From: chris at cogdon.org (Chris Cogdon) Date: Thu Sep 25 21:55:23 2003 Subject: [DB-SIG] Inheriting from a DBAPI 2.0 Cursor Object In-Reply-To: <5.2.1.1.2.20030926031705.043b0db0@127.0.0.1> Message-ID: <771D020B-EFC4-11D7-83A6-000A95E3823E@cogdon.org> On Thursday, Sep 25, 2003, at 18:19 US/Pacific, Peter L. Buschman wrote: > > Chis: > > Thanks so much. I had been so focused on finding an inheritance > solution ( difficult since the Connection and Cursor objects are not > exposed > directly in DBAPI ) that I completely missed simply wrapping the > relevant objects. DBAPI is not that complex, so wrapping the methods > is not > that much extra code. You're welcome. As a general solution, you can also include a 'auto wrapper' that will pass through calls to the wrapped object if they're not defined at the current level. Just put: def __getattr__ ( self, name ): return getattr ( self.db, name ) ... into the class, and then you won't have to write 'trivial' wrappers for all the methods that simply pass the call to the wrapped object. -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From plblists at iotk.com Fri Sep 26 06:38:30 2003 From: plblists at iotk.com (Peter L. Buschman) Date: Fri Sep 26 06:38:50 2003 Subject: [DB-SIG] Inheriting from a DBAPI 2.0 Cursor Object In-Reply-To: <771D020B-EFC4-11D7-83A6-000A95E3823E@cogdon.org> References: <5.2.1.1.2.20030926031705.043b0db0@127.0.0.1> Message-ID: <5.2.1.1.2.20030926123544.0422a420@127.0.0.1> That's even better since that can still allow a pass-through to extended methods that are present in some of the driver's out there ( in the case where the dal is used just as an intelligent driver loader. ) --PLB At 06:55 PM 9/25/2003 -0700, Chris Cogdon wrote: >On Thursday, Sep 25, 2003, at 18:19 US/Pacific, Peter L. Buschman wrote: > >> >>Chis: >> >>Thanks so much. I had been so focused on finding an inheritance solution >>( difficult since the Connection and Cursor objects are not exposed >>directly in DBAPI ) that I completely missed simply wrapping the relevant >>objects. DBAPI is not that complex, so wrapping the methods is not >>that much extra code. > >You're welcome. > >As a general solution, you can also include a 'auto wrapper' that will >pass through calls to the wrapped object if they're not defined at the >current level. > >Just put: > >def __getattr__ ( self, name ): > return getattr ( self.db, name ) > >... into the class, and then you won't have to write 'trivial' wrappers >for all the methods that simply pass the call to the wrapped object. > > >-- > ("`-/")_.-'"``-._ Chris Cogdon > . . `; -._ )-;-,_`) > (v_,)' _ )`-.\ ``-' > _.- _..-_/ / ((.' >((,.-' ((,/ fL > From plblists at iotk.com Fri Sep 26 07:06:18 2003 From: plblists at iotk.com (Peter L. Buschman) Date: Fri Sep 26 07:06:26 2003 Subject: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions Message-ID: <5.2.1.1.2.20030926123906.0422a568@localhost> Here's the next challenge in my ongoing quest to write a DBAPI 2.0-compliant abstraction layer.... wrapping exceptions. For this example, let's assume I am using mysqldb as the underlying driver. class DriverBase(object): def __init__( self, paramstyle=None ): if paramstyle == None: self.paramstyle = 'qmark' # Default public paramstyle else: self.paramstyle = paramstyle import MySQLdb as driver class Driver(DriverBase): name = 'mysqldb' def __init__( self, paramstyle=None ): self._driver = driver # Link to the underlying MySQLdb driver namespace super(Driver, self).__init__(paramstyle) self.version = driver.__version__ self._paramstyle = driver.paramstyle # Underlying private paramstyle self.apilevel = driver.apilevel self.threadsafety = driver.threadsafety The next thing I want to do is wrap the DPABI exceptions of the underlying driver and raise my own equivalents ( which will handle exception arguments consistently... ) whenever they occur. Here is the strategy that comes to mind, but I am wondering how the rest of you might handle a case like this? # First define our own DBAPI exeptions... this example is limited to only three for brevity. import exceptions class Error(exceptions.StandardError): # DAL-specific stuff pass class DatabaseError(Error): # DAL-specific stuff pass class InternalError(DatabaseError): # DAL-specific stuff pass With the exceptions defined, I want to raise them whenever a wrapped method call raises an exception in the underlying driver. Here is how I might do this with the connect() function in the main driver ( this example returns a Connection object from the underlying driver... the final implementation will wrap those objects as well. ) def connect( **kwargs ): try: return driver.connect(**kwargs) except driver.Error, args: raise Error, args except driver.DatabaseError, args: raise DatabaseError, args except driver.InternalError, args: raise InternalError, args Is this type of brute-force exception checking around every wrapped method call necessary or am I missing a much simpler and elegant solution? Thanks again for everyone's great feedback. Peter Buschman -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20030926/0de9bf5f/attachment-0001.html From jacobs at penguin.theopalgroup.com Fri Sep 26 07:31:31 2003 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Sep 26 07:31:43 2003 Subject: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions In-Reply-To: <5.2.1.1.2.20030926123906.0422a568@localhost> Message-ID: On Fri, 26 Sep 2003, Peter L. Buschman wrote: > The next thing I want to do is wrap the DPABI exceptions of the underlying > driver and raise my own equivalents ( which will handle > exception arguments consistently... ) whenever they occur. Here is the > strategy that comes to mind, but I am wondering how the rest of you might > handle a case like this? My DB-API abstraction layer virtualizes exceptions by patching the base classes of the driver exception classes: abstract.py: class Warning (StandardError) : pass class Error (StandardError) : pass class InterfaceError (Error) : pass class DatabaseError (Error) : pass class DataError (DatabaseError) : pass class OperationalError (DatabaseError) : pass class IntegrityError (DatabaseError) : pass class InternalError (DatabaseError) : pass class ProgrammingError (DatabaseError) : pass class NotSupportedError (DatabaseError) : pass dbapi_exceptions = [ 'Warning', 'Error', 'InterfaceError', 'DatabaseError', 'DataError', 'OperationalError', 'IntegrityError', 'InternalError', 'ProgrammingError', 'NotSupportedError' ] def __import_exceptions(module): for e in dbapi_exceptions: sub_exception = getattr(module, e) abstract_exception = globals()[e] sub_exception.__bases__ += (abstract_exception,) Whenever I import a native DB-API driver, I call __import_exceptions, e.g.: import psycopg, MySQLdb, Sybase, PoPy, pgdb, MSSQL # ... from pyPgSQL import PgSQL __import_exceptions(psycopg) __import_exceptions(MySQLdb) __import_exceptions(Sybase) __import_exceptions(PoPy) __import_exceptions(pgdb) __import_exceptions(MSSQL) __import_exceptions(PgSQL) Now I can catch any DB-API exception from the above modules without knowing which one generated it. def query(dbcon, sql): cursor = dbcon.cursor() cursor.execute(sql) return cursor.fetchall() def get_accounts(dbcon): try: return query(dbcon, 'SELECT ACCOUNTID FROM ACCOUNTS;') except abstract.ProgrammingError: # Do not trap errors due to bad SQL raise except abstract.DatabaseError: # All other database related errors return empty results return [] Clearly, this example is a little contrived, but it gets the point across. The above routines do not need to know which underlying driver they are connected to to be able to catch exceptions. Of course, there are a thousand _other_ things that are not abstracted in this example. Feel free to reuse the above idea. Just mention me in the credits somewhere. ;) -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From gy at carderockcapital.com Fri Sep 26 12:14:12 2003 From: gy at carderockcapital.com (Gail Yano) Date: Fri Sep 26 12:10:07 2003 Subject: [DB-SIG] Python vs Delphi; MS SQL Server vs. Anything Message-ID: I have been struggling with re-writing some DOS Paradox DB programs using Delphi and MS SQL. My boss wants me to investigate switching to Python. I can't seem to find anything about MS SQL on the Python website [understandably]. I'm open to suggestions as to starting over with Python and if so, what database to use. PS: I'm an end-user who happens to program a little. Thanks, From guido at python.org Fri Sep 26 12:49:58 2003 From: guido at python.org (Guido van Rossum) Date: Fri Sep 26 12:50:50 2003 Subject: [DB-SIG] Python vs Delphi; MS SQL Server vs. Anything In-Reply-To: Your message of "Fri, 26 Sep 2003 12:14:12 EDT." References: Message-ID: <200309261649.h8QGnw621741@12-236-84-31.client.attbi.com> I don't know what MS SQL is, but I can recommend MySQL. The Python support for it is solid from my (only 8 weeks of) experience. --Guido van Rossum (home page: http://www.python.org/~guido/) From gh at ghaering.de Fri Sep 26 13:13:44 2003 From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=) Date: Fri Sep 26 13:13:49 2003 Subject: [DB-SIG] Python vs Delphi; MS SQL Server vs. Anything In-Reply-To: <200309261649.h8QGnw621741@12-236-84-31.client.attbi.com> References: <200309261649.h8QGnw621741@12-236-84-31.client.attbi.com> Message-ID: <3F7473C8.50509@ghaering.de> Guido van Rossum wrote: > I don't know what MS SQL is, Microsoft SQL Server, recent versions of which are based on Sybase code. > but I can recommend MySQL. The Python support for it is solid from > my (only 8 weeks of) experience. If MySQL fits your (limited) needs, go for it! ;-) -- Gerhard From davidrushby at yahoo.com Fri Sep 26 13:41:56 2003 From: davidrushby at yahoo.com (David Rushby) Date: Fri Sep 26 13:41:59 2003 Subject: [DB-SIG] Python vs Delphi; MS SQL Server vs. Anything In-Reply-To: Message-ID: <20030926174156.18040.qmail@web11001.mail.yahoo.com> --- Gail Yano wrote: > I have been struggling with re-writing some DOS Paradox DB programs > using Delphi and MS SQL. My boss wants me to investigate switching > to Python. I can't seem to find anything about MS SQL on the Python > website [understandably]. I think the Python Sybase module also supports MS SQL to some extent (I have no direct experience with the Sybase module, so I could be wrong): http://www.object-craft.com.au/projects/sybase/ If not, you could access MS SQL via mxODBC, but it's commercial: http://www.egenix.com/files/python/mxODBC.html A third alternative would be to use Microsoft's ADO library via COM. This would only work if your client program doesn't need to run on anything but Windows. To access COM from Python, see: http://starship.python.net/crew/mhammond/win32/Downloads.html > I'm open to suggestions as to starting over with Python... What kind of user interface do the Paradox programs have? What kind are the replacements intended to have? I suspect you'll find GUI construction more difficult in Python than in Delphi, because of Delphi's refined IDE. For a graphical form builder, you might try Boa Constructor (but YMMV as to its level of maturity and stability): http://boa-constructor.sourceforge.net/ Alternatively, PythonCard is touted as an easy GUI toolkit to program directly: http://pythoncard.sourceforge.net/ > ... and if so, what database to use. > > PS: I'm an end-user who happens to program a little. MS SQL is a nice database engine, but it's restrictively licensed. There's a somewhat freely redistributable embedded version called MSDE, but it has a bunch of limitations as compared to full-blown MS SQL Server: http://msdn.microsoft.com/vstudio/downloads/addins/msde/default.aspx --- As an alternative, how about Firebird (an open source descendant of Borland's Interbase)? http://firebirdsql.org/ Firebird suits your situation well because: a) Although it has a broad feature set (far broader than MySQL), it's easy to use only those features that you need and ignore the rest (it's similar to Python in this regard). b) It can operate in embedded or standalone server mode. "Embedded mode" means that the Firebird engine runs as a library in the same process with Python, so you don't have to maintain an external server process. This is primarily useful to simplify distribution and installation, or for deployment on operating systems that support background services poorly, such as Win9x. If multiple programs are to access the same database concurrently, use the server mode instead. c) It runs well on Windows, unlike PostgreSQL. d) Refined and free Firebird administration GUIs are available for Windows. e) It has feature-complete and reasonably well documented Python support via the kinterbasdb module: http://kinterbasdb.sourceforge.net f) It's covered by a permissive Mozilla-style license, rather than a restrictive license like that of MSDE or MySQL. MySQL is only free under two circumstances: 1) You release your MySQL-based application under an open source license. 2) You "never distribute (internally or externally) the MySQL Software in any way". Unlike the GPL, which applies only to in-process code and, furthermore, allows intra-company "internal distribution" without releasing the altered code, the MySQL license applies to any program in your organization that accesses the MySQL server, whether in-process or across a network, and requires a license for "internal distribution". "Internal [intra-company] distribution" of a piece of software could be interpreted to mean almost anything; it's a wide open ambiguity in the MySQL license. You can read my own interpretation of it here: http://listserv.sap.com/pipermail/sapdb.general/2003-May/015583.html or a MySQL employee's interpretation here: http://listserv.sap.com/pipermail/sapdb.general/2003-May/015478.html --- Yet another alternative is SQLite (http://sqlite.org/), an admirably compact database engine with a decent feature set and a good Python module, but no standalone server mode of operation. --- Disclaimer: I'm biased towards Firebird because I'm deeply involved in the Python driver, kinterbasdb. I'm not a zealot, though, and I "eat my own dogfood", which should make my biased commentary less suspect. __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From plblists at iotk.com Fri Sep 26 14:59:02 2003 From: plblists at iotk.com (Peter L. Buschman) Date: Fri Sep 26 14:59:14 2003 Subject: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions In-Reply-To: References: <5.2.1.1.2.20030926123906.0422a568@localhost> Message-ID: <5.2.1.1.2.20030926205550.041bdba0@127.0.0.1> Kevin: That is awesome. It took me a little while to understand how your code example worked, but that is definitely a better approach than what I was thinking of and the effect is exactly what I am aiming for. Thank you. You can be sure that you and everyone else will be mentioned in the credits for answering my newbie questions :) Cheers! --PLB At 07:31 AM 9/26/2003 -0400, Kevin Jacobs wrote: >On Fri, 26 Sep 2003, Peter L. Buschman wrote: > > The next thing I want to do is wrap the DPABI exceptions of the underlying > > driver and raise my own equivalents ( which will handle > > exception arguments consistently... ) whenever they occur. Here is the > > strategy that comes to mind, but I am wondering how the rest of you might > > handle a case like this? > >My DB-API abstraction layer virtualizes exceptions by patching the base >classes of the driver exception classes: > >abstract.py: > > class Warning (StandardError) : pass > class Error (StandardError) : pass > class InterfaceError (Error) : pass > class DatabaseError (Error) : pass > class DataError (DatabaseError) : pass > class OperationalError (DatabaseError) : pass > class IntegrityError (DatabaseError) : pass > class InternalError (DatabaseError) : pass > class ProgrammingError (DatabaseError) : pass > class NotSupportedError (DatabaseError) : pass > > dbapi_exceptions = [ 'Warning', > 'Error', > 'InterfaceError', > 'DatabaseError', > 'DataError', > 'OperationalError', > 'IntegrityError', > 'InternalError', > 'ProgrammingError', > 'NotSupportedError' ] > > def __import_exceptions(module): > for e in dbapi_exceptions: > sub_exception = getattr(module, e) > abstract_exception = globals()[e] > sub_exception.__bases__ += (abstract_exception,) > >Whenever I import a native DB-API driver, I call __import_exceptions, e.g.: > > import psycopg, MySQLdb, Sybase, PoPy, pgdb, MSSQL # ... > from pyPgSQL import PgSQL > > __import_exceptions(psycopg) > __import_exceptions(MySQLdb) > __import_exceptions(Sybase) > __import_exceptions(PoPy) > __import_exceptions(pgdb) > __import_exceptions(MSSQL) > __import_exceptions(PgSQL) > >Now I can catch any DB-API exception from the above modules without knowing >which one generated it. > > def query(dbcon, sql): > cursor = dbcon.cursor() > cursor.execute(sql) > return cursor.fetchall() > > def get_accounts(dbcon): > try: > return query(dbcon, 'SELECT ACCOUNTID FROM ACCOUNTS;') > except abstract.ProgrammingError: > # Do not trap errors due to bad SQL > raise > except abstract.DatabaseError: > # All other database related errors return empty results > return [] > >Clearly, this example is a little contrived, but it gets the point across. >The above routines do not need to know which underlying driver they are >connected to to be able to catch exceptions. Of course, there are a >thousand _other_ things that are not abstracted in this example. > >Feel free to reuse the above idea. Just mention me in the credits >somewhere. ;) > >-Kevin > > >-- >-- >Kevin Jacobs >The OPAL Group - Enterprise Systems Architect >Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com >Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From brian at kempf.com Fri Sep 26 15:58:02 2003 From: brian at kempf.com (Brian Dorsey) Date: Fri Sep 26 15:58:07 2003 Subject: [DB-SIG] Python vs Delphi; MS SQL Server vs. Anything In-Reply-To: <20030926174156.18040.qmail@web11001.mail.yahoo.com> References: <20030926174156.18040.qmail@web11001.mail.yahoo.com> Message-ID: <20030926195802.GA17190@dorseys.org> Gail, In addition to David's excellent recommendations, I'd like to add the adodbapi module. It is only usable on windows, but it makes it very easy to connect to SQL Server from python. I use it regularly. http://adodbapi.sourceforge.net/ Take care, -Brian On Fri, Sep 26, 2003 at 10:41:56AM -0700, David Rushby wrote: > --- Gail Yano wrote: > > I have been struggling with re-writing some DOS Paradox DB programs > > using Delphi and MS SQL. My boss wants me to investigate switching > > to Python. I can't seem to find anything about MS SQL on the Python > > website [understandably]. > > I think the Python Sybase module also supports MS SQL to some extent (I > have no direct experience with the Sybase module, so I could be wrong): > http://www.object-craft.com.au/projects/sybase/ > > If not, you could access MS SQL via mxODBC, but it's commercial: > http://www.egenix.com/files/python/mxODBC.html > > A third alternative would be to use Microsoft's ADO library via COM. > This would only work if your client program doesn't need to run on > anything but Windows. To access COM from Python, see: > http://starship.python.net/crew/mhammond/win32/Downloads.html > > > I'm open to suggestions as to starting over with Python... > > What kind of user interface do the Paradox programs have? What kind > are the replacements intended to have? I suspect you'll find GUI > construction more difficult in Python than in Delphi, because of > Delphi's refined IDE. For a graphical form builder, you might try Boa > Constructor (but YMMV as to its level of maturity and stability): > http://boa-constructor.sourceforge.net/ > > Alternatively, PythonCard is touted as an easy GUI toolkit to program > directly: > http://pythoncard.sourceforge.net/ > > > ... and if so, what database to use. > > > > PS: I'm an end-user who happens to program a little. > > MS SQL is a nice database engine, but it's restrictively licensed. > There's a somewhat freely redistributable embedded version called MSDE, > but it has a bunch of limitations as compared to full-blown MS SQL > Server: > http://msdn.microsoft.com/vstudio/downloads/addins/msde/default.aspx > > --- > > As an alternative, how about Firebird (an open source descendant of > Borland's Interbase)? > http://firebirdsql.org/ > > Firebird suits your situation well because: > a) Although it has a broad feature set (far broader than MySQL), it's > easy to use only those features that you need and ignore the rest (it's > similar to Python in this regard). > > b) It can operate in embedded or standalone server mode. "Embedded > mode" means that the Firebird engine runs as a library in the same > process with Python, so you don't have to maintain an external server > process. This is primarily useful to simplify distribution and > installation, or for deployment on operating systems that support > background services poorly, such as Win9x. If multiple programs are to > access the same database concurrently, use the server mode instead. > > c) It runs well on Windows, unlike PostgreSQL. > > d) Refined and free Firebird administration GUIs are available for > Windows. > > e) It has feature-complete and reasonably well documented Python > support via the kinterbasdb module: > http://kinterbasdb.sourceforge.net > > f) It's covered by a permissive Mozilla-style license, rather than a > restrictive license like that of MSDE or MySQL. MySQL is only free > under two circumstances: > 1) You release your MySQL-based application under an open source > license. > 2) You "never distribute (internally or externally) the MySQL > Software in any way". Unlike the GPL, which applies only to in-process > code and, furthermore, allows intra-company "internal distribution" > without releasing the altered code, the MySQL license applies to any > program in your organization that accesses the MySQL server, whether > in-process or across a network, and requires a license for "internal > distribution". > "Internal [intra-company] distribution" of a piece of software could > be interpreted to mean almost anything; it's a wide open ambiguity in > the MySQL license. You can read my own interpretation of it here: > http://listserv.sap.com/pipermail/sapdb.general/2003-May/015583.html > or a MySQL employee's interpretation here: > http://listserv.sap.com/pipermail/sapdb.general/2003-May/015478.html > > --- > > Yet another alternative is SQLite (http://sqlite.org/), an admirably > compact database engine with a decent feature set and a good Python > module, but no standalone server mode of operation. > > --- > > Disclaimer: I'm biased towards Firebird because I'm deeply involved in > the Python driver, kinterbasdb. I'm not a zealot, though, and I "eat > my own dogfood", which should make my biased commentary less suspect. > > __________________________________ > Do you Yahoo!? > The New Yahoo! Shopping - with improved product search > http://shopping.yahoo.com > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig From jfranz at neurokodeXREMOVEX.com Fri Sep 26 16:28:12 2003 From: jfranz at neurokodeXREMOVEX.com (Jon Franz_antispam) Date: Fri Sep 26 16:24:25 2003 Subject: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions References: Message-ID: <072701c3846c$b561dd50$7401a8c0@voidmk9> As for PDO, it defines its own exception classes for the different types of exceptions (connect error, etc), and places information about the underlying db module specific exception into the string description of the exception. Thus, users of PDO get one set of exceptions to trap for, but still get all the information they need to track down the cause. ~Jon Franz NeuroKode Labs, LLC ----- Original Message ----- From: "Kevin Jacobs" To: "Peter L. Buschman" Cc: Sent: Friday, September 26, 2003 7:31 AM Subject: Re: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions > On Fri, 26 Sep 2003, Peter L. Buschman wrote: > > The next thing I want to do is wrap the DPABI exceptions of the underlying > > driver and raise my own equivalents ( which will handle > > exception arguments consistently... ) whenever they occur. Here is the > > strategy that comes to mind, but I am wondering how the rest of you might > > handle a case like this? > > My DB-API abstraction layer virtualizes exceptions by patching the base > classes of the driver exception classes: > > abstract.py: > > class Warning (StandardError) : pass > class Error (StandardError) : pass > class InterfaceError (Error) : pass > class DatabaseError (Error) : pass > class DataError (DatabaseError) : pass > class OperationalError (DatabaseError) : pass > class IntegrityError (DatabaseError) : pass > class InternalError (DatabaseError) : pass > class ProgrammingError (DatabaseError) : pass > class NotSupportedError (DatabaseError) : pass > > dbapi_exceptions = [ 'Warning', > 'Error', > 'InterfaceError', > 'DatabaseError', > 'DataError', > 'OperationalError', > 'IntegrityError', > 'InternalError', > 'ProgrammingError', > 'NotSupportedError' ] > > def __import_exceptions(module): > for e in dbapi_exceptions: > sub_exception = getattr(module, e) > abstract_exception = globals()[e] > sub_exception.__bases__ += (abstract_exception,) > > Whenever I import a native DB-API driver, I call __import_exceptions, e.g.: > > import psycopg, MySQLdb, Sybase, PoPy, pgdb, MSSQL # ... > from pyPgSQL import PgSQL > > __import_exceptions(psycopg) > __import_exceptions(MySQLdb) > __import_exceptions(Sybase) > __import_exceptions(PoPy) > __import_exceptions(pgdb) > __import_exceptions(MSSQL) > __import_exceptions(PgSQL) > > Now I can catch any DB-API exception from the above modules without knowing > which one generated it. > > def query(dbcon, sql): > cursor = dbcon.cursor() > cursor.execute(sql) > return cursor.fetchall() > > def get_accounts(dbcon): > try: > return query(dbcon, 'SELECT ACCOUNTID FROM ACCOUNTS;') > except abstract.ProgrammingError: > # Do not trap errors due to bad SQL > raise > except abstract.DatabaseError: > # All other database related errors return empty results > return [] > > Clearly, this example is a little contrived, but it gets the point across. > The above routines do not need to know which underlying driver they are > connected to to be able to catch exceptions. Of course, there are a > thousand _other_ things that are not abstracted in this example. > > Feel free to reuse the above idea. Just mention me in the credits > somewhere. ;) > > -Kevin > > > -- > -- > Kevin Jacobs > The OPAL Group - Enterprise Systems Architect > Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com > Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ > > > > From plblists at iotk.com Sat Sep 27 13:14:27 2003 From: plblists at iotk.com (Peter L. Buschman) Date: Sat Sep 27 13:14:42 2003 Subject: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions In-Reply-To: <072701c3846c$b561dd50$7401a8c0@voidmk9> References: Message-ID: <5.2.1.1.2.20030927190701.044c89d8@127.0.0.1> Very cool. :) That's where I'm going with this too, only I'll be using the same exception names as DBAPI. One of the issues I have with DBAPI right now is that the arguments passed to exceptions are inconsistent, making it hard to get the error message to the end user. My exceptions will handle this intelligently and pass the error message (and possibly optional extra stuff depending on the driver) in the same format regardless of the underlying driver. The low-level effect will be a database abstraction layer that makes it appear as if all the DBAPI driver authors had coded to the same style in all the areas where DBAPI offers multiple options ( paramstyle, exception arguments, etc., ) Just like DBO, users get one set of exceptions to trap for, but they are DBAPI-2.0 compliant exceptions. The high-level effect will go on to extend this architecture, offering common features that were never in DBAPI at all, but doing it for all of the drivers supported at the low-level. Supporting differences between SQL dialects will be in there. --PLB At 04:28 PM 9/26/2003 -0400, Jon Franz_antispam wrote: >As for PDO, it defines its own exception classes for the different types of >exceptions (connect error, etc), >and places information about the underlying db module specific exception >into the string description >of the exception. Thus, users of PDO get one set of exceptions to trap for, >but still get all the information >they need to track down the cause. > >~Jon Franz >NeuroKode Labs, LLC > >----- Original Message ----- >From: "Kevin Jacobs" >To: "Peter L. Buschman" >Cc: >Sent: Friday, September 26, 2003 7:31 AM >Subject: Re: [DB-SIG] (Database Abstraction Layer) Wrapping DBAPI Exceptions > > > > On Fri, 26 Sep 2003, Peter L. Buschman wrote: > > > The next thing I want to do is wrap the DPABI exceptions of the >underlying > > > driver and raise my own equivalents ( which will handle > > > exception arguments consistently... ) whenever they occur. Here is the > > > strategy that comes to mind, but I am wondering how the rest of you >might > > > handle a case like this? > > > > My DB-API abstraction layer virtualizes exceptions by patching the base > > classes of the driver exception classes: > > > > abstract.py: > > > > class Warning (StandardError) : pass > > class Error (StandardError) : pass > > class InterfaceError (Error) : pass > > class DatabaseError (Error) : pass > > class DataError (DatabaseError) : pass > > class OperationalError (DatabaseError) : pass > > class IntegrityError (DatabaseError) : pass > > class InternalError (DatabaseError) : pass > > class ProgrammingError (DatabaseError) : pass > > class NotSupportedError (DatabaseError) : pass > > > > dbapi_exceptions = [ 'Warning', > > 'Error', > > 'InterfaceError', > > 'DatabaseError', > > 'DataError', > > 'OperationalError', > > 'IntegrityError', > > 'InternalError', > > 'ProgrammingError', > > 'NotSupportedError' ] > > > > def __import_exceptions(module): > > for e in dbapi_exceptions: > > sub_exception = getattr(module, e) > > abstract_exception = globals()[e] > > sub_exception.__bases__ += (abstract_exception,) > > > > Whenever I import a native DB-API driver, I call __import_exceptions, >e.g.: > > > > import psycopg, MySQLdb, Sybase, PoPy, pgdb, MSSQL # ... > > from pyPgSQL import PgSQL > > > > __import_exceptions(psycopg) > > __import_exceptions(MySQLdb) > > __import_exceptions(Sybase) > > __import_exceptions(PoPy) > > __import_exceptions(pgdb) > > __import_exceptions(MSSQL) > > __import_exceptions(PgSQL) > > > > Now I can catch any DB-API exception from the above modules without >knowing > > which one generated it. > > > > def query(dbcon, sql): > > cursor = dbcon.cursor() > > cursor.execute(sql) > > return cursor.fetchall() > > > > def get_accounts(dbcon): > > try: > > return query(dbcon, 'SELECT ACCOUNTID FROM ACCOUNTS;') > > except abstract.ProgrammingError: > > # Do not trap errors due to bad SQL > > raise > > except abstract.DatabaseError: > > # All other database related errors return empty results > > return [] > > > > Clearly, this example is a little contrived, but it gets the point across. > > The above routines do not need to know which underlying driver they are > > connected to to be able to catch exceptions. Of course, there are a > > thousand _other_ things that are not abstracted in this example. > > > > Feel free to reuse the above idea. Just mention me in the credits > > somewhere. ;) > > > > -Kevin > > > > > > -- > > -- > > Kevin Jacobs > > The OPAL Group - Enterprise Systems Architect > > Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com > > Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ > > > > > > > > From pythontutor at venix.com Sat Sep 27 18:26:50 2003 From: pythontutor at venix.com (Lloyd Kvam) Date: Sat Sep 27 18:26:52 2003 Subject: [DB-SIG] DBMS with easy administration and replication Message-ID: <3F760EAA.3010901@venix.com> I have software that uses MySQL in a loosely coupled environment. Replication is used to export changes from a central datbase to the remote databases (about 40) using modem connections. The remote replication logs are uploaded using the modem connections, collected at the central server, and fed back into the central database. The application is handling fairly low volume sales and reservations. There is no tech support at the remote locations, so database administration must be minimal. The central server is running RedHat 9 Linux while the remote locations are running Win2k. Everything works reasonably well, but I am curious as to whether I could make this work with another DBMS. Firebird seems like a possible candidate. I could not find any details about replication on the Firebird web site. I think distributed updates are too restrictive, because the remote databases are normally off-line. I was hoping that someone on the list would know if Firebird supported replication or if there was another DBMS that had the same mix of easy administration and replication support that made MySQL a good choice. -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-443-6155 fax: 801-459-9582 From davidrushby at yahoo.com Sun Sep 28 00:03:55 2003 From: davidrushby at yahoo.com (David Rushby) Date: Sun Sep 28 00:03:59 2003 Subject: [DB-SIG] DBMS with easy administration and replication In-Reply-To: <3F760EAA.3010901@venix.com> Message-ID: <20030928040355.62151.qmail@web11001.mail.yahoo.com> --- Lloyd Kvam wrote: > There is no tech support at the remote locations, so database > administration must be minimal. The central server is running > RedHat 9 Linux while the remote locations are running Win2k. Firebird's ease of administration is top notch, in part because its predecessor, Interbase, has long been included with Borland developer tools as an embedded database solution (these days, both are more typically run as a standalone server, AFAIK). Obviously, embedded databases have stringent requirements for ease of administration. I leave the Firebird server running for months at a time at client sites without anything but automated administration (for backups). kinterbasdb also thickly wraps Firebird's administrative API (called the Services API), so you can perform administrative tasks like backup and restore from a Pythonic API instead of scripting command line tools. > Everything works reasonably well, but I am curious as to whether > I could make this work with another DBMS. Firebird seems like > a possible candidate. > I could not find any details about replication on the Firebird > web site. AFAIK, the Firebird core engine contains no support for replication. I know of a few third-party, bolt-on solutions that provide asynchronous master/slave replication, usually implemented via triggers. They look pretty lame compared to Oracle-level replication. Here's one open source bolt-on replicator (in "early beta"): http://fibre.sourceforge.net/ and two commercial ones: http://www.ibphoenix.com/a415.htm http://www.2p.cz/en/ibrepl/ I haven't used any of these, so I can't comment on how well they work. --- As far as the other open source relational databases, neither PostgreSQL nor SAPDB has built-in replication support, though in the case of PostgreSQL, there are some bolt-on implementations similar to those available for Firebird. However, replication is high on the PostgreSQL to-do list: http://developer.postgresql.org/todo.php PostgreSQL's proposed replication includes features such as multi-master replication, cross-server data partitioning, and replication over non-persistent links. I applaud them for aiming to provide a featureful implementation rather than something simplistic. MySQL supports only asynchronous master/slave replication, which is just the tip of the iceberg. Who knows when PostgreSQL's support will arrive, though--they haven't even finished adding two-phase commit support yet. --- As an off-topic bit of trivia, isn't using MySQL on 41 servers expensive? Or maybe your app is open source, or you get a volume discount? __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From mal at lemburg.com Sun Sep 28 06:32:12 2003 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Sep 28 06:32:18 2003 Subject: [DB-SIG] DBMS with easy administration and replication In-Reply-To: <20030928040355.62151.qmail@web11001.mail.yahoo.com> References: <20030928040355.62151.qmail@web11001.mail.yahoo.com> Message-ID: <3F76B8AC.309@lemburg.com> David Rushby wrote: > As far as the other open source relational databases, neither > PostgreSQL nor SAPDB has built-in replication support. SAP DB does come with its own replication manager which is also made available via the Python interfaces that come with SAP DB. However, it only provides means for offline replication. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Sep 28 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From davidrushby at yahoo.com Sun Sep 28 14:15:39 2003 From: davidrushby at yahoo.com (David Rushby) Date: Sun Sep 28 14:15:46 2003 Subject: [DB-SIG] DBMS with easy administration and replication In-Reply-To: <3F76B8AC.309@lemburg.com> Message-ID: <20030928181539.33959.qmail@web11003.mail.yahoo.com> --- "M.-A. Lemburg" wrote: > David Rushby wrote: > > As far as the other open source relational databases, neither > > PostgreSQL nor SAPDB has built-in replication support. > > SAP DB does come with its own replication manager which > is also made available via the Python interfaces that come > with SAP DB. > > However, it only provides means for offline replication. I'm aware of that component of SAPDB, but I consider "Replication Manager" a misnomer, because the component actually provides the functionality that other RDBMSes would refer to as "bulk copy". Apparently the SAPDB folks also regard it as a misnomer, because they renamed it to "Loader" in 7.4, and they themselves maintain that "SAP DB does not support replication features" (http://listserv.sap.com/pipermail/sapdb.general/2000-December/000563.html ). __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From pythontutor at venix.com Mon Sep 29 11:37:18 2003 From: pythontutor at venix.com (Lloyd Kvam) Date: Mon Sep 29 11:37:23 2003 Subject: [DB-SIG] DBMS with easy administration and replication In-Reply-To: <20030928040355.62151.qmail@web11001.mail.yahoo.com> References: <20030928040355.62151.qmail@web11001.mail.yahoo.com> Message-ID: <3F7851AE.5030009@venix.com> Thanks for the very detailed reply. The MySQL log based replication has worked well. The main issue was writing a "collector" to take the logs from the remote locations and combine them into a single replication stream for the central database. > As an off-topic bit of trivia, isn't using MySQL on 41 servers > expensive? Or maybe your app is open source, or you get a volume discount? Yes the application has a GPL. David Rushby wrote: > --- Lloyd Kvam wrote: > >>There is no tech support at the remote locations, so database >>administration must be minimal. The central server is running >>RedHat 9 Linux while the remote locations are running Win2k. > > > Firebird's ease of administration is top notch, in part because its > predecessor, Interbase, has long been included with Borland developer > tools as an embedded database solution (these days, both are more > typically run as a standalone server, AFAIK). Obviously, embedded > databases have stringent requirements for ease of administration. I > leave the Firebird server running for months at a time at client sites > without anything but automated administration (for backups). > > kinterbasdb also thickly wraps Firebird's administrative API (called > the Services API), so you can perform administrative tasks like backup > and restore from a Pythonic API instead of scripting command line > tools. > > >>Everything works reasonably well, but I am curious as to whether >>I could make this work with another DBMS. Firebird seems like >>a possible candidate. >>I could not find any details about replication on the Firebird >>web site. > > > AFAIK, the Firebird core engine contains no support for replication. > > I know of a few third-party, bolt-on solutions that provide > asynchronous master/slave replication, usually implemented via > triggers. They look pretty lame compared to Oracle-level replication. > > Here's one open source bolt-on replicator (in "early beta"): > http://fibre.sourceforge.net/ > > and two commercial ones: > http://www.ibphoenix.com/a415.htm > http://www.2p.cz/en/ibrepl/ > > I haven't used any of these, so I can't comment on how well they work. > > --- > > As far as the other open source relational databases, neither > PostgreSQL nor SAPDB has built-in replication support, though in the > case of PostgreSQL, there are some bolt-on implementations similar to > those available for Firebird. > > However, replication is high on the PostgreSQL to-do list: > http://developer.postgresql.org/todo.php > PostgreSQL's proposed replication includes features such as > multi-master replication, cross-server data partitioning, and > replication over non-persistent links. I applaud them for aiming to > provide a featureful implementation rather than something simplistic. > MySQL supports only asynchronous master/slave replication, which is > just the tip of the iceberg. Who knows when PostgreSQL's support will > arrive, though--they haven't even finished adding two-phase commit > support yet. > > --- > > As an off-topic bit of trivia, isn't using MySQL on 41 servers > expensive? Or maybe your app is open source, or you get a volume discount? > > __________________________________ > Do you Yahoo!? > The New Yahoo! Shopping - with improved product search > http://shopping.yahoo.com > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig > -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-443-6155 fax: 801-459-9582