[Twisted-Python] adbapi, PostgreSQL and Unicode
I keep getting - Unhandled error in Deferred I'm trying to perform an INSERT on my db (using adbapi with *pgdb* module). INSERT which comprises *Unicode* string e.g. - unicodeChar1 = u"Character=[\N{GREEK CAPITAL LETTER GAMMA}]" the db is defined to Unicode *client_encoding* value is set to *Unicode* when checking its value in *cp_openfun *called function execute the same INSERT directly with *pgdb *works great Does anyone has experience with adbapi with Unicode db ?!?! Do I need to define something special for this to happen ?? Thanks for your help, Michal Eldar
Michal Eldar wrote:
I keep getting - Unhandled error in Deferred
You should attach an errback to the deferred and handle, or at least log, the exception. That's what "Unhandled error in Deferred" means ;-).
Yes, but only using psycopg and pgasync. It works fine for me.
Do I need to define something special for this to happen ??
You shouldn't need to. adbapi is nothing more that a wrapper around the real (blocking) database module that dispatches db calls to a thread. Attach the errback first and find out what the exception actually is before trying to fix anything. It could be something as simple as a typo that you haven't spotted or it could be something subtle. It's impossible to know at this stage. I suspect you're building the a complete SQL string and passing that to execute() when you should be passing a SQL string with insertion markers and a tuple/dict of args. But, that's just a guess ;-). Hope this helps. Cheers, Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt@pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer.
Matt Goodall wrote:
<snip> I also use pgdb with adbapi and unicode without problems. Insert's with unicode strings should not be problem even if the database is set to SQLASCII. The pgdb module inspects the string before sending it to the postgres machinery. If it's a unicode string (isinstance('the string', unicode) ) it's encoded to utf-8. I've submitted a patch to Darcy at pygresql.org for enhanced unicode support for pgdb. This patch takes care of the client_encoding stuff and makes it possible to fetch unicode strings from select queries. -- Remy --
Remy Cool wrote:
First of all - Thanks !!! It was a nice surprise to find responses to my quesion :) I read about Unicode in the Twisted Archive and a link to the FAQ gave me the solution: I'm suppose to *encode* the unicode string before I'm passing it over... This was THE difference when working with Twisted. http://twistedmatrix.com/pipermail/twisted-python/2005-April/010198.html http://twistedmatrix.com/projects/core/documentation/howto/faq.html#auto25 Matt - I did try to attach an errback, but it failed to print the error, because it probably included some non-ascii chars... I don't know why Twisted don't show an appropriate error about this problem Remi - is it not possible to retrieve Unicode string from SELECT query with pgdb ? what is the behaviour nowadays? Cheers! and Thanks again, Michal
On May 13, 2005, at 2:33 PM, Michal Eldar wrote:
Actually, that doesn't really apply to ADBAPI. Twisted's ADBAPI doesn't care what you pass to its runQuery/etc methods -- it simply passes that on to the database. It can be a unicode string or any random object, as long as the database adapter accepts it. It's hard to say what might be going wrong without a traceback, of course. James
Michal Eldar wrote:
The standard behaviour of pgdb is that unicode strings are automagically encoded to utf-8 when inserting something in the database and you get plain python strings for select queries. I've looked at the code of other python database connectors and added (transparent) functionality to pgdb so that it can it return unicode strings if you want it to.
Michal Eldar wrote:
I keep getting - Unhandled error in Deferred
You should attach an errback to the deferred and handle, or at least log, the exception. That's what "Unhandled error in Deferred" means ;-).
Yes, but only using psycopg and pgasync. It works fine for me.
Do I need to define something special for this to happen ??
You shouldn't need to. adbapi is nothing more that a wrapper around the real (blocking) database module that dispatches db calls to a thread. Attach the errback first and find out what the exception actually is before trying to fix anything. It could be something as simple as a typo that you haven't spotted or it could be something subtle. It's impossible to know at this stage. I suspect you're building the a complete SQL string and passing that to execute() when you should be passing a SQL string with insertion markers and a tuple/dict of args. But, that's just a guess ;-). Hope this helps. Cheers, Matt -- __ / \__ Matt Goodall, Pollenation Internet Ltd \__/ \ w: http://www.pollenation.net __/ \__/ e: matt@pollenation.net / \__/ \ t: +44 (0)113 2252500 \__/ \__/ / \ Any views expressed are my own and do not necessarily \__/ reflect the views of my employer.
Matt Goodall wrote:
<snip> I also use pgdb with adbapi and unicode without problems. Insert's with unicode strings should not be problem even if the database is set to SQLASCII. The pgdb module inspects the string before sending it to the postgres machinery. If it's a unicode string (isinstance('the string', unicode) ) it's encoded to utf-8. I've submitted a patch to Darcy at pygresql.org for enhanced unicode support for pgdb. This patch takes care of the client_encoding stuff and makes it possible to fetch unicode strings from select queries. -- Remy --
Remy Cool wrote:
First of all - Thanks !!! It was a nice surprise to find responses to my quesion :) I read about Unicode in the Twisted Archive and a link to the FAQ gave me the solution: I'm suppose to *encode* the unicode string before I'm passing it over... This was THE difference when working with Twisted. http://twistedmatrix.com/pipermail/twisted-python/2005-April/010198.html http://twistedmatrix.com/projects/core/documentation/howto/faq.html#auto25 Matt - I did try to attach an errback, but it failed to print the error, because it probably included some non-ascii chars... I don't know why Twisted don't show an appropriate error about this problem Remi - is it not possible to retrieve Unicode string from SELECT query with pgdb ? what is the behaviour nowadays? Cheers! and Thanks again, Michal
On May 13, 2005, at 2:33 PM, Michal Eldar wrote:
Actually, that doesn't really apply to ADBAPI. Twisted's ADBAPI doesn't care what you pass to its runQuery/etc methods -- it simply passes that on to the database. It can be a unicode string or any random object, as long as the database adapter accepts it. It's hard to say what might be going wrong without a traceback, of course. James
Michal Eldar wrote:
The standard behaviour of pgdb is that unicode strings are automagically encoded to utf-8 when inserting something in the database and you get plain python strings for select queries. I've looked at the code of other python database connectors and added (transparent) functionality to pgdb so that it can it return unicode strings if you want it to.
participants (4)
-
James Y Knight -
Matt Goodall -
Michal Eldar -
Remy Cool