[Twisted-Python] unicode for pyPgSQL

Hello, does pyPgSQL doesn't support unicode? execute(query, params) #all items in params list are of <type 'unicode'> I'm getting 'ascii' codec can't encode character u'\xfc' in position 1: ordinal not in range(128) Do I need encode my params before sending a query and then decode all results back to unicode? Pet

On Wed, Aug 12, 2009 at 11:28 AM, Pet <petshmidt@googlemail.com> wrote:
Hello,
does pyPgSQL doesn't support unicode?
Sounds like a question for the pyPgSQL list, not the Twisted list :) <https://lists.sourceforge.net/lists/listinfo/pypgsql-users>

On 03:28 pm, petshmidt@googlemail.com wrote:
I don't know about pyPgSQL's unicode support. I did recently learn that pyPgSQL doesn't support bind parameters, apparently resulting in almost any use of it insecure and vulnerable to SQL injection attacks. You may want to investigate this further before deciding if it's worth figuring out how unicode works. Jean-Paul

On Wed, Aug 12, 2009 at 12:51 PM, <exarkun@twistedmatrix.com> wrote:
I believe exarkun is referring to this article: < http://www.cerias.purdue.edu/site/blog/post/beware_sql_injections_due_to_mis...
I had a private discussion with the author to clarify. In order to avoid spreading FUD, I'd like to draw your attention to this note he added to the bottom of the post as a result of that discussion: P.P.S.: To clarify, I haven't demonstrated an SQL injection vulnerability in pyPGSQL. It's not about the performance penalty either. It's about escaping done by the client library (the basic implementation of bind parameters without using the database's support) being a second-rate security solution to explicitly telling the database "here is the code. Now here's the data" (prepared statements). It's about decreasing code complexity and reducing chances for "misunderstandings" (and configuration, e.g., encoding, discrepancies). It's about assurance and choosing safer technologies and architectures. So, to be clear, I don't believe there are any *demonstrated* injection vulnerabilities (although if someone has a reference to a current one, that would be helpful); it's just that pyPgSQL goes with an unnecessarily risky and poorly-performing implementation strategy for quoting parameters. This dates back to Postgres itself having poor client-library support for bind parameters in versions previous to 7; unfortunately this means that most of the other database bindings for postgres use a similarly bad strategy (and as Mr. Meunier mentions in his post, the ones which do support bind parameters have other issues).

On Wed, Aug 12, 2009 at 11:28 AM, Pet <petshmidt@googlemail.com> wrote:
Hello,
does pyPgSQL doesn't support unicode?
Sounds like a question for the pyPgSQL list, not the Twisted list :) <https://lists.sourceforge.net/lists/listinfo/pypgsql-users>

On 03:28 pm, petshmidt@googlemail.com wrote:
I don't know about pyPgSQL's unicode support. I did recently learn that pyPgSQL doesn't support bind parameters, apparently resulting in almost any use of it insecure and vulnerable to SQL injection attacks. You may want to investigate this further before deciding if it's worth figuring out how unicode works. Jean-Paul

On Wed, Aug 12, 2009 at 12:51 PM, <exarkun@twistedmatrix.com> wrote:
I believe exarkun is referring to this article: < http://www.cerias.purdue.edu/site/blog/post/beware_sql_injections_due_to_mis...
I had a private discussion with the author to clarify. In order to avoid spreading FUD, I'd like to draw your attention to this note he added to the bottom of the post as a result of that discussion: P.P.S.: To clarify, I haven't demonstrated an SQL injection vulnerability in pyPGSQL. It's not about the performance penalty either. It's about escaping done by the client library (the basic implementation of bind parameters without using the database's support) being a second-rate security solution to explicitly telling the database "here is the code. Now here's the data" (prepared statements). It's about decreasing code complexity and reducing chances for "misunderstandings" (and configuration, e.g., encoding, discrepancies). It's about assurance and choosing safer technologies and architectures. So, to be clear, I don't believe there are any *demonstrated* injection vulnerabilities (although if someone has a reference to a current one, that would be helpful); it's just that pyPgSQL goes with an unnecessarily risky and poorly-performing implementation strategy for quoting parameters. This dates back to Postgres itself having poor client-library support for bind parameters in versions previous to 7; unfortunately this means that most of the other database bindings for postgres use a similarly bad strategy (and as Mr. Meunier mentions in his post, the ones which do support bind parameters have other issues).
participants (3)
-
exarkun@twistedmatrix.com
-
Glyph Lefkowitz
-
Pet