[Twisted-Python] using Twisted with postgresql
Hi, I'm new to Twisted and is currently looking for a way to asynchronously do queries to a postgres database. From what I've seen it looks like there's a project called pgasync (http://jamwt.com/pgasync/) that does this, or I can use Twisted's adbapi and some other synchronous module. I'm wondering if anyone has used pgasync and how it compares to using adbapi and threads? From the pgasync site it seems the last update was in 2005, so I'm not sure if anyone's still maintaining it. If using adbapi's better, can I get some suggestions as to which postgres module I should use? So far I've seen pyPgSQL and psychopg that're both DB API 2 compliant. Any help will be appreciated! Thanks, Doris
On 16 Oct 2007, at 13:40, Doris Lam wrote:
I'm new to Twisted and is currently looking for a way to asynchronously do queries to a postgres database. From what I've seen it looks like there's a project called pgasync (http:// jamwt.com/pgasync/) that does this, or I can use Twisted's adbapi and some other synchronous module. I'm wondering if anyone has used pgasync and how it compares to using adbapi and threads? From the pgasync site it seems the last update was in 2005, so I'm not sure if anyone's still maintaining it. If using adbapi's better, can I get some suggestions as to which postgres module I should use? So far I've seen pyPgSQL and psychopg that're both DB API 2 compliant. Any help will be appreciated!
I don't have any experience with PgAsync, but (while it's a very cool idea), I personally would prefer to use a database wrapper that wraps libpq - the standard, Postgres-supplied client library, shared with and debugged by nearly every other tool that talks to Postgres - rather than a wrapper that uses its own ground-up reimplementation. I also don't have any experience with pyPgSQL, but I have used psycopg2 and been impressed with its features and performance.
I'm using psycopg2 with dictionary cursor. Create module twistedpg.py in the python path(in my variant it's twisted's project directory) from psycopg2 import * from psycopg2 import _psycopg as _2psycopg from psycopg2.extensions import connection as _2connection from psycopg2.extras import DictCursor del connect def connect(*args, **kwargs): kwargs['connection_factory'] = connection return _2psycopg.connect(*args, **kwargs) class connection(_2connection): def cursor(self): return _2connection.cursor(self, cursor_factory=DictCursor) In a twisted's plugin i using standard ConnectionPool master_db = ConnectionPool('twistedpg', user='pron', password='xxx', host='127.0.0.1', database='playboy', port=5436, cp_max=5, cp_noisy=True, cp_reconnect=True) Tim Allen wrote:
On 16 Oct 2007, at 13:40, Doris Lam wrote:
I'm new to Twisted and is currently looking for a way to asynchronously do queries to a postgres database. From what I've seen it looks likes.here's a project called pgasync (http://jamwt.com/pgasync/) that does this, or I can use Twisted's adbapi and some other synchronous module. I'm wondering if anyone has used pgasync and how it compares to using adbapi and threads? From the pgasync site it seems the last update was in 2005, so I'm not sure if anyone's still maintaining it. If using adbapi's better, can I get some suggestions as to which postgres module I should use? So far I've seen pyPgSQL and psychopg that're both DB API 2 compliant. Any help will be appreciated!
I don't have any experience with PgAsync, but (while it's a very cool idea), I personally would prefer to use a database wrapper that wraps libpq - the standard, Postgres-supplied client library, shared with and debugged by nearly every other tool that talks to Postgres - rather than a wrapper that uses its own ground-up reimplementation.
I also don't have any experience with pyPgSQL, but I have used psycopg2 and been impressed with its features and performance. -- http://fats.sourceforge.net http://movinglights.org http://burus.org
Doris Lam ha scritto:
Hi,
I'm new to Twisted and is currently looking for a way to asynchronously do queries to a postgres database. From what I've seen it looks like there's a project called pgasync (http://jamwt.com/pgasync/) that does this, or I can use Twisted's adbapi and some other synchronous module.
Some time ago, I have refactored the pgasync implementation in pglib: http://hg.mperillo.ath.cx/twisted/pglib/ It offers an API for almost all the functionality of the Posgres protocol. The only thing missing is a connection pool manager, and support for prepared queries.
I'm wondering if anyone has used pgasync and how it compares to using adbapi and threads? From the pgasync site it seems the last update was in 2005, so I'm not sure if anyone's still maintaining it. If using adbapi's better, can I get some suggestions as to which postgres module I should use? So far I've seen pyPgSQL and psychopg that're both DB API 2 compliant. Any help will be appreciated!
A pure asynchronous implementation has better performance, in theory, since you don't need to use thread locks. Moreover, pglib can be very useful, as an example, if you want to "consume" each row at a time, or if you want to listen for database notifications. However using libpq has many advantages. You can also use the asynchronous interface that psycopg2 implements: http://www.initd.org/svn/psycopg/psycopg2/trunk/doc/async.txt However you *need* a good connection pool manager for asynchronous connections, and, as far as I know, no one has yet written such a thing. I'm interested in this topic, since I'm writing a WSGI implementation for Nginx and it, like Twisted, is asynchronous (and *without* threads support): http://hg.mperillo.ath.cx/nginx/mod_wsgi/ Regards Manlio Perillo
participants (4)
-
Burus Gmail
-
Doris Lam
-
Manlio Perillo
-
Tim Allen