[Twisted-Python] deferreds, and .rpy resources
On Thu, 23 Jan 2003, Mario Ruggier wrote:
I'm no expert, but you can do something like this (UNTESTED): ----------------------------------------------------------------- # replace this import with whatever DBAPI module you use... import cx_Oracle # twisted imports from twisted.internet import threads from twisted.web import resource from twisted.web.server import NOT_DONE_YET def databaseOperation(): con = cx_Oracle.connect("foo/bar@mydb.mysvr.org") cur = con.cursor() cur.execute("SELECT COUNT(*) FROM MyTable") databaseResult = cur.fetchall() cur.close() con.close() return databaseResult[0][0] def databaseResult(result, httpRequest): httpRequest.write(result) httpRequest.finish() class MyResource(resource.Resource): def render(self, request): d = threads.deferToThread(databaseOperation) d.addCallback(databaseResult, request) return NOT_DONE_YET resource = MyResource() ----------------------------------------------------------------- Of course, you could also use twisted.enterprise.adbapi, but I've experienced a lot of memory leaks occurring when going that route, so I just use deferToThread instead. There is probably a better way to code this, but I'm not far enough along in my Python/Twisted experience to provide it. L. Daniel Burr
Thanks a lot Daniel, that was very helpful. The code you sent works almost verbatim on a test table in postgres. I have also tried mapped the same solution to adbapi. Tthe rpy code for both versions is below. To switch between the 2 just toggle commenting the resource defns. In the adbapi version, I have some questions: 1. First of all, would this be the correct way to do this with adbapi? 2. If I stop the reactor at the end of databaseResult_adbapi, twistdweb returns the http request with correct content, but crashes (consistently on each request). 3. If i just import the reactor privately inside the render() method (thus, never call stop() on it), everything seems to work fine. But is this the way it should be done? Thanks again, mario
...
======================== # common twisted imports from twisted.web import resource from twisted.web.server import NOT_DONE_YET dbname = 'db' dbuser = 'user' dbpass = 'pw' ### dbapi def databaseResult(result, httpRequest): httpRequest.write(str(result)) httpRequest.finish() def errBack(err, httpRequest): httpRequest.write('error: ' + str(err) ) httpRequest.finish() def databaseOperation(): from pyPgSQL import PgSQL con = PgSQL.connect(database=dbname,user=dbuser,password=dbpass) cur = con.cursor() cur.execute("SELECT foo_id,name,weight FROM foo") databaseResult = cur.fetchall() cur.close() con.close() return databaseResult #[0][0] class MyResource(resource.Resource): def render(self, request): from twisted.internet import threads d = threads.deferToThread(databaseOperation) d.addCallback(databaseResult, request) d.addErrback(errBack, request) return NOT_DONE_YET ### adpapi from twisted.internet import reactor from twisted.enterprise import row class FooRow(row.RowObject): rowColumns = [ ('foo_id', 'int'), ('name', 'varchar'), ('weight', 'int'), ] rowKeyColumns = [('foo_id', 'int4')] rowTableName = 'foo' #rowFactoryMethod = ['testFactoryMethod'] def databaseResult_adbapi(result, httpRequest): if result: for foo in result: httpRequest.write('foo: '+str(foo.__dict__)+'<br/>') else: httpRequest.write('stop:') httpRequest.finish() #reactor.stop() def errBack_adbapi(err, httpRequest): httpRequest.write('error: '+str(err) ) httpRequest.finish() #reactor.stop() class MyResource_adbapi(resource.Resource): def render(self, request): #from twisted.internet import reactor from twisted.enterprise import adbapi, reflector from twisted.enterprise.sqlreflector import SQLReflector dbpool = adbapi.ConnectionPool('pyPgSQL.PgSQL',database=dbname,user=dbuser,passwo rd=dbpass) r = SQLReflector(dbpool,[FooRow]) d = r.loadObjectsFrom('foo',whereClause=[('foo_id', reflector.LESSTHAN, 5)]) d.addCallback(databaseResult_adbapi, request) d.addErrback(errBack_adbapi, request) reactor.run() return NOT_DONE_YET ### #resource = MyResource() resource = MyResource_adbapi() ### ========================
Easy. Because i am morphing the example code bits around, to make the same functionality addressable via an http request, and i swear that when i tried removing he reactor (as i imagined it was not needed in such a case) it **did** not work If i do not work! Now you point it out I have tried removing it again, and now it does work just fine!!! I suggest that one or both of these (corrected) examples be added to the list of examples bundled with twisted, as i would expect that many prospective users will not find this very obvious. If i knew what i was doing i would offer to do it myself... All the best, mario
Thanks a lot Daniel, that was very helpful. The code you sent works almost verbatim on a test table in postgres. I have also tried mapped the same solution to adbapi. Tthe rpy code for both versions is below. To switch between the 2 just toggle commenting the resource defns. In the adbapi version, I have some questions: 1. First of all, would this be the correct way to do this with adbapi? 2. If I stop the reactor at the end of databaseResult_adbapi, twistdweb returns the http request with correct content, but crashes (consistently on each request). 3. If i just import the reactor privately inside the render() method (thus, never call stop() on it), everything seems to work fine. But is this the way it should be done? Thanks again, mario
...
======================== # common twisted imports from twisted.web import resource from twisted.web.server import NOT_DONE_YET dbname = 'db' dbuser = 'user' dbpass = 'pw' ### dbapi def databaseResult(result, httpRequest): httpRequest.write(str(result)) httpRequest.finish() def errBack(err, httpRequest): httpRequest.write('error: ' + str(err) ) httpRequest.finish() def databaseOperation(): from pyPgSQL import PgSQL con = PgSQL.connect(database=dbname,user=dbuser,password=dbpass) cur = con.cursor() cur.execute("SELECT foo_id,name,weight FROM foo") databaseResult = cur.fetchall() cur.close() con.close() return databaseResult #[0][0] class MyResource(resource.Resource): def render(self, request): from twisted.internet import threads d = threads.deferToThread(databaseOperation) d.addCallback(databaseResult, request) d.addErrback(errBack, request) return NOT_DONE_YET ### adpapi from twisted.internet import reactor from twisted.enterprise import row class FooRow(row.RowObject): rowColumns = [ ('foo_id', 'int'), ('name', 'varchar'), ('weight', 'int'), ] rowKeyColumns = [('foo_id', 'int4')] rowTableName = 'foo' #rowFactoryMethod = ['testFactoryMethod'] def databaseResult_adbapi(result, httpRequest): if result: for foo in result: httpRequest.write('foo: '+str(foo.__dict__)+'<br/>') else: httpRequest.write('stop:') httpRequest.finish() #reactor.stop() def errBack_adbapi(err, httpRequest): httpRequest.write('error: '+str(err) ) httpRequest.finish() #reactor.stop() class MyResource_adbapi(resource.Resource): def render(self, request): #from twisted.internet import reactor from twisted.enterprise import adbapi, reflector from twisted.enterprise.sqlreflector import SQLReflector dbpool = adbapi.ConnectionPool('pyPgSQL.PgSQL',database=dbname,user=dbuser,passwo rd=dbpass) r = SQLReflector(dbpool,[FooRow]) d = r.loadObjectsFrom('foo',whereClause=[('foo_id', reflector.LESSTHAN, 5)]) d.addCallback(databaseResult_adbapi, request) d.addErrback(errBack_adbapi, request) reactor.run() return NOT_DONE_YET ### #resource = MyResource() resource = MyResource_adbapi() ### ========================
Easy. Because i am morphing the example code bits around, to make the same functionality addressable via an http request, and i swear that when i tried removing he reactor (as i imagined it was not needed in such a case) it **did** not work If i do not work! Now you point it out I have tried removing it again, and now it does work just fine!!! I suggest that one or both of these (corrected) examples be added to the list of examples bundled with twisted, as i would expect that many prospective users will not find this very obvious. If i knew what i was doing i would offer to do it myself... All the best, mario
participants (4)
-
Dave Peticolas -
Itamar Shtull-Trauring -
L. Daniel Burr -
Mario Ruggier