[Twisted-Python] deferreds, and .rpy resources
On Thu, 23 Jan 2003, Mario Ruggier wrote:
Hi, sorry for the newbie questions but I do not understand how I can combine the resource interface with deferred callbacks. Or, specifically, can anyone point me to a simple example of a .rpy that does a simple database call, and returns the result to the http response?
Many thanks,
Mario
I'm no expert, but you can do something like this (UNTESTED): ----------------------------------------------------------------- # replace this import with whatever DBAPI module you use... import cx_Oracle # twisted imports from twisted.internet import threads from twisted.web import resource from twisted.web.server import NOT_DONE_YET def databaseOperation(): con = cx_Oracle.connect("foo/bar@mydb.mysvr.org") cur = con.cursor() cur.execute("SELECT COUNT(*) FROM MyTable") databaseResult = cur.fetchall() cur.close() con.close() return databaseResult[0][0] def databaseResult(result, httpRequest): httpRequest.write(result) httpRequest.finish() class MyResource(resource.Resource): def render(self, request): d = threads.deferToThread(databaseOperation) d.addCallback(databaseResult, request) return NOT_DONE_YET resource = MyResource() ----------------------------------------------------------------- Of course, you could also use twisted.enterprise.adbapi, but I've experienced a lot of memory leaks occurring when going that route, so I just use deferToThread instead. There is probably a better way to code this, but I'm not far enough along in my Python/Twisted experience to provide it. L. Daniel Burr
On Thu, 2003-01-23 at 17:25, L. Daniel Burr wrote:
Of course, you could also use twisted.enterprise.adbapi, but I've experienced a lot of memory leaks occurring when going that route, so I just use deferToThread instead.
I'm about to start using t.e.adbapi pretty heavily, so I'm interested in this report. Do you have more details? thanks, dave
Thanks a lot Daniel, that was very helpful. The code you sent works almost verbatim on a test table in postgres. I have also tried mapped the same solution to adbapi. Tthe rpy code for both versions is below. To switch between the 2 just toggle commenting the resource defns. In the adbapi version, I have some questions: 1. First of all, would this be the correct way to do this with adbapi? 2. If I stop the reactor at the end of databaseResult_adbapi, twistdweb returns the http request with correct content, but crashes (consistently on each request). 3. If i just import the reactor privately inside the render() method (thus, never call stop() on it), everything seems to work fine. But is this the way it should be done? Thanks again, mario
Hi, sorry for the newbie questions but I do not understand how I can combine the resource interface with deferred callbacks. Or, specifically, can anyone point me to a simple example of a .rpy that does a simple database call, and returns the result to the http response?
Many thanks,
Mario
I'm no expert, but you can do something like this (UNTESTED):
...
Of course, you could also use twisted.enterprise.adbapi, but I've experienced a lot of memory leaks occurring when going that route, so I just use deferToThread instead.
There is probably a better way to code this, but I'm not far enough along in my Python/Twisted experience to provide it.
L. Daniel Burr
======================== # common twisted imports from twisted.web import resource from twisted.web.server import NOT_DONE_YET dbname = 'db' dbuser = 'user' dbpass = 'pw' ### dbapi def databaseResult(result, httpRequest): httpRequest.write(str(result)) httpRequest.finish() def errBack(err, httpRequest): httpRequest.write('error: ' + str(err) ) httpRequest.finish() def databaseOperation(): from pyPgSQL import PgSQL con = PgSQL.connect(database=dbname,user=dbuser,password=dbpass) cur = con.cursor() cur.execute("SELECT foo_id,name,weight FROM foo") databaseResult = cur.fetchall() cur.close() con.close() return databaseResult #[0][0] class MyResource(resource.Resource): def render(self, request): from twisted.internet import threads d = threads.deferToThread(databaseOperation) d.addCallback(databaseResult, request) d.addErrback(errBack, request) return NOT_DONE_YET ### adpapi from twisted.internet import reactor from twisted.enterprise import row class FooRow(row.RowObject): rowColumns = [ ('foo_id', 'int'), ('name', 'varchar'), ('weight', 'int'), ] rowKeyColumns = [('foo_id', 'int4')] rowTableName = 'foo' #rowFactoryMethod = ['testFactoryMethod'] def databaseResult_adbapi(result, httpRequest): if result: for foo in result: httpRequest.write('foo: '+str(foo.__dict__)+'<br/>') else: httpRequest.write('stop:') httpRequest.finish() #reactor.stop() def errBack_adbapi(err, httpRequest): httpRequest.write('error: '+str(err) ) httpRequest.finish() #reactor.stop() class MyResource_adbapi(resource.Resource): def render(self, request): #from twisted.internet import reactor from twisted.enterprise import adbapi, reflector from twisted.enterprise.sqlreflector import SQLReflector dbpool = adbapi.ConnectionPool('pyPgSQL.PgSQL',database=dbname,user=dbuser,passwo rd=dbpass) r = SQLReflector(dbpool,[FooRow]) d = r.loadObjectsFrom('foo',whereClause=[('foo_id', reflector.LESSTHAN, 5)]) d.addCallback(databaseResult_adbapi, request) d.addErrback(errBack_adbapi, request) reactor.run() return NOT_DONE_YET ### #resource = MyResource() resource = MyResource_adbapi() ### ========================
On Fri, 24 Jan 2003 11:14:56 +0100
Mario Ruggier
3. If i just import the reactor privately inside the render() method (thus, never call stop() on it), everything seems to work fine. But is this the way it should be done?
Yes. I'm not even sure why you need to get at the reactor at all.
On Fri, 24 Jan 2003 11:14:56 +0100 Mario Ruggier
wrote: 3. If i just import the reactor privately inside the render() method (thus, never call stop() on it), everything seems to work fine. But is this the way it should be done?
Yes. I'm not even sure why you need to get at the reactor at all.
Easy. Because i am morphing the example code bits around, to make the same functionality addressable via an http request, and i swear that when i tried removing he reactor (as i imagined it was not needed in such a case) it **did** not work If i do not work! Now you point it out I have tried removing it again, and now it does work just fine!!! I suggest that one or both of these (corrected) examples be added to the list of examples bundled with twisted, as i would expect that many prospective users will not find this very obvious. If i knew what i was doing i would offer to do it myself... All the best, mario
participants (4)
-
Dave Peticolas
-
Itamar Shtull-Trauring
-
L. Daniel Burr
-
Mario Ruggier