[Twisted-Python] adbapi/postgresql and xml-rpc
Have a design question to pass to the list. I've gotten my distributed app working - pulling snort data from a postgresql database via twisted's adbapi and handling via xml-rpc - though xml-rpc is screaming about not being able to serialize postgresql's timestamp format. For instance, this query works (using a query test client and a "take any SQL string in and process it" xmlrpc method on the server local to database): # snipped python code setting things up. here's the xmlrpc call: (answer,) = connection.remote("fetch", ('SELECT sid, cid, signature from event LIMIT 5',)) print answer prints: [[1, 1, 1], [1, 2, 2], [1, 3, 1], [1, 4, 3], [1, 5, 1]] (nice and easy to work with on client side) but by adding the timestamp to the select query (SELECT sid, cid, signature, timestamp...), I get the dreaded can't serialize output error: Traceback (most recent call last): File "x8.py", line 59, in ? (answer,) = connection.remote("fetch", ('SELECT sid, cid, signature, timestamp from event LIMIT 5',)) ## some deleted File "/usr/lib/python2.3/xmlrpclib.py", line 742, in close raise Fault(**self._stack[0]) xmlrpclib.Fault: <Fault 8002: "can't serialize output"> QUESTION: While I think I could build more precise queries on the server side, e.g. xmlrpc_getAllSIDbyDate and massage timestamp and other formats before dumping into xmlrpc, this might not be the most elegant method. It makes every query something both ends have to be recoded for - not just the client - simply because I can't get timestamp to pass through XMLRPC. I've tried converting the tuples from the adbapi output to strings before handing back but that has other issues. Is this a limitation of XMLRPC that would merit using an approach like PB instead? Eventually my text client will go the way of a Qt GUI and I thought XMLRPC would make a nice lightweight distributed interface, but the serialization issues are making me wonder if the approach is right. Jamie
James R. Saker Jr. wrote:
I also send adbapi results over xml-rpc, using strftime to convert the mxDateTimes to an ISO string format and then converting them back to mx.DateTimes with DateTime.ISO.ParseDateTime. Not elegant, but it does the job. Note that PB's jelly will also complain about DateTimes. Steve
Lì mercoledì, 2004/05/12 alle 16:30, -0400, Stephen Waterbury ha scritto:
why don't you people switch to psycopg nd register a custom type converter using new_type/register_type? you can easily nd automagically convert dates and times to strings with a 3-lines callable object. federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org We are all dust, Saqi, so play the lute We are all wind, Saqi, so bring wine. -- Omar Khayam
Federico Di Gregorio wrote:
Mine is part of a general type converter that I have (cook()/uncook()) that handles all types I need to serialize. psycopg might make the implementation of my cook/uncook functions slightly more concise, but right now they are independent of the specific dbapi that I use, so I might want to keep it that way (more componentized ;). - Steve
James R. Saker Jr. wrote:
I also send adbapi results over xml-rpc, using strftime to convert the mxDateTimes to an ISO string format and then converting them back to mx.DateTimes with DateTime.ISO.ParseDateTime. Not elegant, but it does the job. Note that PB's jelly will also complain about DateTimes. Steve
Lì mercoledì, 2004/05/12 alle 16:30, -0400, Stephen Waterbury ha scritto:
why don't you people switch to psycopg nd register a custom type converter using new_type/register_type? you can easily nd automagically convert dates and times to strings with a 3-lines callable object. federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org We are all dust, Saqi, so play the lute We are all wind, Saqi, so bring wine. -- Omar Khayam
Federico Di Gregorio wrote:
Mine is part of a general type converter that I have (cook()/uncook()) that handles all types I need to serialize. psycopg might make the implementation of my cook/uncook functions slightly more concise, but right now they are independent of the specific dbapi that I use, so I might want to keep it that way (more componentized ;). - Steve
participants (3)
-
Federico Di Gregorio -
James R. Saker Jr. -
Stephen Waterbury