Mailman 3 June 2009 - Twisted

[Twisted-Python] adbapi and multiple queries in single transaction.
by Vishal Shetye June 23, 2009

June 23, 2009

Hi, I am working on an application that uses twisted and twisted.e.adbapi with sqlite in the backend. All db queries are put in separate functions in class say tableGateway.py which uses runOperation, runQuery etc as appropriate. A layer that uses it, lets say objectGateway.py, defines operations. Thus, lets say an add operation may actually call 2-3 or more functions from tableGateway.py. If my application quits or crashes in middle of an operation then it is likely that db will be inconsistent. What I want is each objectGateway level operation should be done in a single transaction. I initially thought of putting all the queries per operation in a runInteraction. However this results in code-duplication as many queries are shared between different operations. What I could think of for preventing this is: 1. do a runInteraction at objectGateway and then txn that I get will be passed to appropriate functions of tableGateway. 2. An objectGateway level log based recovery. Option 1 requires all the tableGateway functions to take txn as param. This can be avoided by instantiating tableGateway with a txn for each operation, however tableGateway instantiation is not that cheap. Even so, if any objectGateway function calls other objectGateway function then the txn will have to be passed to that as well. Any comments/suggestions? Regards, vishal DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

2 1

[Twisted-Python] Fwd: SerialPort.loseConnection() in Win32, workaround
by biziap biziap June 23, 2009

June 23, 2009

Hi, I have finished the neat version of using the SerialPort in Win32. Without the bothering loseConnection tracebacks. First of all, I have to apology for the mistake in my last post. I claimed that calling the flashInput and flashOutput can be used as the workaround.I am wrong. The way I use with SerialPort is different from the standard way: SerialPort(protocol_instance, comport, reactor, baudrate) instead, I create a SerialConnect and connect the serial port by: connector = reactor.connectWith(SerialConnector,comport,baudrate,factory) I welcome someone else to improve this connector for it to be more reliable if this approach is worthy. For the script, please see the attachment. Iap BTW: I am not sure if it is proper to attach the script in mail list. I tried to paste it in the mail body. After pasting it into the gmail's editor, the format of the script is terrible.

1 0

[Twisted-Python] twisted.web2 only can serve one connection at the same time when using scgi?
by Chris June 23, 2009

June 23, 2009

Hi everyone, I am trying to use apache2 + scgi to write a application,I found twisted.web2 supports the scgi module.I began to use this library to write some code to test how to use it,just like this: from twisted.web2 import server, channel, resource,http,http_headers from twisted.application import service, strports from twisted.internet import reactor, defer class Test(resource.Resource): def getResponse(self, result = None): rep = "The output which shall be transferred to the browser." res = http.Response( 200, {'content-type': http_headers.MimeType('text','plain')}, rep) print "---- getResponse returning: ", res return res def render(self,request): print 'a new request is coming' d = defer.Deferred() import time d.addCallback(lambda x:x).addCallback(self.getResponse) reactor.callLater(10,d.callback,0) return d toplevel = Test() site = server.Site(toplevel) application = service.Application("demoserver") s = strports.service('tcp:8888', channel.SCGIFactory(site)) s.setServiceParent(application) in the render function,I used a defer to simulate a time-consuming operation which takes ten seconds.IF in twisted.web,I can first return a server.NOT_DONE_YET,and use request.write() and request.finish() when the time-consuming operation is done(still use defer to do this).but twisted.web2 only supports returning a iweb.IResponse.it seems returning a defer is the only way. _|| <http://python.net/crew/mwh/apidocs/twisted.web2.iweb.IResponse.html>_ here is the output: 2009-06-23 15:45:11+0800 [-] Log opened. 2009-06-23 15:45:11+0800 [-] twistd 8.2.0 (/usr/bin/python 2.5.2) starting up. 2009-06-23 15:45:11+0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2009-06-23 15:45:11+0800 [-] twisted.web2.channel.scgi.SCGIFactory starting on 8888 2009-06-23 15:45:11+0800 [-] Starting factory <twisted.web2.channel.scgi.SCGIFactory instance at 0xf213f8> 2009-06-23 15:45:26+0800 [SCGIChannelRequest,0,192.168.0.138] a new request is coming 2009-06-23 15:45:36+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> 2009-06-23 15:45:36+0800 [SCGIChannelRequest,1,192.168.0.138] a new request is coming 2009-06-23 15:45:46+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> 2009-06-23 15:45:46+0800 [SCGIChannelRequest,2,192.168.0.138] a new request is coming 2009-06-23 15:45:56+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> I use the 3 browsers to access the URL within 1 second,but as we all see in the log,the server accepts the requests one by one,that is really not acceptable.I checked a little about the twisted.web2 source code, this render function is in a defer's callbacks chain,when I return a defer in a defer's callbacks chain,sure,the rest of the callbacks will wait for my defer to finish ,even I run it with reactor.callLater.so far,I feel I can do nothing about this,anyone can help me about this? Regards Chris

1 0

[Twisted-Python] SIP Registrar
by Guenter Dannoritzer June 22, 2009

June 22, 2009

Hi, I am trying to run a SIP registrar. So far I used an example that comes with the Perl Net:Sip Package: http://search.cpan.org/~sullr/Net-SIP/ As I am more familiar with Python, but not yet with twisted, I tried to find a simple example that runs a SIP registrar based on twisted. I searched this mailing list and found a post about rather using the Divmod Sine sip implementation than this one. On the other side it seems like that some of this functionality is back ported into twisted. Does the current twisted implementation provide enough functionality to implement a SIP registrar? I found in the mailing list this simple example about a SIP proxy: http://twistedmatrix.com/pipermail/twisted-python/2008-July/018047.html Would that be about the way to go for a registrar? Thanks for your help. Cheers, Guenter

2 1

[Twisted-Python] adbapi : sqlite3.OperationalError : database is locked
by Gabriel Rossetti June 22, 2009

June 22, 2009

Hello everyone, I am using twisted.enterprise.adbapi with sqlite3 database adapter and "runInteraction()". I keep on getting "sqlite3.OperationalError : database is locked" errors. I understand the problem, multiple threads try to access the DB and it being already locked by a thread it raises this exception. My question is what is the standard/correct Twisted way of dealing with this? Thank you, Gabriel

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 21, 2009

June 21, 2009

1 0

Re: [Twisted-Python] AsyncQueue is still unavailable
by mardiros June 19, 2009

June 19, 2009

Hey, thanks for helping me. Steve Steiner (listsin) a écrit : I have a more current version of the code, but it would be better to go with something actively maintained. S That's exactly what I thint about too.... I am looking in Storm for the moment. I have copy the driver in the twisted-integration, but it tries with the both driver pymssql and adodbapi, and i think only the mxodbc worked. Now i execute deferred query with a modified one, and it work's fine. I will try orm feature now :-) Créez votre adresse électronique prenom.nom(a)laposte.net 1 Go d'espace de stockage, anti-spam et anti-virus intégrés.

2 1

[Twisted-Python] Need help with writing a secure Iran Proxy
by Andreas Trawoeger June 19, 2009

June 19, 2009

Hi! Could I kindly ask all the Twisted Gurus to take a look Austin Heaps Best Proxy Practices for Iran http://blog.austinheap.com/ The current setup (Squid & iptables) everybody uses is extremely complicated to setup and it should be trivial to implement the requirements in Twisted. Turn it into an egg and allow everybody to simple type easy_install iranproxy. I know a little bit of Twisted, because I'm an avid Zenoss user and in different circumstanced I would write the proxy on my own. But in this case a safe implementation is a real must and I'm really worried to make some stupid beginner mistakes. So some (example) code and links to websites I should read would be greatly appreciated. Thanks to everybody! cu andreas

2 1

[Twisted-Python] AsyncQueue is still unavailable
by mardiros June 18, 2009

June 18, 2009

I everyone, I have too write a network application, and I think twisted is the best choice for it! I need to do some sql transaction, and I prefer too use an ORM too request the DB. But the server I have to connect is an SqlServer, so sqlalchemy is, I think the best choice. And every google search get back to AsyncQueue http://foss.eepatents.com/AsynQueue But the site is done since a few moment. Does anybody have news about this project ? Is it still alive ? Which alternative ? Thanks. mardiros Créez votre adresse électronique prenom.nom(a)laposte.net 1 Go d'espace de stockage, anti-spam et anti-virus intégrés.

4 3

[Twisted-Python] Dropin.cache errors after clean install of twisted-8.2.0
by Rutt, Benjamin June 18, 2009

June 18, 2009

I recently installed python-2.6.1 and then twisted-8.2.0 (outside of the python tree). One of my users is complaining that when he uses trial to test some twisted programs, although his test passes, he has to stare at a stack trace like this which mentions dropin.cache.new failing to be written: $ cat foo.py #!/sw/external/python-2.6.1/bin/python from twisted.trial import unittest class MyTestClass(unittest.TestCase): def test_foobar(self): print 'hello, world!' self.failUnless(1 == 1) $ /sw/external/twisted-8.2.0/bin/trial ./foo.py /sw/external/twisted-8.2.0/lib/python/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys /sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha /sw/external/twisted-8.2.0/lib/python/twisted/web/microdom.py:181: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert (oldChild.parentNode is self, Traceback (most recent call last): File "/sw/external/twisted-8.2.0/lib/python/twisted/python/usage.py", line 241, in parseOptions self.postOptions() File "/sw/external/twisted-8.2.0/lib/python/twisted/scripts/trial.py", line 293, in postOptions self['reporter'] = self._loadReporterByName(self['reporter']) File "/sw/external/twisted-8.2.0/lib/python/twisted/scripts/trial.py", line 279, in _loadReporterByName for p in plugin.getPlugins(itrial.IReporter): File "/sw/external/twisted-8.2.0/lib/python/twisted/plugin.py", line 200, in getPlugins allDropins = getCache(package) --- <exception caught here> --- File "/sw/external/twisted-8.2.0/lib/python/twisted/plugin.py", line 179, in getCache dropinPath.setContent(pickle.dumps(dropinDotCache)) File "/sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py", line 623, in setContent f = sib.open('w') File "/sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py", line 433, in open return open(self.path, mode+'b') exceptions.IOError: [Errno 30] Read-only file system: '/sw/external/twisted-8.2.0/lib/python/twisted/plugins/dropin.cache.new' foo MyTestClass test_foobar ... hello, world! [OK] ------------------------------------------------------------------------ ------- Ran 1 tests in 0.023s PASSED (successes=1) I would consider this to be an install bug. Evidently, trial uses plugins, which upon reading http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#a uto3 I find that plugins need up-to-date dropin.cache files to be built, or else they'll be built dynamically; the latter step fails in my case due to my site-wide install being on a read-only directory. The docs say "when installing or removing software which provides Twisted plugins, the site administrator should be sure the cache is regenerated"; however all I did was install twisted itself, not any specific plugins. All I did to install twisted was 'python setup.py install --home=...' and at no later point did I ever add new plugins to it. Why can't the install take care of initial dropin.cache generation for me? I see a related issue http://twistedmatrix.com/trac/ticket/2409. Regardless of whether ticket 2409 goes anywhere, should I raise an issue that the install should really take care of initial dropin.cache generation, much like python install takes care of std lib .py->.pyc generation? Thanks.

4 7