Mailman 3 June 2009 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 21, 2009

June 21, 2009

1 0

Re: [Twisted-Python] AsyncQueue is still unavailable
by mardiros June 19, 2009

June 19, 2009

Hey, thanks for helping me. Steve Steiner (listsin) a écrit : I have a more current version of the code, but it would be better to go with something actively maintained. S That's exactly what I thint about too.... I am looking in Storm for the moment. I have copy the driver in the twisted-integration, but it tries with the both driver pymssql and adodbapi, and i think only the mxodbc worked. Now i execute deferred query with a modified one, and it work's fine. … [View More]

2 1

[Twisted-Python] Need help with writing a secure Iran Proxy
by Andreas Trawoeger June 19, 2009

June 19, 2009

Hi! Could I kindly ask all the Twisted Gurus to take a look Austin Heaps Best Proxy Practices for Iran http://blog.austinheap.com/ The current setup (Squid & iptables) everybody uses is extremely complicated to setup and it should be trivial to implement the requirements in Twisted. Turn it into an egg and allow everybody to simple type easy_install iranproxy. I know a little bit of Twisted, because I'm an avid Zenoss user and in different circumstanced I would write the proxy on my own. … [View More]

2 1

[Twisted-Python] AsyncQueue is still unavailable
by mardiros June 18, 2009

June 18, 2009

I everyone, I have too write a network application, and I think twisted is the best choice for it! I need to do some sql transaction, and I prefer too use an ORM too request the DB. But the server I have to connect is an SqlServer, so sqlalchemy is, I think the best choice. And every google search get back to AsyncQueue http://foss.eepatents.com/AsynQueue But the site is done since a few moment. Does anybody have news about this project ? Is it still alive ? Which alternative ? Thanks. … [View More]

4 3

[Twisted-Python] Dropin.cache errors after clean install of twisted-8.2.0
by Rutt, Benjamin June 18, 2009

June 18, 2009

I recently installed python-2.6.1 and then twisted-8.2.0 (outside of the python tree). One of my users is complaining that when he uses trial to test some twisted programs, although his test passes, he has to stare at a stack trace like this which mentions dropin.cache.new failing to be written: $ cat foo.py #!/sw/external/python-2.6.1/bin/python from twisted.trial import unittest class MyTestClass(unittest.TestCase): def test_foobar(self): print 'hello, world!' self.… [View More]failUnless(1 == 1) $ /sw/external/twisted-8.2.0/bin/trial ./foo.py /sw/external/twisted-8.2.0/lib/python/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys /sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha /sw/external/twisted-8.2.0/lib/python/twisted/web/microdom.py:181: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert (oldChild.parentNode is self, Traceback (most recent call last): File "/sw/external/twisted-8.2.0/lib/python/twisted/python/usage.py", line 241, in parseOptions self.postOptions() File "/sw/external/twisted-8.2.0/lib/python/twisted/scripts/trial.py", line 293, in postOptions self['reporter'] = self._loadReporterByName(self['reporter']) File "/sw/external/twisted-8.2.0/lib/python/twisted/scripts/trial.py", line 279, in _loadReporterByName for p in plugin.getPlugins(itrial.IReporter): File "/sw/external/twisted-8.2.0/lib/python/twisted/plugin.py", line 200, in getPlugins allDropins = getCache(package) --- <exception caught here> --- File "/sw/external/twisted-8.2.0/lib/python/twisted/plugin.py", line 179, in getCache dropinPath.setContent(pickle.dumps(dropinDotCache)) File "/sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py", line 623, in setContent f = sib.open('w') File "/sw/external/twisted-8.2.0/lib/python/twisted/python/filepath.py", line 433, in open return open(self.path, mode+'b') exceptions.IOError: [Errno 30] Read-only file system: '/sw/external/twisted-8.2.0/lib/python/twisted/plugins/dropin.cache.new' foo MyTestClass test_foobar ... hello, world! [OK] ------------------------------------------------------------------------ ------- Ran 1 tests in 0.023s PASSED (successes=1) I would consider this to be an install bug. Evidently, trial uses plugins, which upon reading http://twistedmatrix.com/projects/core/documentation/howto/plugin.html#a uto3 I find that plugins need up-to-date dropin.cache files to be built, or else they'll be built dynamically; the latter step fails in my case due to my site-wide install being on a read-only directory. The docs say "when installing or removing software which provides Twisted plugins, the site administrator should be sure the cache is regenerated"; however all I did was install twisted itself, not any specific plugins. All I did to install twisted was 'python setup.py install --home=...' and at no later point did I ever add new plugins to it. Why can't the install take care of initial dropin.cache generation for me? I see a related issue http://twistedmatrix.com/trac/ticket/2409. Regardless of whether ticket 2409 goes anywhere, should I raise an issue that the install should really take care of initial dropin.cache generation, much like python install takes care of std lib .py->.pyc generation? Thanks. [View Less]

4 7

[Twisted-Python] SerialPort.loseConnection() BUG
by Nestor A. Diaz June 17, 2009

June 17, 2009

Hello, there is a bug on the SerialPort Win32 implementation (don't know about the others), when i call the loseConnection method, it returns: twisted matrix exceptions.AttributeError: 'SerialPort' object has no attribute '_tempDataBuffer' if i wrap the serialport class and put a : self._tempDataBuffer = '' in the __init__ method, i get: raise NotImplementedError("%s does not implement writeSomeData" % exceptions.NotImplementedError: __main__.MySerialPort does not implement … [View More]

6 10

[Twisted-Python] Twisted perspective broker on port 80, problem
by hbd June 17, 2009

June 17, 2009

Is there any reason why I am getting this error when I try to get perspective broker to be on port 80? twisted.internet.error.CannotListenError: Couldn't listen on any:80: (13, 'Permission denied'). I made sure port 80 was not used by Apache or anything else by checking with netstat. Here is the relevant pieces of twisted code: application = twisted.application.service.Application("eventserver") rpcservice = TheService() flashservice = XMLSocket(FlashReceiver) myServiceCollection = … [View More]

6 6

[Twisted-Python] High Concurrency
by 陶艺夫 June 17, 2009

June 17, 2009

Hi, I want to use twisted to write a server which is to get purchase orders from about tens of thousands clients sent by wireless net work (tcp protocol) and store the orders into a DBMS system. I wonder if it is possible and how to deal with the high concurrency. any idea? thanks in advance!

2 1

[Twisted-Python] can addCallback accept list of deferreds?
by Pet June 16, 2009

June 16, 2009

Hi, I'm trying to process several items in callback function writeLog d.addCallback(self.writeLog,g=g) see below. Can writeLog return DeferredList instead of Deferred? WriteLog should run several inserts into db (some of which may fail). Or am I completely wrong here? Thanks, Pet def getData(self, *args, **kwargs ): try: g = MyClass(self.db) d = self.getResult(params=kwargs['params'], g=g) if d: d.addCallback(self.sendResult) d.… [View More]

2 1

Re: [Twisted-Python] Question about File Descriptors
by Andrew Francis June 15, 2009

June 15, 2009

Hi Itamar: --- On Mon, 6/15/09, twisted-python-request(a)twistedmatrix.com <twisted-python-request(a)twistedmatrix.com> wrote: > Message: 2 > Date: Mon, 15 Jun 2009 09:45:19 -0400 > From: Itamar Shtull-Trauring <itamar(a)itamarst.org> > Subject: Re: [Twisted-Python] Question about File > Descriptors > To: Twisted general discussion <twisted-python(a)twistedmatrix.com> > Message-ID: <1245073519.8754.107.camel@pintsize> > Content-Type: text/plain … [View More]

1 0

[Twisted-Python] Question about File Descriptors
by Andrew Francis June 15, 2009

June 15, 2009

Hi Folks: This is not a Twisted question per se but here goes. I am using Linux. I want to do some tests and I need to increase the numbers of file descriptors. What are the memory ramifications (i.e, I raise the max descriptor count to 65535)? Also should I move to the EPoll reactor? Thanks in advance. Cheers, Andrew

2 1

[Twisted-Python] serial port protocols
by Brendan Simon June 15, 2009

June 15, 2009

Can twisted be used to access a serial port on OS X, MS Windows, linux, etc ??? Is there any support for serial protocols (eg. BISYNC, PPP, etc) ??? Can twisted be used to create custom serial protocols ??? Thanks, Brendan.

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 14, 2009

June 14, 2009

1 0

[Twisted-Python] serial port and http program
by Brendan Simon June 13, 2009

June 13, 2009

I have a small program that I want to write in python with a wxpython gui. The program has to talk to a hand held device via a serial port and then talk to a http server which returns an xml stream/file, which is then parsed and information sent back to the hand held device over the serial port. Is there an advantage to using twisted for such an application ?? Is it more complicated to do so (i.e. using twisted versus not using twisted). Would it just be an added dependency that would be … [View More]

2 1

[Twisted-Python] Custom logging for twistd plugin
by Garret Heaton June 12, 2009

June 12, 2009

I'm looking to make some changes to the way a twistd plugin<http://twistedmatrix.com/projects/core/documentation/howto/plugin.html>logs information. The docs on logging<http://twistedmatrix.com/projects/core/documentation/howto/logging.html>and the application framework<http://twistedmatrix.com/projects/core/documentation/howto/application.html…>don't currently mention how plugins fit in. My main goal is to have the log include information about the log level of each message … [View More]

1 0

[Twisted-Python] stop conditions and deferToThread()
by Thomas Jakobsen June 12, 2009

June 12, 2009

Hi As discussed in a previous thread http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html a task put in its own thread via deferToThread() won't stop even though the reactor has stopped. It has to carry out its own check that the reactor is running and stop if that is not the case. My question is: Is it possible to use anything else than the reactor.running as stop condition in this way? In my current twisted application I would like to keep the reactor running but … [View More]

4 4

Re: [Twisted-Python] Need working examples of imap4 client.
by Pywinder Singh June 12, 2009

June 12, 2009

Tarjei, Yep. I actually grabbed it, but on perusal, but its use of twisted seemed a little too layered to help me at my current twisted expertise. Of course, now that I think I have the problem narrowed down to the login phase, it might help. Tnx. Tarjei wrote: > On 06/11/2009 02:03 AM, Pywinder Singh wrote: > > Can someone point me to a working example of imap4 client? > > > > I've tried the example off the site, code from Abe Fettig's book, and > > every example … [View More]

1 0

[Twisted-Python] Need working examples of imap4 client.
by Pywinder Singh June 11, 2009

June 11, 2009

Can someone point me to a working example of imap4 client? I've tried the example off the site, code from Abe Fettig's book, and every example I could find on the web. (most of them seem outdated, as do most discussions I've found on the subject) All of them invariably end with: twisted.internet.error.TimeoutError: User timeout caused connection failure or twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion. I've pointed the code at two … [View More]

3 2

[Twisted-Python] Application Design help - Concurrent but not Protocols based.
by Senthil Kumaran June 10, 2009

June 10, 2009

Hello Twisted Developers/Users, This is my first concurrent application design and my first trial with twisted. I have read the documentation and understand where twisted plays its part. Unfortunately, I could not directly relate it to my requirements and hence, could not go forward with designing and building on top using the examples as a reference. I need your guidance in helping me design an application. My Application Details: 1) I need to constantly monitor a particular directory for … [View More]

5 5

Re: [Twisted-Python] Application Design help - Concurrent but not Protocols based.
by Senthil Kumaran June 10, 2009

June 10, 2009

On Wed, Jun 03, 2009 at 10:43:34AM -0700, Minesh Patel wrote: > For this, since your data is huge and seems like it will need a lot of > CPU utilization, you might have to deferToThread, since Twisted runs > in a single thread and will block AFAIK. > > d = threads.deferToThread(do_a, arg1_to_do_a, arg2...) > d.addCallback(do_a2, ...) > I delayed and forgot to respond to this thread. Sorry for that. Actually, I had asked the question on doing asynchronous monitoring of … [View More]

1 0

[Twisted-Python] copy domish element
by Jack Bates June 9, 2009

June 9, 2009

i have a domish element - when i try the following, >>> message = element >>> del message['from'] >>> element['from'] - i get a key error to avoid this key error, i think i want to copy the element, so i can drop the 'from' attribute from one copy without affecting the other? but i didn't find anything to make a copy in the domish element api how should i avoid this key error?

3 3

[Twisted-Python] some callback function params problems
by hoooooosety June 9, 2009

June 9, 2009

Hello, I have some callback function params problems. Here is a simple example. def deferDoSomeInsert(paramA, paramB): insertSQL = "..." db.runQuery(insertSQL) def deferDoSomeDelete(paramA, paramB): deleteSQL = "..." db.runQuery(deleteSQL) #these two functions are individual and basic, and then, I need compound function which first insert then delete. #so def deferCompoundFunc(paramA, paramB): return deferDoSomeInsert.addCallback(deferDoSomeDelete, paramA, paramB) here the problem … [View More]

2 1

Re: [Twisted-Python] Twisted Words
by Alex Patino June 9, 2009

June 9, 2009

I am looking for a chat system to add to my Django-based chat site. I don't need something ready-made but I don't want to build it on a framework that is no longer supported. What's the status of the Twisted Words project? Regards, Alex On Mon, Jun 8, 2009 at 11:25 AM, Alex Patino <patino(a)gmail.com> wrote: > I am looking for a chat system to add to my Django-based chat site. I > don't need something ready-made but I don't want to build it on a framework > that is no … [View More]

3 2

Re: [Twisted-Python] inlineCallbacksDecorator
by Terry Jones June 8, 2009

June 8, 2009

>>>>> "Drew" == Drew Smathers <drew.smathers(a)gmail.com> writes: Drew> On Sat, Jun 6, 2009 at 9:41 PM, Terry Jones<terry(a)jon.es> wrote: >> One inconvenience with inlineCallbacks is that you might have some kind >> of processing you want done no matter how the function returns, or >> wherever an error occurs. Two solutions here are 1) to put try/except >> calls around your various yields, and/or to perhaps do something else >> with … [View More]various callbacks that might call defer.returnValue, or 2) expect >> each caller of your method to deal with the result. I don't like the >> first of those much (depending on the code), and don't like the second >> at all. Drew> How does inlineCallbacks preclude you from adding "processing you Drew> want done no matter how the function returns" or currying callbacks? Drew> From my understanding of the problem, this is already solved by Drew> attaching callbacks/errbacks in the regular way: Drew> @inlineCallbacks Drew> def foo(a, b, c): Drew> ... Drew> foo(a, b, c).addBoth(cb, a, b, c) Right, that's what I meant by the (2) in my paragraph. The reason that's not always what you want is that there may be things foo needs to get done before it returns, things that are best or can only be computed inside foo. In that case you don't really want to make all your callers have to add a callback to add functionality/logic that belongs inside foo. Under normal circumstances you'd have code like this: def foo(): d = someOtherFunc() d.addCallbacks(...) return d or def foo(): d = defer.Callback() # Set up some stuff that will fire d return d so the deferred that comes back from foo is under control of foo and if there's any logical last steps to be done to it, foo can easily add them. But with inlineCallbacks, foo never has its hands on the deferred that will be returned. Yes, you could do: @inlineCallbacks def foo(): try: # # All sorts of stuff, yielding deferreds # except ...: # yield somethingElse or raise something But that's not so attractive, and it only deals with errbacks. To deal with all returns, you could catch the undocumented _DefGen_Return, do your processing and then re-raise it but that's even more ugly. So I'm just trying to offer a clean way to always get hold of the deferred and be able to consistently process it using code in the same file (or class, given a self) as foo. Terry [View Less]

1 0

[Twisted-Python] inlineCallbacksDecorator
by Terry Jones June 8, 2009

June 8, 2009

I try to avoid using inlineCallbacks. There are two cases where I will happily use it though: when I write a method that would need more than a few callback functions, and, more importantly, when the logical flow of a method is non-trivial (i.e., it depends on the returned values or errors of several deferreds). One inconvenience with inlineCallbacks is that you might have some kind of processing you want done no matter how the function returns, or wherever an error occurs. Two solutions here … [View More]are 1) to put try/except calls around your various yields, and/or to perhaps do something else with various callbacks that might call defer.returnValue, or 2) expect each caller of your method to deal with the result. I don't like the first of those much (depending on the code), and don't like the second at all. So I wrote a decorator specifically for inlineCallbacks decorated functions: from twisted.internet import defer def inlineCallbacksDecorator(callback, errback=defer.passthru): def wrap(f): def wrapper(*args, **kw): d = f(*args, **kw) if isinstance(d, defer.Deferred): return d.addCallbacks(callback, errback, callbackArgs=args, callbackKeywords=kw, errbackArgs=args, errbackKeywords=kw) # We were used to decorate a non-inlineCallbacks function. raise Exception( 'Function %r was decorated with inlineCallbacksDecorator but ' 'did not return a deferred (did you forget to also decorate ' 'with inlineCallbacks?)' % f.__name__) return mergeFunctionMetadata(f, wrapper) return wrap You can use it like this: def _cbok(result, a, b=None): print 'In _cbok: a=%r b=%r' % (a, b) return result def _cberr(failure, *args, **kw): pass # Do something.... and maybe also return the failure. @inlineCallbacksDecorator(_cbok, _cberr) @defer.inlineCallbacks def frog(a, b=None): print 'a = %r, b = %r' % (a, b) result = yield produceDeferred() # yield produceErr() defer.returnValue(a) The nice/interesting thing about this is that the callback and errback functions get called with the deferred result of calling frog (as decorated by inlineCallbacks) *and* the original arguments passed to frog. You can of course ignore the original args if you're not interested in them. You can argue that this doesn't really buy you anything. That's of course true. It's only a decorator that makes it neater to do certain things. Logging, accounting and error processing come immediately to mind. Just keeping code looking simpler/cleaner is enough of a reason for me. After writing this, I realized it deals with what I was clumsily trying to achieve here http://twistedmatrix.com/pipermail/twisted-python/2009-April/019492.html back in April. Terry [View Less]

3 2