Mailman 3 September 2008 - Twisted

[Twisted-Python] Couched 0.1: CouchDB twisted client
by Henrik Thostrup Jensen Sept. 10, 2008

Sept. 10, 2008

Hi I've written a small python/twisted client library for CouchDB. It's mostly a toy, but it can be fetched from here: http://www.cs.aau.dk/~htj/code/couched-0.1.tar.gz if anyone is interested. License: MIT I'm aware of paisley (https://launchpad.net/paisley), The main difference from paisley is that the connection and database abstraction are separated. Also couched does connection caching and uses web2 instead of web. Currently paisley is more feature rich, but couched has slightly nicer error handling. -- - Henrik

6 8

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Sept. 7, 2008

Sept. 7, 2008

1 0

[Twisted-Python] "_wait" is blowing up?!
by Brian Granger Sept. 6, 2008

Sept. 6, 2008

File "/usr/lib/python2.5/site-packages/twisted/trial/unittest.py", line 763, in run self._wait(d) File "/usr/lib/python2.5/site-packages/twisted/trial/unittest.py", line 901, in _wait reactor.run() File "/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py", Hi, We have a pretty big tests suite that uses trial to test all sorts of twisted things. We are pretty careful to do the right thing in our tests and leave the reactor clean....but recently we began seeing a transient really wierd bug when we run our test suite. Some characteristics of the problem: 1. The problem only appears sometimes - seems random 2. The problem shows up in different test methods 3. Sometimes it hangs the tests suite for a long time - eventually twisted continues though Here is the traceback we get. line 220, in run self.mainLoop() File "/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py", line 228, in mainLoop self.runUntilCurrent() --- <exception caught here> --- File "/usr/lib/python2.5/site-packages/twisted/internet/base.py", line 533, in runUntilCurrent f(*a, **kw) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 261, in errback self._startRunCallbacks(fail) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 290, in _startRunCallbacks raise AlreadyCalledError twisted.internet.defer.AlreadyCalledError: ERROR It appears that something is going on in the deadly _wait. Any thoughts on how we might fix this? Thanks Brian

3 4

Re: [Twisted-Python] Moderately-long running processes timing out in Twisted
by Steve Lewis Sept. 5, 2008

Sept. 5, 2008

We're using good_sql already. We're doing our best to move away from Oracle, honest! ;) Steve ----- Original Message ---- From: "glyph(a)divmod.com" <glyph(a)divmod.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Friday, September 5, 2008 2:22:08 PM Subject: Re: [Twisted-Python] Moderately-long running processes timing out in Twisted It seems that you have been a victim of at least one recently-fixed bug: http://twistedmatrix.com/trac/ticket/3244 and at least one long-standing one: http://twistedmatrix.com/trac/ticket/1753 As you can see from the tone of my (unfortunate) comment on the latter ticket, I don't think a lot of people use Oracle with Twisted. However, the issue is simple to hack around temporarily: set your ConnectionPool's "good_sql" attribute to 'select 1 from dual' as itamar suggests on #1753. Then, please submit a patch, and ideally a buildslave with a copy of oracle installed so that we can maintain support for it :). On 06:06 pm, spiritualmechanic(a)yahoo.com wrote: >I looked at the adbapi code and it appears the ConnectionLost exception >is coming from there, not from cx_Oracle. At least that's what is in >the error log: > >2008-09-05 12:25:48-0400 [HTTPChannel,9003,127.0.0.1] adbapi closing: >cx_Oracle >2008-09-05 12:25:48-0400 [HTTPChannel,9003,127.0.0.1] Connection lost. >2008-09-05 12:25:48-0400 [-] Unhandled Error > Traceback (most recent call last): > File "/opt/Python25/lib/python2.5/threading.py", line 440, in >run > self.__target(*self.__args, **self.__kwargs) > File "/opt/python/lib/python2.5/site- >packages/twisted/python/threadpool.py", line 161, in _worker > context.call(ctx, function, *args, **kwargs) > File "/opt/python/lib/python2.5/site- >packages/twisted/python/context.py", line 59, in callWithContext > return self.currentContext().callWithContext(ctx, func, >*args, **kw) > File "/opt/python/lib/python2.5/site- >packages/twisted/python/context.py", line 37, in callWithContext > return func(*args,**kw) > --- <exception caught here> --- > File "/opt/python/lib/python2.5/site- >packages/twisted/internet/threads.py", line 24, in _putResultInDeferred > result = f(*args, **kwargs) > File "/opt/python/lib/python2.5/site- >packages/twisted/enterprise/adbapi.py", line 380, in _runInteraction > conn.rollback() > File "/opt/python/lib/python2.5/site- >packages/twisted/enterprise/adbapi.py", line 65, in rollback > raise ConnectionLost() > twisted.enterprise.adbapi.ConnectionLost: > >I don't know what > > --- <exception caught here> --- > >means. I'm guessing there's some sort of issue with Oracle but our DBA >didn't see anything obvious in his logs. Is there a way for me to tell >what exactly the "exception caught here" is? > >Steve > > > >----- Original Message ---- >From: L. Daniel Burr <ldanielburr(a)mac.com> >To: Twisted general discussion <twisted-python(a)twistedmatrix.com> >Sent: Friday, September 5, 2008 1:15:40 PM >Subject: Re: [Twisted-Python] Moderately-long running processes timing >out in Twisted > >Hi Steve, > >On Fri, 05 Sep 2008 11:51:15 -0500, Steve Lewis ><spiritualmechanic(a)yahoo.com> wrote: >>We're trying to use twisted as a sort of bus, a centralized place to >>manage all of our SOA enpoints (mostly through XML-RPC). We have a few >>services that access the Oracle DB directly (adbapi). In production >>we're getting intermittent ConnectionLost exceptions from cx_Oracle. >>Most of these are queries that take a couple of seconds. > >Hmm, it is kind of hard to say what the problem could be, without >seeing the code that is using adbapi and cx_Oracle. > >When you say you are receiving ConnectionLost exceptions from >cx_Oracle, >are they *really* from cx_Oracle, i.e., "ORA-03135 - Connection >lost...", >or are they just the result of adbapi calling its connectionLost >method? >>Has anyone seen this kind of behavior before? What we're probably >>going >>to do in the short term is just remove twisted from the equation for >>these methods that seem to be timing out. But we're relatively new to >>twisted and I wonder if we're doing something that's obviously wrong. > >I've used cx_Oracle with twisted many times, with many different >workloads, >including stored procedures with OUT parameters, and it has always >worked >as expected. If you are able to provide some code, this list might be >able to provide you with more prescriptive advice. > >Hope this helps, > >L. Daniel Burr > >_______________________________________________ >Twisted-Python mailing list >Twisted-Python(a)twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > > > >_______________________________________________ >Twisted-Python mailing list >Twisted-Python(a)twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

3 2

Re: [Twisted-Python] Moderately-long running processes timing out in Twisted
by Steve Lewis Sept. 5, 2008

Sept. 5, 2008

I looked at the adbapi code and it appears the ConnectionLost exception is coming from there, not from cx_Oracle. At least that's what is in the error log: 2008-09-05 12:25:48-0400 [HTTPChannel,9003,127.0.0.1] adbapi closing: cx_Oracle 2008-09-05 12:25:48-0400 [HTTPChannel,9003,127.0.0.1] Connection lost. 2008-09-05 12:25:48-0400 [-] Unhandled Error Traceback (most recent call last): File "/opt/Python25/lib/python2.5/threading.py", line 440, in run self.__target(*self.__args, **self.__kwargs) File "/opt/python/lib/python2.5/site-packages/twisted/python/threadpool.py", line 161, in _worker context.call(ctx, function, *args, **kwargs) File "/opt/python/lib/python2.5/site-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/opt/python/lib/python2.5/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) --- <exception caught here> --- File "/opt/python/lib/python2.5/site-packages/twisted/internet/threads.py", line 24, in _putResultInDeferred result = f(*args, **kwargs) File "/opt/python/lib/python2.5/site-packages/twisted/enterprise/adbapi.py", line 380, in _runInteraction conn.rollback() File "/opt/python/lib/python2.5/site-packages/twisted/enterprise/adbapi.py", line 65, in rollback raise ConnectionLost() twisted.enterprise.adbapi.ConnectionLost: I don't know what --- <exception caught here> --- means. I'm guessing there's some sort of issue with Oracle but our DBA didn't see anything obvious in his logs. Is there a way for me to tell what exactly the "exception caught here" is? Steve ----- Original Message ---- From: L. Daniel Burr <ldanielburr(a)mac.com> To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Sent: Friday, September 5, 2008 1:15:40 PM Subject: Re: [Twisted-Python] Moderately-long running processes timing out in Twisted Hi Steve, On Fri, 05 Sep 2008 11:51:15 -0500, Steve Lewis <spiritualmechanic(a)yahoo.com> wrote: > We're trying to use twisted as a sort of bus, a centralized place to > manage all of our SOA enpoints (mostly through XML-RPC). We have a few > services that access the Oracle DB directly (adbapi). In production > we're getting intermittent ConnectionLost exceptions from cx_Oracle. > Most of these are queries that take a couple of seconds. > Hmm, it is kind of hard to say what the problem could be, without seeing the code that is using adbapi and cx_Oracle. When you say you are receiving ConnectionLost exceptions from cx_Oracle, are they *really* from cx_Oracle, i.e., "ORA-03135 - Connection lost...", or are they just the result of adbapi calling its connectionLost method? > Has anyone seen this kind of behavior before? What we're probably going > to do in the short term is just remove twisted from the equation for > these methods that seem to be timing out. But we're relatively new to > twisted and I wonder if we're doing something that's obviously wrong. > I've used cx_Oracle with twisted many times, with many different workloads, including stored procedures with OUT parameters, and it has always worked as expected. If you are able to provide some code, this list might be able to provide you with more prescriptive advice. Hope this helps, L. Daniel Burr _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

3 2

[Twisted-Python] Moderately-long running processes timing out in Twisted
by Steve Lewis Sept. 5, 2008

Sept. 5, 2008

We're trying to use twisted as a sort of bus, a centralized place to manage all of our SOA enpoints (mostly through XML-RPC). We have a few services that access the Oracle DB directly (adbapi). In production we're getting intermittent ConnectionLost exceptions from cx_Oracle. Most of these are queries that take a couple of seconds. Has anyone seen this kind of behavior before? What we're probably going to do in the short term is just remove twisted from the equation for these methods that seem to be timing out. But we're relatively new to twisted and I wonder if we're doing something that's obviously wrong. Steve

2 1

[Twisted-Python] XMPP authentication question : when is the username sent?
by Gabriel Rossetti Sept. 5, 2008

Sept. 5, 2008

Hello everyone, I am trying to figure out how the auth works, for this I used the simple example client. The question is, when is the username sent? In the output I never see the username being sent, I see the server at some point sending it to me. Thank you, Gabriel Username is : test_user@localhost Output : Connected. SEND: <stream:stream xmlns:stream='http://etherx.jabber.org/streams' xmlns='jabber:client' to='localhost' version='1.0'> RECV: <?xml version='1.0'?><stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' id='4203649965' from='localhost' version='1.0' xml:lang='en'> RECV: <stream:features><starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'/><mechanisms xmlns='urn:ietf:params:xml:ns:xmpp-sasl'><mechanism>DIGEST-MD5</mechanism><mechanism>PLAIN</mechanism></mechanisms><register xmlns='http://jabber.org/features/iq-register'/></stream:features> SEND: <starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'/> RECV: <proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'/> SEND: <stream:stream xmlns:stream='http://etherx.jabber.org/streams' xmlns='jabber:client' to='localhost' version='1.0'> RECV: <?xml version='1.0'?><stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' id='1885368444' from='localhost' version='1.0' xml:lang='en'> RECV: <stream:features><mechanisms xmlns='urn:ietf:params:xml:ns:xmpp-sasl'><mechanism>DIGEST-MD5</mechanism><mechanism>PLAIN</mechanism></mechanisms><register xmlns='http://jabber.org/features/iq-register'/></stream:features> SEND: <auth xmlns='urn:ietf:params:xml:ns:xmpp-sasl' mechanism='DIGEST-MD5'/> RECV: <challenge xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>bm9uY2U9IjE0NTM4NDY3NDMiLHFvcD0iYXV0aCIsY2hhcnNldD11dGYtOCxhbGdvcml0aG09bWQ1LXNlc3M=</challenge> SEND: <response xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>dXNlcm5hbWU9InRlc3RfdXNlciIsbm9uY2U9IjE0NTM4NDY3NDMiLGNoYXJzZXQ9dXRmLTgscmVhbG09ImxvY2FsaG9zdCIscW9wPWF1dGgsY25vbmNlPSJhN2RkOTY1MGFmNTA4MTU2YTc1ZjE3ODQwMzZmNTQ0OCIsbmM9MDAwMDAwMDEsZGlnZXN0LXVyaT0ieG1wcC9sb2NhbGhvc3QiLHJlc3BvbnNlPTg1MWZiZmMwYTczNWRmZDFiODZkNmMzZDg0MmI2NWQ3</response> RECV: <challenge xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>cnNwYXV0aD03ZWQ3ZWZmNTI2NWRmMTE2MmZlNTdjNGE0NDI1ZGEwMA==</challenge> SEND: <response xmlns='urn:ietf:params:xml:ns:xmpp-sasl'/> RECV: <success xmlns='urn:ietf:params:xml:ns:xmpp-sasl'/> SEND: <stream:stream xmlns:stream='http://etherx.jabber.org/streams' xmlns='jabber:client' to='localhost' version='1.0'> RECV: <?xml version='1.0'?><stream:stream xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' id='4179606135' from='localhost' version='1.0' xml:lang='en'> RECV: <stream:features><bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'/><session xmlns='urn:ietf:params:xml:ns:xmpp-session'/></stream:features> SEND: <iq type='set' id='H_0'><bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'/></iq> RECV: <iq id='H_0' type='result'><bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'><jid>test_user@localhost/26511133671220620792673712</jid></bind></iq> SEND: <iq type='set' id='H_1'><session xmlns='urn:ietf:params:xml:ns:xmpp-session'/></iq> RECV: <iq type='result' id='H_1'><session xmlns='urn:ietf:params:xml:ns:xmpp-session'/></iq> Authenticated.

2 2

[Twisted-Python] Twisted XMPP Roadmap
by Duncan McGreggor Sept. 4, 2008

Sept. 4, 2008

Hey all, I wanted to take a quick moment to share some stuff with the general email audience that is interested in XMPP/Jabber in Twisted. First off, some folks don't realize that there is a dedicated Twisted-Jabber email list for discussing XMPP/Jabber topics. This list is managed by Twisted's XMPP guru, Ralph Meijer. You can subscribe and check out the archives here: https://mailman.ik.nu/mailman/listinfo/twisted-jabber Secondly, we've been having some interesting discussions with Twisted sponsors lately about deep and abiding interests in Twisted XMPP: many sponsors want us to spend time creating docs and tutorials, improving support and adding new features. This interest was spearheaded by Jack Moffitt of Chesspark and given additional momentum by Dhris DiBona of Google. Furthermore, we've received vocal support for XMPP improvements from Hydra Labs, Lucasfilm, Security Metrics, and individual sponsors. Thirdly, today on #twisted, we had an intense and very productive discussion about XMPP in general, about it's current support in Twisted, and then a provisional roadmap for future developments. It was an excellent experience for all, and one that can be relived by everyone, thanks to IRC logs :-) A new ticket was created to address the next step that will to be taken, and that ticket is here: http://twistedmatrix.com/trac/ticket/3407 In that ticket, I linked to the notes and IRC log of our chat: http://twistedmatrix.com/trac/wiki/XMPPServerArchitecture http://twistedmatrix.com/trac/wiki/XMPPIRCTranscript20080903 There's lots to do and we're all quite excited about both the immediate and long-term benefits to be had by Twisted users, community members, and the project itself with regard to XMPP and messaging development efforts. Thanks, d

7 9

[Twisted-Python] FilePaging not working?
by Samson Yeung Sept. 3, 2008

Sept. 3, 2008

Hello all, I am trying to transfer files over a PerspectiveBroker connection with FilePager and I've run into a problem. The attached cut-down example works fine with Twisted-2.5.0, but not with Twisted-8.0 or Twisted-8.1.0. This is with python 2.5. On version 8 of twisted, a 0-byte file is created and the client never finished; but with version 2.5, the full file is transferred. What am I doing wrong; or is this a bug? Thanks all, -Samson

1 0

[Twisted-Python] create new domish.Element from string XML
by Gabriel Rossetti Sept. 3, 2008

Sept. 3, 2008

Hello everyone, I sometimes have to create domish.Element from a string and I can't find any way of doing this. I saw that I can add children from a string but not create a whole XML tree. Is there a way of doing this other than creating a dummy element, adding the XML string as a child and then extracting the child? Thank you, Gabriel

1 2