Mailman 3 October 2009 - Twisted

[Twisted-Python] reactor.run() do not return after reactor.stop()
by Stanislav Yudin 10 Oct '09

10 Oct '09

Hi everyone, I am having a problem with current twisted trunk (rev 27375). I am trying to implement simple ssh client with conch.ssh but right now my implementation works wrong way. I've made a wrapper class for my client factory, to separate host/creds information and command I want to execute. So in execute method of the wrapper I am calling reactor.run() - (that blocks execute method) and in one of callbacks for success/error calling reactor.stop(). So the problem is that the first call to execute returns from reactor.run() right after reactor.stop(), but second call to same method of same instance do not return ever. I believe I am doing something wrong, so please help with find out what I've missed. Problem code attached.

4 5

Re: [Twisted-Python] HTTPClientFactory's deferred never finishing download on .vcf (vcard file) link
by exarkun＠twistedmatrix.com 10 Oct '09

10 Oct '09

On 05:34 am, listsin(a)integrateddevcorp.com wrote: > >On Oct 9, 2009, at 5:53 AM, Reza Lotun wrote: >>>I haven't found any way to dump the actual headers from within >>>Twisted >> >>The getPage function in twisted.web is suited for simple calls but >>won't return returns. What you have to do is create your own slightly >>modified version of getPage that keeps track of the factory used for >>the call (by attaching it to the deferred or something). > >Yah, I kind of think this code was intended to do his own getPage() >but I'm not exactly sure what the point was. Maybe he used it for >debugging along the way, but the original intent was lost due to lack >of VCS (as was he!) > >I ended up rewriting it to use the simple client.getPage() > >Along the way, I ran into: > >http://python.net/crew/mwh/apidocs/twisted.web.http.Request.html > >Good Lord, no wonder I'm confused. > >Half PEP-8, http://www.python.org/dev/peps/pep-0008/, half JavaRama. > >I finally figured out that it was a stupid string formatting bug in >the code that was being echoed by Twisted but swallowed by the "Grand >Catcher of all Things Exception" in Twisted.. > >Once I found and fixed that, everything works fine -- until the next >time. As Reza Lotun pointed out, exceptions raised by application code and caught by Twisted are logged. If you find a place where application code can raise an exception and Twisted doesn't log it, this is a bug and should be reported. Jean-Paul

1 0

[Twisted-Python] Disturbing aspect of zope.interface
by Brian Granger 09 Oct '09

09 Oct '09

Hi, I know this is not the zope list, but I am encountering this in a twisted project, so I wanted to get some ideas here first. I am finding that zope.interface is like an infectious disease. Here is a simple example: In [35]: import zope.interface as zi In [37]: class A(object): ....: pass In [38]: class IB(zi.Interface): ....: pass In [39]: hasattr(A, '__provides__') Out[39]: False In [41]: # Inheriting from A gives A additional methods In [42]: class B(A): ....: zi.implements(IB) In [43]: hasattr(A, '__provides__') Out[43]: True In [45]: # C is clean of the additional methods In [46]: class C(object): ....: pass In [47]: hasattr(C, '__provides__') Out[47]: False The disturbing thing is that now A (which is just an object) has been hacked on by zope.interface. It has additional methods (__provides__, etc.) that are specific to zope. Summary: If a class A is later subclassed by something B that calls zi.implements, the original class A becomes infected with all the zope.interface stuff. Is there a way to avoid this? Doesn't this seem like a bad idea? Cheers, Brian

9 11

[Twisted-Python] IMAP IDLE client support
by Adam MacBeth 08 Oct '09

08 Oct '09

It doesn't appear that Twisted's IMAP client (twisted.mail.imap.IMAP4Client) supports the IDLE command. Are there any plans for this or does anyone have any thoughts on the complexity of implementing it? Thanks, Adam

1 0

[Twisted-Python] PB vs JMS
by jacopo.pecci＠gmail.com 08 Oct '09

08 Oct '09

I looked at PB for an architecture with one client distributing some processing to several servers. Now I came across JMS and I have seen that using ActiveMQ with the Stomp protocol there would be a good support for Python. Surprising I couldn't find any article comparing the two technologies. I wonder if they target different problems (it doesn't look to me). Is anyone able to sketch advantages of one solution against the other? Thanks, Jacopo

5 4

Re: [Twisted-Python] Problem with XMLRPC resource wrapped with guard basic auth
by asset 07 Oct '09

07 Oct '09

I was indeed being tricked. I actually tested soap with a real client, but with xmlrpc i was using the browser. I was expecting to see the same error message, and instead got a new one that was presented by the modifications I pulled out of the trunk to support the issue in #4014. I was expecting to see the older error message and it was replaced. A quick test via, >>> import xmlrpclib >>> server = xmlrpclib.Server('http://user:password@localhost:9999/rpc2') >>> server.quote() Showed it indeed worked like it should. Swapped it to use SSL and now I'm all set. Thank you for the clarification. Next time I'll test more before posting to the list. ;) TWKiel ---------------------------------------- From: exarkun(a)twistedmatrix.com Sent: Tuesday, October 06, 2009 6:43 PM To: asset(a)impactdamage.com, "Twisted general discussion" <twisted-python(a)twistedmatrix.com> Subject: Re: [Twisted-Python] Problem with XMLRPC resource wrapped with guard basic auth On 12:00 am, asset(a)impactdamage.com wrote: >Using the current trunk r27366 (which is after #4014 fixed a related >issue), I am having trouble with an implementation of web.guard wrapped >XMLRPC. This is a new test implementation to expose both a soap and >xmlrpc >interface. SOAP works, but xmlrpc throws UnsupportedMethod POST. I think you're being tricked by the confusing way in which this exception is displayed and by a slight implementation difference (and indeed externally visible behavioral difference) between XMLRPC and SOAPPublisher. SOAPPublisher defines "render" and no other render methods. So it will accept any request method and treat it the same way. It will never raise UnsupportedMethod. XMLRPC, on the other hand, defines "render_POST" and no other render methods, so it will only accept POST requests. For any other request, it will raise UnsupportedMethod and create that exception with a tuple of which methods it does allow. ('POST',) in this case. The exception I see when I approach this server at /rpc2 with my web browser (which is the only thing I've tried, because it's too much work to put together a real xml-rpc client that supports basic auth) is just what I'd expect. The browser issues a GET, the XMLRPC resource rejects this, indicating it only accepts POSTs. It may be worth improving the way UnsupportedMethod exceptions are stringified to make it more clear what's going on. Or, if this doesn't actually explain your problem, feel free to point that out and provide more details about how the client you're using behaves. Jean-Paul

1 0

[Twisted-Python] using perspective broker to talk to a subprocess?
by Ryan Burns 07 Oct '09

07 Oct '09

Hi All, I'm writing a twisted application where I'd like to run and communicate with subprocesses that also happen to be twisted apps. I'm wondering if I can somehow combine twisted's process and perspective broker code so that I can launch a twisted server in a subprocess and communicate with it via the perspective broker using the subprocess's stdin/stdout. Ideally, it would be great if this could scale, say up to 200 subprocesses. Disregarding reasons of why I would want to do this, is this something that is possible? I've looked at the perspective broker classes and the processProtocol as well as some of the helpers, but I don't quite see how to make it happen. Should I just use a socket instead and handle managing unique names/ports per subprocess on my own? Or any alternative ideas? Ultimately I just want to run subprocesses from my twisted app and communicate with them without having to do a bunch of parsing on my own and the subprocesses will be twisted as well, so I figure why not make use of twisted for my IPC. Thanks, Ryan

2 2

[Twisted-Python] Problem with XMLRPC resource wrapped with guard basic auth
by asset 07 Oct '09

07 Oct '09

Using the current trunk r27366 (which is after #4014 fixed a related issue), I am having trouble with an implementation of web.guard wrapped XMLRPC. This is a new test implementation to expose both a soap and xmlrpc interface. SOAP works, but xmlrpc throws UnsupportedMethod POST. Here is my test code, can anybody tell me if im doing something wrong? Again, /soap works, but /rpc2 freaks on the POST method being unavailable. ###### test-script.py from zope.interface import implements from twisted.internet import reactor from twisted.web.resource import IResource, Resource from twisted.web import server, guard from twisted.cred.portal import IRealm from twisted.python import log from zope.interface import implements from twisted.python import log from twisted.internet import reactor from twisted.web import server, resource, guard, xmlrpc, soap from twisted.cred.portal import IRealm, Portal from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse import sys def getQuote(): return "Victory to the burgeois, you capitalist swine!" class XMLRPCQuoter(xmlrpc.XMLRPC): def xmlrpc_quote(self): return getQuote() class SOAPQuoter(soap.SOAPPublisher): def soap_quote(self): return getQuote() class WebServicesRealm(object): implements(IRealm) def requestAvatar(self, avatarId, mind, *interfaces): if resource.IResource in interfaces: node = resource.Resource() node.putChild("rpc2", XMLRPCQuoter()) node.putChild("soap", SOAPQuoter()) return resource.IResource, node, lambda: None raise NotImplementedError() if __name__ == "__main__": log.startLogging(sys.stdout) checker = [InMemoryUsernamePasswordDatabaseDontUse(foo='bar')] webServicesWrapper = guard.HTTPAuthSessionWrapper(Portal(WebServicesRealm(), checker), [guard.BasicCredentialFactory("test")]) reactor.listenTCP(9999, server.Site(webServicesWrapper)) reactor.run() ###### http://localhost:9999/soap *good* http://localhost:9999/rpc2 *bad; POST isnt an allowed method* Thanks, TWKiel

6 9

[Twisted-Python] Example of a long-polling asynchronous web servic?
by Stephen Mattison 07 Oct '09

07 Oct '09

Hey guys I'm trying to implement some web-services over JSON-RPC, and one of the methods needs to use server-push. I will try and explain my problem as well as I can. I have a client, which is an iPhone, natively running a JSON-RPC( or XML-RPC ) client implementation. I need to do something like this from the iPhone service = ServerProxy("http://mydomain.com/webservice") init_data = service.subscribe_to_updates() ... ... ... some_result = service.some_regular_function(params) How can I implement this in Twisted? -- Stephen Mattison

2 2

[Twisted-Python] Structural issues in Lore XHTML documents
by Kevin Horn 06 Oct '09

06 Oct '09

I've been looking over the xhtml documents used to generate the twisted documentation, and I've noticed a number of issues: - some docs do not have a DOCTYPE declaration, I think they should all have one - of those documents that do have DOCTYPEs, some are using xhmtl-strict, and some are using xhtml-transitional, which is preferred? I think they should all use the same one - some of the docs are lacking an xml namespace attribute in their root <html> element...I think they should either all have one, or none of them should - according to: http://twistedmatrix.com/projects/lore/documentation/howto/lore.html all of the docs should have the same text in both their <title> element and their <h1> element...this is not the case I'm planning to correct some of these issues, but I wanted to get others' opinions on exactly how to go about it. For example, what DOCTYPE to use, XMLNS to use, etc. Thoughts? Suggestions? I've opened a ticket (#4050) at http://twistedmatrix.com/trac/ticket/4050 Please comment. Thanks, Kevin Horn

2 3

[Twisted-Python] Is it necessary to utilize twisted.cred in twisted web?
by biziap biziap 06 Oct '09

06 Oct '09

I have googled this topic and found and example in (A) http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg01796.html well, another simpler example is (B) http://www.mail-archive.com/twisted-web@twistedmatrix.com/msg01788.html My questions are: 1. Does the approach in (A) be recommended? To generate resource dynamically seems not efficient and not necessary for simple scenario. Is there other way to bind twisted.cred and twisted.web together? (except the deprecated twisted.web.guard) 2. The approach in (B) which suggests that request.getSession() along is quite enough to implement an simple authentication feature. Here the "simple scenario" means to guard some resource with username and password. To do it: In a protected resource, just to check for a flag in the session, if failure, then redirect to login page. If succeeded, render the resource. Why shall we bother the portal, credentials, checker,.... ? Any suggestion? Thanks in advance.

3 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 04 Oct '09

04 Oct '09

1 0

[Twisted-Python] XML parsing on twisted
by gary clark 04 Oct '09

04 Oct '09

Hello, I went to use the SAX library to perform parsing. However I hit an unusual problem. Its having a problem parsing a token that has a '&' it just crashes. It certainly woke me up when I witnessed this. <books> Jim & Nancy </books> are passed to the ContentHandler class. When characters(self,chars) member gets called I perform self.book+=chars to reassemble the string. I dump out the self.book string as its being assembled. If I replace the '&' character with "and" then its Ok. Of course I dont want to have to replace '&' with 'and'. Not good. So has anybody else seen this problem. I'm assuming it thinks its a reference and becomes confused when I attempt to add a reference. Much appreciated, Garyc

2 2

[Twisted-Python] Boston Twisted Sprint
by Jessica McKellar 02 Oct '09

02 Oct '09

Let's try this again: The proposal was for a sprint in Boston from September 28-October 2. That starts a week from today. At this point: 1. There will be no funding for transportation/lodging for out of town folks 2. The location would be classrooms at MIT unless someone has an alternative 3. There will be no funding for food or snacks or drinks or anything like that Facts about using MIT classrooms as a Sprint location: 1. Room reservations would be for after school hours, aka after 5pm, for weekdays. 2. Guests register their MAC address and have 14 days of wireless. It's plausible that some of you visit MIT and have used up your free wireless. I don't have a solution for that besides "find a different computer to register and use". 3. The doors on campus are mostly locked at night and I won't be there to let you in all the time. Some special doors remain unlocked. Bottom line: you can get it and get to the rooms but it may be quite circuitous. I'm cool with that if you're cool with that. Are you cool with that? Please send me an e-mail off-list with: 1. if you'd attend, and if so which days 2. regardless of 1, if you believe that the sprint should be delayed until funding/better location/more notice/whatever has a chance to happen -jesstess

6 10

[Twisted-Python] twisted python usage
by vitaly＠synapticvision.com 02 Oct '09

02 Oct '09

hi, I'm trying to accomplish to work couple of CLI commands without luck. http://twistedmatrix.com/projects/core/documentation/howto/options.html and test/test_usage.py were not really helpful. Commands: 1.python test.py cmd1 -s 'dfsgdfg' -e 'sdfgdfg' 2.python test.py cmd2 -t 123 3.python test.py cmd3 cmd4 -t '2345ge' -y 'rebtr' cmd5 -p 'wgerg' Appreciate any help.

3 5

[Twisted-Python] Use of 'twistd' utility with command-line arguments
by Valeriy Pogrebitskiy 02 Oct '09

02 Oct '09

I came across some problem using 'twistd' utility - that I would not have expected from it. The problem is - using 'twistd' to start application that requires command-line arguments. In other words, I would expect that 'twistd' passes command-line arguments to Python application - just as Python does. But... I get following error: Usage: twistd [options] Options: --savestats save the Stats object rather than the text output of the profiler. -o, --no_save do not save state on shutdown -e, --encrypted The specified tap/aos/xml file is encrypted. ......................... ......................... /usr/bin/twistd: option -a not recognized when application is started either as 'twistd -y <fileName.py>' or 'twistd --file <fileName.py>' followed by a list of command-line arguments My question is: how is it possible to use 'twistd' to start applications that do require command-line arguments? Does anyone have suggestions? In other words, I want to be able to start application using 'twistd' - just as it would have been started using standard Python: > python <fileName.py> [argList] > twistd -y <fileName.py> [argList] Second question to Twisted development team: how such an important case has slipped attention? I suggest this to be added to bug fix requests, with high priority. Kind regards, Valeriy Pogrebitskiy vpogrebi(a)verizon.net

5 6

[Twisted-Python] getPeer() inside conch
by Benjamin Rutt 02 Oct '09

02 Oct '09

Hello. I'm moderately experienced with twisted, and am starting out writing an ssh server using conch. I'm writing a protocol which inherits from the recvline.HistoricRecvLine class. It has connectionMade() and lineReceived() methods so it feels a lot like the twisted.protocols.basic.LineReceiver class I'm used to using. But, it seems the recvline.HistoricRecvLine class wants me to use self.terminal rather than self.transport to talk to the client. That's fine, but it seems that self.terminal lacks a getPeer() method. How do I find out the IP address of the client inside a conch ssh server? (For starters, I wish to log all successful client connections so I know who's connecting). I searched and found the below issue. I assume it is related, but could someone confirm? http://twistedmatrix.com/trac/ticket/2453 Thanks. -- Benjamin Rutt

3 6

[Twisted-Python] Knowing end of packet
by gary clark 02 Oct '09

02 Oct '09

Hello, I am sending a large amount of data to a twisted server. I want to know how the protocol class knows that no more data is to be received. I dont pass the length of the data in the packet, hence I'm looking for a termination field in the packet.Like </MESSAGE>. I did see a logging class but not sure if thats what I want. If I have to overload the dataReceived method, and buffer the data until I see the termination field thats ok, just need to know if this seems correct in twisted. Much appreciated for any help. Thanks, Garyc

4 8

[Twisted-Python] twisted-wave?
by Stephen Waterbury 02 Oct '09

02 Oct '09

Just curious what the twisted minions think of google wave and whether anyone is considering or working on a twisted implementation of the federation protocol, <http://www.waveprotocol.org/draft-protocol-spec>. Steve

4 3

Re: [Twisted-Python] Twisted and db table changes
by Steve 02 Oct '09

02 Oct '09

3 3

Re: [Twisted-Python] Twisted and db table changes
by Steve 01 Oct '09

01 Oct '09

If you are using PostgreSQL you can use this technique http://www.divillo.com -------- Original Message -------- Subject: Re: [Twisted-Python] Twisted and db table changes From: Mark Visser <markv(a)lumierevfx.com> Date: Tue, September 29, 2009 12:04 pm To: Twisted general discussion <twisted-python(a)twistedmatrix.com> gary clark wrote: > Hello, > > I have a server that needs to track changes on a table database. > > When a table changes it needs the server accesses the table and send data out. > > Not sure what the best approach to do this in twisted. Do I create a thread in twisted and monitor for any changes in a table or can I create > a callback method to be triggered when a change occurs to table. > Check out twisted.internet.task.LoopingCall. It will run a callback periodically, during which you can poll for changes. -- Mark Visser, Software Director Lumière VFX Email: markv(a)lumierevfx.com Phone: +1-514-316-1080 x3030 _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1