Mailman 3 May 2007 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] Reconnecting PB Client
by Don Smith 12 Sep '07

12 Sep '07

Does anyone have some example code for building a reconnecting PB client? This is what I have now: from twisted.spread import pb from twisted.internet import reactor from twisted.python import util events = pb.PBClientFactory() reactor.connectTCP("localhost", 8789, events) queue = events.getRootObject()

3 2

[Twisted-Python] Win32 IOCPReactor?
by Trent Nelson 04 Sep '07

04 Sep '07

I'm new to Twisted. Perusing the documentation, I came across IOCPReactor. It's marked as 'extremely experimental', and it appears it only provides support for TCP events. It seems the last check in to anything IOCPReactor-related was back in 2005. Is it dead? Is anyone using it? Is anyone planning to pick development back up? Were there specific issues with IOCP that impeded development, or just a general lack of people with time, inclination, or Win32-fu? Assuming the IOCPReactor's not feasible for high load production deployment, can anyone comment on how the default select()-based reactor for Win32 fares under high load? Trent.

19 31

[Twisted-Python] Media Resource Control Protocol (MRCP)
by Ivan Uemlianin 04 Sep '07

04 Sep '07

Dear All I am interested in writing an MRCP (Media Resource Control Protocol [1]) server. In particular I want to write an MRCP speech recognition resource server as part of my trefnydd [2] project. I am posting here to ask for advice about the network programming part. I shall also be writing an MRCP client (if only to test the server). All the code will be GPL'd. As far as I understand it, an overview of the networking would go like this: An MRCP session between client and server is initiated using the Session Initiation Protocol (SIP). Client and server negotiate using SIP messages containing SDP message bodies to define the participants and the resources required in the session. After session initiation, two channels are launched: a media channel using RTP to carry binary audio (i.e., speech) data, and a control channel using TCP (control channel messages are text-based and in MRCP format). It seems to me that both client and server software would have to cope with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the SIP/SDP and MRCP control channel messages). Here are my questions: - Does anyone know of a FOSS project to write an MRCP server? preferably in python. I know of mrcp4j [3]. - My first plan would be to make the MRCP client/server a subclass of an SIP client/server, as SIP is used to set up (and tear down) the session. Does that sound sensible? - Can twisted.protocols.sip be used to write a SIP server? Would it give me everything I need, or would I also need something like pjsip [4] or jain-sip [5] for the low-level stuff? - Have I missed anything? Any advice would be gratefully received. I'll report back with progress. Best wishes Ivan [1] MRCP: http://www.llaisdy.com/static/tech/mrcp/ [2] trefnydd: http://www.llaisdy.com/static/trefnydd/ [3] mrcp4j: "MRCP4J provides a Java API that encapsulates the MRCPv2 protocol and can be used to implement MRCPv2 clients and/or servers." http://mrcp4j.sourceforge.net/ [4] pjsip: http://www.pjsip.org/ [5] jain-sip: https://jain-sip.dev.java.net/

6 10

[Twisted-Python] New to twisted- need help to hit stride
by Keith Gunderson 31 May '07

31 May '07

Hello group, I've been lurking and watching for a while. I need to change hats now and get some coding done. I'm an engineer/ sysadmin/ dev and a public university who's been willingly forced to be a sysadmin after years of C/C++ development. I'll be on IRC in the evenings. I'm hoping to converse with some of you well-known folks (and others) so I can pick up speed w/ twisted/ python. I definitely want to do things "by the books" (w/ respect to twisted methodology.) I seem to be a mere step or two away from clicking, but I'm thinking in circles at the transition between design and implementation. I need to decide how much to code from scratch, how much to borrow from twisted-related libraries, and how much is available from twisted itself. Please offer suggestion for which classes/ libraries to use (and avoid) and even project skeletons to start with. Use Cases: - Emacs fragment manager: sent & receive blocks of text to a twisted server from emacs. Sort by user, time, name. -SSH command - line manager : send command strings to twisted server. Another fuzzy goal would be to have the commands run on other hosts by the twisted server. -LDAP - user database authentication. After implemeting those features, I expect to re-evaluate my skills and design so as to begin phase 2. Feel free to make suggestions or wait for more pointed questions. KGunderson --------------------------------- No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started.

2 1

[Twisted-Python] RE: Modifying a web proxy to use SSL
by Adams, Larry 30 May '07

30 May '07

Here is a little more detail on my problem. I've been able to verify that the connection is made to the https server and that data is received. In fact, I've overridden the ProxyRequest.handleResponseEnd() method to save this data to a database. I can see the raw html and image data in the database, but nothing is returned to the browser. Does anyone have any experience implementing their own proxy with SSL that might be able to point me in the right direction? Any assistance would be greatly appreciated. -larry Here is my custom ProxyRequest class (i had to modify some stuff because secure urls have the https stripped and :443 added to the uri for some reason): class SerpicoProxyRequest(proxy.ProxyRequest): #protocols = {'http': SerpicoProxyClientFactory} #LEA - added https protocols = {'http': SerpicoProxyClientFactory, 'https' : SerpicoProxyClientFactory} ports = {'http' : 80, 'https' : 443} def __init__(self, *args): proxy.ProxyRequest.__init__(self, *args) def process(self): if (self.uri.find(':443')>0): self.uri = 'https://' + self.uri.replace(':443','') self.method = 'GET' parsed = urlparse.urlparse(self.uri) protocol = parsed[0] host = parsed[1] port = self.ports[protocol] self.setHost(host,port,1) if ':' in host: host, port = host.split(':') port = int(port) rest = urlparse.urlunparse(('','')+parsed[2:]) if not rest: rest = rest+'/' class_ = self.protocols[protocol] headers = self.getAllHeaders().copy() if not headers.has_key('host'): headers['host'] = host self.content.seek(0,0) s = self.content.read() clientFactory = class_(self.method, rest, self.clientproto, headers, s, self) clientContextFactory = ssl.ClientContextFactory() c = reactor.connectSSL(host, port, clientFactory, clientContextFactory) else: proxy.ProxyRequest.process(self) Here is my custom ProxyClient (the Page() object is what I'm using to save the data to the database): class SerpicoProxyClient(proxy.ProxyClient): def connectionMade(self): global CURRENTJOBID global CURRENTCHILDID log.msg("initializing connection") self.page = Page() if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.instance = machine_name self.page.job_id = int(CURRENTJOBID or 0) self.page.task_id = int(CURRENTCHILDID or 0) # self.father refers to the http.Request object self.page.request_uri = self.father.uri self.page.save() log.msg("MACHINE NAME IS %s" % machine_name) log.msg("PROXYREQUEST %s" % self.father.uri) log.msg("TRANSPORT %s" % self.transport) log.msg("ISSECURE %s" % self.father.isSecure()) proxy.ProxyClient.connectionMade(self) def handleHeader(self, key, value): proxy.ProxyClient.handleHeader(self, key, value) # log.msg("%s : %s added" % (key, value)) if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.header_set.create(key=key, value=value) def handleResponsePart(self, data): if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.data = data proxy.ProxyClient.handleResponsePart(self, data) def handleResponseEnd(self): if not self.father.uri == "%s/command/get_agent_task/%s" % (command_site,machine_name): self.page.save() log.msg('ending response with my data::%s\n\nfatherdata::%s\n\nmytransport::%s' % (self.page.data, self.father.c hannel.transport, self.transport)) self.transport.loseConnection() self.father.channel.transport.loseConnection() ________________________________ From: Adams, Larry Sent: Tuesday, May 29, 2007 3:42 PM To: twisted-python(a)twistedmatrix.com Subject: Modifying a web proxy to use SSL I'm trying to use twisted to proxy all web requests for a set of PCs (virtual machines actually). I've created my own ProxyRequest class to override the process() method so that it will use SSL if the URL is secure. It makes the connection and retrieves the data successfully, but the browser still doesn't display the HTML. I must be missing some other class/method that I have to override, but I'm at a loss as to which one at the moment. Below is some logging from my application for the same page called via http and https respectively. Any help or pointers to working examples would be greatly appreciated! Thanks, Larry Adams 2007/05/29 15:10 -0500 [SerpicoProxy,11,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:10 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 80) 2007/05/29 15:10 -0500 [Uninitialized] initializing connection 2007/05/29 15:10 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:10 -0500 [Uninitialized] PROXYREQUEST http://www.connectcommerce.com/ <http://www.connectcommerce.com/> 2007/05/29 15:10 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [Uninitialized] ISSECURE False 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:11:27 GMT added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #11 on 74007> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:11 -0500 [SerpicoProxy,18,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c> 2007/05/29 15:11 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 443) 2007/05/29 15:11 -0500 [Uninitialized] initializing connection 2007/05/29 15:11 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:11 -0500 [Uninitialized] PROXYREQUEST https://www.connectcommerce.com <https://www.connectcommerce.com> 2007/05/29 15:11 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [Uninitialized] ISSECURE True 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:12:00 GMT added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #18 on 74007> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c>

1 0

[Twisted-Python] twisted.names.client.Resolver user neither 'search' nor 'domain' from resolv.conf?
by Bjorn Andersson 30 May '07

30 May '07

Hi, I just briefly tested twisted.names.client.Resolver with a resolv.conf configuration file. I noticed that it doesn't utilize the configured 'domain' or 'search' directives. Is there any plan to implement this in the near future or is there a deliberate reason why these aren't used? Bjorn A.

2 1

[Twisted-Python] better way to do this?
by Eli Criffield 29 May '07

29 May '07

What i want to do is validate the "From" host of an xmlrpc server against a list of allowed servers. Sounds easy enough but "connectionMade(self)" Call is way back in http.HTTPChannel. That is set as the protocol in http.HTTPFactory and HTTPFactory is inherits by server.Site. So whats the best way to change one little function in http.HTTPChannel ? Here is what i have now, but the I have to include all of server.Site in my example, even though its unchanged except it inherits myHTTPFactory instead of http.HTTPFactory. Is there a simpler way to do this? Eli Criffield --example--- #!/usr/bin/env python from twisted.web import xmlrpc, server, http, resource from twisted.internet import reactor from twisted.python.log import startLogging from sys import stdout import copy class Example(xmlrpc.XMLRPC): """An example object to be published.""" def xmlrpc_echo(self, x): """Return all passed args.""" print "echo called" return x def xmlrpc_add(self, a, b): """Return sum of arguments.""" print "add called" return a + b class myHTTPChannel(http.HTTPChannel): def connectionMade(self): print "Check here if connection is allowed from", self.transport.getPeer() self.setTimeout(self.timeOut) class myHTTPFactory(http.HTTPFactory): protocol = myHTTPChannel class mySite(myHTTPFactory): counter = 0 requestFactory = server.Request displayTracebacks = True def __init__(self, resource, logPath=None, timeout=60*60*12): """Initialize. """ http.HTTPFactory.__init__(self, logPath=logPath, timeout=timeout) self.sessions = {} self.resource = resource def _openLogFile(self, path): from twisted.python import logfile return logfile.LogFile(os.path.basename(path), os.path.dirname(path)) def __getstate__(self): d = self.__dict__.copy() d['sessions'] = {} return d def _mkuid(self): """(internal) Generate an opaque, unique ID for a user's session. """ import md5, random self.counter = self.counter + 1 return md5.new("%s_%s" % (str(random.random()) , str(self.counter))).hexdigest() def makeSession(self): """Generate a new Session instance, and store it for future reference. """ uid = self._mkuid() session = self.sessions[uid] = Session(self, uid) session.checkExpiredLoop.start(1800) return session def getSession(self, uid): """Get a previously generated session, by its unique ID. This raises a KeyError if the session is not found. """ return self.sessions[uid] def buildProtocol(self, addr): """Generate a channel attached to this site. """ channel = http.HTTPFactory.buildProtocol(self, addr) channel.requestFactory = self.requestFactory channel.site = self return channel isLeaf = 0 def render(self, request): """Redirect because a Site is always a directory. """ request.redirect(request.prePathURL() + '/') request.finish() def getChildWithDefault(self, pathEl, request): """Emulate a resource's getChild method. """ request.site = self return self.resource.getChildWithDefault(pathEl, request) def getResourceFor(self, request): """Get a resource for a request. This iterates through the resource heirarchy, calling getChildWithDefault on each resource it finds for a path element, stopping when it hits an element where isLeaf is true. """ request.site = self # Sitepath is used to determine cookie names between distributed # servers and disconnected sites. request.sitepath = copy.copy(request.prepath) return resource.getChildForRequest(self.resource, request) if __name__ == '__main__': r = Example() reactor.listenTCP(7080,mySite(r),ctx) startLogging(stdout) reactor.run()

2 2

[Twisted-Python] Modifying a web proxy to use SSL
by Adams, Larry 29 May '07

29 May '07

I'm trying to use twisted to proxy all web requests for a set of PCs (virtual machines actually). I've created my own ProxyRequest class to override the process() method so that it will use SSL if the URL is secure. It makes the connection and retrieves the data successfully, but the browser still doesn't display the HTML. I must be missing some other class/method that I have to override, but I'm at a loss as to which one at the moment. Below is some logging from my application for the same page called via http and https respectively. Any help or pointers to working examples would be greatly appreciated! Thanks, Larry Adams 2007/05/29 15:10 -0500 [SerpicoProxy,11,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:10 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 80) 2007/05/29 15:10 -0500 [Uninitialized] initializing connection 2007/05/29 15:10 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:10 -0500 [Uninitialized] PROXYREQUEST http://www.connectcommerce.com/ <http://www.connectcommerce.com/> 2007/05/29 15:10 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [Uninitialized] ISSECURE False 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:11:27 GMT added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.Client'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 80) at b71d8ccc> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #11 on 74007> 2007/05/29 15:10 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8cec> 2007/05/29 15:11 -0500 [SerpicoProxy,18,172.18.36.22] Starting factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c> 2007/05/29 15:11 -0500 [Uninitialized] building protocol for addr: IPv4Address(TCP, 'www.connectcommerce.com <file://www.connectcommerce.com> ', 443) 2007/05/29 15:11 -0500 [Uninitialized] initializing connection 2007/05/29 15:11 -0500 [Uninitialized] MACHINE NAME IS surfsidekick 2007/05/29 15:11 -0500 [Uninitialized] PROXYREQUEST https://www.connectcommerce.com <https://www.connectcommerce.com> 2007/05/29 15:11 -0500 [Uninitialized] TRANSPORT <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [Uninitialized] ISSECURE True 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Date : Tue, 29 May 2007 20:12:00 GMT added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Server : Apache/1.3.33 (Unix) (Gentoo/Linux) added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Location : http://www.connectcommerce.com/global/login.html <http://www.connectcommerce.com/global/login.html> added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Connection : close added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Content-Type : text/html; charset=iso-8859-1 added 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] trying to lose connection for transport: <<class 'twisted.internet.tcp.TLSConnection'> to ('www.connectcommerce.com <file://www.connectcommerce.com> ', 443) at b71bd86c> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] parent transport : <SerpicoProxy #18 on 74007> 2007/05/29 15:11 -0500 [SerpicoProxyClient,client] Stopping factory <__main__.SerpicoProxyClientFactory instance at 0xb71d8d0c>

1 0