Mailman 3 May 2003 - Twisted

RE: [Twisted-Python] [Tutor] Help on personnal Proxy
by POYEN OP Olivier (DCL) 27 May '03

27 May '03

> The short answer to your question is to replace: > > f.HTTPChannel=proxy.Proxy > > with: > > f.protocol=proxy.Proxy > Can I stress that f.HTTPChannel comes from the docstring from the Proxy Class in the proxy.py file from Twisted. If this is the "official" way to do, maybe should someone change the docstring.... Anyway. When doing so, I have some trouble with the "as-is" proxy. Some exceptions occured: My current parameters are : Twisted 1.0.5 for python 2.2 export http_proxy='http://localhost:8765' # for the proxy And the current web server is running on localhost:8080 The issue is that, when doing a 'wget http://localhost:8080', in web.proxy.py, there is, at line 113 host,port= string.split Thus, port became a string. And in internet.tcp.py, there is a test for isInstance(port,StringTypes), which matches..... With a single port=int(port) before calling reactor.connectTcP in web.prxoy, ezverything works. Am I dead wrong ? > Or, I've written a (yet another) web proxy that you might > want to try too. Using my MinProxy .... > I'll gladly have a look ! Thanks ! OPQ ----------------------------------------------------------------- Ce message est confidentiel ; son contenu ne represente en aucun cas un engagement de la part de Dexia Credit Local ou de Dexia CLF Banque. Toute publication, utilisation ou diffusion, meme partielle, doit etre autorisee prealablement par l'emetteur. Si vous n'etes pas destinataire de ce message, merci d'en avertir immediatement l'expediteur. This message is confidential ; its contents do not constitute a commitment by Dexia Credit Local or Dexia CLF Banque. Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. If you are not the intended recipient of the message, please notify the sender immediately. ----------------------------------------------------------------- Consultez notre site internet www.dexia-clf.fr La cle d'une gestion optimisee du secteur public local -----------------------------------------------------------------

1 0

[Twisted-Python] [Tutor] Help on personnal Proxy
by POYEN OP Olivier (DCL) 27 May '03

27 May '03

Hi all, I tried to read some parts of the how-to, tried to google around in order to create a simple personnal web proxy with no luck. For now, I don't want to do anything special, but having a proxy on my localhost, watching for my connections. Later, I may want to enhance its functionnality (a la "proxomitron") I thought that twisted may certainly be the perfect tool to do that, and I thought this would be a good way to start using it. I know about the proxy class, but, when I try to use it: (taken from Proxy class docstring): from twisted.protocols import http from twisted.web import proxy from twisted.internet import reactor f=http.HTTPFactory() f.HTTPChannel=proxy.Proxy reactor.listenTCP(8765,f) reactor.run() I then try an 'export http_proxy=http://localhost:8765' and wget http://localhost:8080 (where a nice twisted.web server is waiting) Not working. Even a telnet on port 8765 shown that nothing is responding. I then thougt about proxy.Proxy which perhaps need to be instanciated: f.HTTPChannel=proxy.Proxy() Same punition. Then, I thougth that I certainly need to subclass my proxy to have it do some stuff: class myProx(proxy.Proxy): def connectionMade(self,*args): print "cool...." def dataReceived(self,*args): print args I then again try with myProx *and*ù myProx(). Nothing worked. Well, I search for anything on the mailing list archive, but with no luck. Do I have to say that any help or RTFM would be appreciated ! Thanx in advance, OPQ ----------------------------------------------------------------- Ce message est confidentiel ; son contenu ne represente en aucun cas un engagement de la part de Dexia Credit Local ou de Dexia CLF Banque. Toute publication, utilisation ou diffusion, meme partielle, doit etre autorisee prealablement par l'emetteur. Si vous n'etes pas destinataire de ce message, merci d'en avertir immediatement l'expediteur. This message is confidential ; its contents do not constitute a commitment by Dexia Credit Local or Dexia CLF Banque. Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. If you are not the intended recipient of the message, please notify the sender immediately. ----------------------------------------------------------------- Consultez notre site internet www.dexia-clf.fr La cle d'une gestion optimisee du secteur public local -----------------------------------------------------------------

2 1

Re: [Twisted-Python] getProcessOutput ok to use in a DeferredList?
by Stuart Hungerford 26 May '03

26 May '03

> [...] >I'm going to guess what your problem might be, based on a problem I once >had. :) > >If you have multiple Deferreds, some "depend" on others, DeferredList's >default behaviour isn't what you want. By "depend" I mean if Deferred A >fails (and so fires its errback, not callback), then Deferred B will never >be called, i.e. B depends on A. (This is not official terminology! :). > >In that case, you probably want to use the fireOnOneErrback flag of >DeferredList. This changes the behaviour of DeferredList so that it will >fire its callbacks as soon as either: > - All its Deferreds have fired their callbacks, or > - any of its Deferreds have fired an errback. Aha! A brilliant piece of remote empathetic debugging ;-) The lack of an errback was masking an error from the Deferreds. Many thanks. Cheers, Stu -- :: Stuart Hungerford (stuart.hungerford(a)anu.edu.au) :: ANU Internet Futures Group

1 0

[Twisted-Python] Re: SOAP and XML-RPC stuff
by Andrew Dalke 26 May '03

26 May '03

Andrew Bennetts: > Obviously you can't have multiple things listening on port 7080. Perhaps > this should raise an exception, though, so as to not seem like it can. I figured the first was the case, but I didn't know if there was some way to support content negotiation appropriately, so that a SOAP request on a given URL goes to one handler and an XML-RPC request on the same URL goes to another handler. (Someday I would like to play around more with conneg, so I think this is cute, but the realistic side of me says it's a bad idea.) > I suspect you can probably do: > from twisted.web.resource import Resource > > root = Resource() # An empty resource ... Bingo! It works now, and I'm one step further on my way to Twisted Enlightenment. Thanks Andrew! BTW, I looked up twisted.web.resource on the doc page at http://www.twistedmatrix.com/documents/TwistedDocs/current/api/ I don't see 'twisted.web.resource' in the 'Everything' section in the lower left, which means it should be labeled 'Not Quite Everything.' ;) Andrew dalke(a)dalkescientific.com

2 1

[Twisted-Python] HowTo -- load a business-object in a Twisted Web Server ??
by Thomas Weholt 26 May '03

26 May '03

I need to load a business-object in a Twisted Web server serving static content and rpys, and it should be available in all parts of the application. It has to be able to run a method or methods at specified intervals, fetching some files from the net, scanning a filesystem or extracting data from a database, which in return will be available in the business-object in the web-server. What is the Twisted-way of doing this? The tasks the business-object will perform from time to time can be time consuming and probably blocking too. Should a start it in a seperate thread? What ? I need clues or a small example. best regards and thanks in advance, Thomas Weholt

2 1

[Twisted-Python] Re: [Twisted-commits] Boatloads of things, from Anders Hammarquist.
by Itamar Shtull-Trauring 26 May '03

26 May '03

Move the LimitedLog to twisted.python.logfile, maybe? Seems like a better place. And many thanks to JP for finally checking this in.

3 2

[Twisted-Python] getProcessOutput ok to use in a DeferredList?
by Stuart Hungerford 26 May '03

26 May '03

Hi all, I have an application that uses a bunch of Deferred's to handle blocking system calls and converting the results of those calls to HTML. I'd like to make use of the very neat DeferredList and also the utility function getProcessOutput(). My understanding is that getProcessOutput() returns a Deferred that returns a string of the process output. Are there any issues with passing a list of getProcessOutput() results to a DeferredList? I understand the DeferredList still needs its own callback registered for completion of the list of deferreds. My initial tests with this setup just seem to suspend and not return, which is almost certainly something I've misunderstood in using Deferreds. Has anyone done this before? Cheers, Stu -- :: Stuart Hungerford (stuart.hungerford(a)anu.edu.au) :: ANU Internet Futures Group

2 1

[Twisted-Python] SOAP and XML-RPC stuff
by Andrew Dalke 26 May '03

26 May '03

http://www.twistedmatrix.com/documents/howto/xmlrpc Has a link under "In addition, XML-RPC published methods can return twisted.python.defer.Deferred" The last word is hyperlinked to http://twistedmatrix.com/documents/TwistedDocs/current/api/public/ twisted.python.defer.Deferred.html which does not exist. == http://twistedmatrix.com/documents/TwistedDocs/current/api/public/ twisted.web.xmlrpc.XMLRPC.html says Faults, Binray, Boolean, DateTime, Deferreds, or Handler instances. typo: "Binray" --> "Binary" == Here's a patch to work with SOAPpy 0.10 --- /usr/local/lib/python2.3/site-packages/twisted/web/soap.py Sun May 25 21:46:12 2003 +++ /Users/dalke/ftps/Twisted-1.0.5/twisted/web/soap.py Wed Oct 23 19:55:20 2002 @@ -13,14 +13,11 @@ """ # SOAPpy -try: - import SOAPpy as SOAP -except ImportError: - import SOAP +import SOAP - # fix 2.2 issues with SOAP - SOAP.SOAPBuilder.dump_str = SOAP.SOAPBuilder.dump_string - SOAP.SOAPBuilder.dump_dict = SOAP.SOAPBuilder.dump_dictionary +# fix 2.2 issues with SOAP +SOAP.SOAPBuilder.dump_str = SOAP.SOAPBuilder.dump_string +SOAP.SOAPBuilder.dump_dict = SOAP.SOAPBuilder.dump_dictionary # twisted imports from twisted.web import server, resource Or, in short version, change # SOAPpy import SOAP to # SOAPpy try: import SOAPpy as SOAP except ImportError: import SOAP # fix 2.2 issues with SOAP SOAP.SOAPBuilder.dump_str = SOAP.SOAPBuilder.dump_string SOAP.SOAPBuilder.dump_dict = SOAP.SOAPBuilder.dump_dictionary (I checked - 0.10 handles the typename changes from 2.2) === How do I run both an XML-RPC and a SOAP server on the same server? I tried something like def main(): from twisted.internet.app import Application app = Application("xmlrpc") r = XMLRPC_Calc() app.listenTCP(7080, server.Site(r)) r = SOAP_Calc() app.listenTCP(7080, server.Site(r)) return app application = main() if __name__ == "__main__": application.run(save=0) But that doesn't work - the SOAP requests go to the XML-RPC server. Is there some way to say that "/RPC2" is for XML-RPC and that "/SOAP" is for SOAP? Andrew dalke(a)dalkescientific.com

2 1

[Twisted-Python] Anomaly found between twisted XML-RPC client and twisted server (Bug?)
by Wari Wahab 25 May '03

25 May '03

Hi there, I was testing around the XML-RPC support in twisted and found the following: same twisted client and server (one instance) = error.connectionLost twisted client -> twisted server = error.connectionLost xmlprclib client -> twisted server = got my results twisted client -> xmlrpclib client = got my results. First time round, the twisted xmlrpc client sits together with the server in one application, the client calls on a reactor.callLater(). This does not work, so I figured I must be doing something wrong. I split up the client and the server and I got the same error, split up the program network wise, again the same error, on the client, its: [Failure instance: Traceback: twisted.internet.error.ConnectionLost,] and on the Server: 2003/05/22 23:30 SGT [HTTPChannel,5,192.168.10.15] Traceback (most recent call last): File "/usr/lib/python2.2/site-packages/twisted/protocols/basic.py", line 173, in dataReceived self.lineReceived(line) File "/usr/lib/python2.2/site-packages/twisted/protocols/http.py", line 893, in lineReceived self.allContentReceived() File "/usr/lib/python2.2/site-packages/twisted/protocols/http.py", line 932, in allContentReceived req.requestReceived(command, path, version) File "/usr/lib/python2.2/site-packages/twisted/protocols/http.py", line 504, in requestReceived self.process() --- <exception caught here> --- File "/usr/lib/python2.2/site-packages/twisted/web/server.py", line 156, in process self.render(resrc) File "/usr/lib/python2.2/site-packages/twisted/web/server.py", line 162, in render body = resrc.render(self) File "/usr/lib/python2.2/site-packages/twisted/web/xmlrpc.py", line 103, in render args, functionPath = xmlrpclib.loads(request.content.read()) File "/var/tmp/python-2.2.2-root/usr/lib/python2.2/xmlrpclib.py", line 805, in loads p.close() File "/var/tmp/python-2.2.2-root/usr/lib/python2.2/xmlrpclib.py", line 393, in close self._parser.Parse("", 1) # end of data xml.parsers.expat.ExpatError: no element found: line 1, column 0 Here's what I did in detail: The server is the exact one found at http://twistedmatrix.com/documents/howto/xmlrpc The client is something like the one I found at the same page as well, but this time on an interactive python session. ---------------- Python 2.2.2 (#1, May 19 2003, 14:32:06) [GCC 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from twisted.web.xmlrpc import Proxy >>> from twisted.internet import reactor >>> >>> def printValue(value): ... print repr(value) ... reactor.stop() ... >>> def printError(error): ... print 'error', error ... reactor.stop() ... >>> p = Proxy('http://192.168.11.1:7080') >>> #### Calling the add method with twisted >>> p.callRemote('add', 3, 5).addCallbacks(printValue, printError) <Deferred at 0x83d3f84> >>> reactor.run() error [Failure instance: Traceback: twisted.internet.error.ConnectionLost, ] >>> from xmlrpclib import Server >>> s=Server('http://192.168.11.1:7080') >>> #### Calling the add method with xmlrpclib >>> s.add(3,5) 8 >>> p = Proxy('http://some.web.site/RPC2') >>> #### Using proxy again to talk to an xmlrpclib based server (pyblosxom) >>> p.callRemote('blogger.getUsersBlogs', '', 'myusername', 'mypassword').addCallbacks(printValue, printError) <Deferred at 0x83e8ac4> >>> reactor.run() [{'url': 'http://some.web.site/blog/', 'blogid': '/', 'blogName': '/'}, {'url': 'http://some.web.site/blog/GeekyStuff/', 'blogid': '/GeekyStuff/', 'blogName': '/GeekyStuff/'}] >>> ----------------- What could be happening here?

4 9

[Twisted-Python] Fixes for distutils (disregard previous mail)
by Daniel Waites 25 May '03

25 May '03

Disregard my previous mail. Sorry, but I got the diff backwards :( The following patches will get "python setup.py bdist_rpm" from Python distutils working. Also, I added some fixes for including the documentation in the rpm distribution. Unfortunately, RPM 4 creates an extra debuginfo RPM when running rpmbuild. This breaks bdist_rpm.py, since the distutils coder assumed only one binary RPM would be created. This causes an assertion error, but the RPMs still get build before the error happens. The RPMs are located in build/build-blahblah/rpm/RPMS/<arch>/ =================================================================== RCS file: /cvs/Twisted/MANIFEST.in,v retrieving revision 1.9 diff -r1.9 MANIFEST.in 21a22 > include ChangeLog 33c34,36 < --- > recursive-include twisted *.c > recursive-include twisted *.mgp *.html *.fla *.js *.swf > recursive-include twisted/test *.pem *.message Index: setup.cfg =================================================================== RCS file: /cvs/Twisted/setup.cfg,v retrieving revision 1.3 diff -r1.3 setup.cfg 28c28,32 < --- > doc_files=ChangeLog > CREDITS > LICENSE > README > doc/ ---------------- Daniel Waites danielwaites(a)earthlink.net

2 1

Twisted May 2003