Mailman 3 February 2010 - Twisted

[Twisted-Python] Nevow question
by Sury Soni 21 Feb '10

21 Feb '10

Hi, How do we disable traceback on Nevow production site? Right now, by default, any nevow site will print full length of traceback on web browser if there is some exception within the application. I have gone through the ticket http://twistedmatrix.com/trac/ticket/135 but could not find quick place to disable/enable the same in my nevow application. Sury Prakash Soni

2 2

[Twisted-Python] fast high load protocol
by Vlad Shevchenko 21 Feb '10

21 Feb '10

Hi, I have developed two kind of servers: api - http server memory_db - dict-based server with some logic of putting data to dict and getting it back. Api communicate with memory_db via txJSON-RPC but response time is huge under high load. I also try implementation with replacement txJSON-RPC -> Perspective Broker with no success. PB implementation has 10+ times faster responses when returning data is small (dict with few keys and values like list of 1-5 strings). However, PM much depends on size of returning value. When I return a list of 600 items (each is string with length 20) response time incredibly growing up. Unlike PB, json-rpc implementation keeps it response-time near the same lever as with small return data. Can you suggest some protocol with a small response? Or should I look into development of my own? -- WBR, Vlad Shevchenko

5 8

[Twisted-Python] Need help for structured application development
by Einar S. Idsø 21 Feb '10

21 Feb '10

Hi, I am planning to develop an application that will query a number of services at given intervals, and Twisted seems like a very nice platform to do so. However, because of little previous experience with Twisted, especially with building proper applications, using interfaces and using plugins, I could really need some pointers to get me started. In the following I will try to explain in broad terms what the application will do, and humbly ask for input on how I may structure it using factories, protocols etc.: 1. A number of more-or-less similar server applications (5-10) have a number of concurrent instances/servers (1-30) each. 2. I need to acquire the status of each instance at given intervals. These will typically range from every 20 seconds to every 3 minutes, and depend on the which service the instance is based on. 3. A list of the server instances shall be acquired via an XML-query every few minutes to pick up on new or removed instances. 4. An executable already exists which can be used to status-poll almost all services using different parameters. I will be calling it from my application to do the polling instead of recreating it in Python. 5. I need to be able to create my own polling-protocol for at least one of the service types which is incompatible with the executable. 6. Some services can not only be polled at intervals, but additionally stream information that I am interested in listening to. 7. At a later point in time, I wish to be able to communicate with IRC, MSN and other services/protocols using the queried and streamed data. My initial general thought is the following: 1. Design a class for each server application, where all classes implement the same interface. The class defines how to deal with server instances of its corresponding server application type. Classes are created as Twisted-plugins. 2. For each new server instance that is discovered, create an object which contains a timer and polls the server at specific intervals, writing the result to a globally accessible data structure. 3. For server instances that stream data continuously in addition to being polled at regular intervals, create a separate object which listens to the server. I believe I should somehow have a global data structure which contains references to all the server instance objects, regardless of type, and allows them to store data. This is sort of like the factory does for protocols, except in my case I don't think I can use a regular factory since I need both clients (to poll at intervals) and servers (to listen to streaming data). I suppose I could use a ClientFactory for the pollers and a Factory for the stream-listeners, and let them have a common data-store? But is it possible to use a single ClientFactory to spawn all poller-objects regardless of server type? Or do I need one ClientFactory for each server type? Do I perhaps need one ClientFactory per server instance? Or should I just ignore factories all together since it is primarily the timer and asyncronicity of Twisted I need? I am quite certain there is a correct twistified way of handling this sort of problem. I am just not sure what it is, and would sincerely welcome all feedback on how this should be done. Feel free to point to examples of applications that are similar to what I sketched above, and tell me to study the code :) Cheers, Einar

4 7

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 21 Feb '10

21 Feb '10

1 0

[Twisted-Python] Streaming File Transfer Protocol?
by Darren Govoni 19 Feb '10

19 Feb '10

Hi, Is there an existing protocol that can provide the following? - Accept stream binary data FROM a client (e.g. very large file transfer) - Receive data IN ORDER (i.e. stream. not out of order random packets) I want to stream FROM a client to the protocol server and have the server process the stream bytes incrementally so it doesn't have to store or write the entire data stream (too large). I looked at FileTransferServer and Client, but I'm not sure it provides what I need. Any tips appreciated! Darren

10 27

[Twisted-Python] Running php script on twisted server
by Tux Attack 18 Feb '10

18 Feb '10

I am trying to run a php script on a webserver written in twisted.For that i am using twcgi module. Here is my code. This is code for webServer.py -> from twisted.internet import reactor from twisted.web import static,script,twcgi,server from twisted.python.log import startLogging import sys class PHPScript(twcgi.FilteredScript): filter='/media/DATA/Documents/project-docs/twisted/twisted-programs/testPHP/twisted-php.py' root=static.File('html_files') root.indexNames=['index.php'] root.processors={'.php':twcgi.PHPScript} site=server.Site(root) reactor.listenTCP(8000,site) startLogging(sys.stdout) reactor.run() And here is another file used in program twisted-php.py -> #!/usr/bin/env python > import os, sys, commands > print "Content-type: text/html\n\n"; > command = 'php ' + sys.argv.pop(1) > data = commands.getstatusoutput(command) > print data[1] > Now my problem is if i try to acess some url like http://localhost:8000/list.php then everything works fine but if i enter url with parameters like http://localhost:8000/list.php?file=sample.txt then twisted is not passing parameters after '?' to twisted-php.py I verified this in twisted-php.py Is there any way i can use such php scripts with parameters in twisted?

1 0

[Twisted-Python] Authenticating with md5 hashed passwords
by Ramiro Alba Queipo 17 Feb '10

17 Feb '10

Hello everybody, I am trying to build a a client/server application using Perspective Broker and wanting to authenticate against a PostgreSQL database. Everything works fine if I user plain text passwords, but when trying to hash then using md5 using from hashlib import md5 md5Password = md5(password).hexdigest() then it does not authenticate (I use credentials.checkMD5Password(password) at the checker class) Then after reading twisted/spread/pb.py I saw that everything is done in the functions: respond(challenge, password) challenge() and the methods checkMD5Password(self, md5Password) checkPassword(self, password) at the class _PortalAuthChallenger(Referenceable, _JellyableAvatarMixin) By changing digest() with hexdigest(), it works. The question is: I there some way to make it work without making changes at the 'pb.py' module? Yes. I should use md5Password = md5(password).digest() to produce the password, but then I cant authenticate with a 'pure-ftpd' daemon I need to work with. Any alternatives? Thanks in advance. Regards -- Ramiro Alba Centre Tecnològic de Tranferència de Calor http://www.cttc.upc.edu Escola Tècnica Superior d'Enginyeries Industrial i Aeronàutica de Terrassa Colom 11, E-08222, Terrassa, Barcelona, Spain Tel: (+34) 93 739 86 46 -- Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que està net.

10 22

[Twisted-Python] Help Twisted at PyCon (or anywhere else)
by Itamar Turner-Trauring 17 Feb '10

17 Feb '10

Hello Twisted users, Does Twisted help you with your job, hobby or homework? Here's how you can help Twisted in return, especially if you're coming to PyCon: * Talk to us! We want to hear what features you want, what problems you are having. But especially, we want to hear about the cool stuff you build with Twisted, because that makes us happy (and even happier to work on Twisted and help you.) * Develop Twisted! Whether it's writing documentation or fixing bugs, we always appreciate help from new developers. Some of us will be hacking at PyCon (aka "talking a lot and occasionally coding"), so drop by and join us. After PyCon some developers will be staying around for a sprint, which you are welcome to join -- some topics people have expressed interest in: web client wsgi container websocket inotify generic sasl win32 stdio/console sphinx docs conversion PyWeek / "Twisted Game" web2 deferred cancellation endpoints release deferToProcess spread conch bug fixing toolchain improvements for casual contributors * Get your company to donate! Or at least, introduce us to your boss. Much of our development (coding and bug reviews) is done by volunteers, but a large chunk is also paid for by tax-deductible donations via our friends at the Software Freedom Conservancy. Donating helps us fix bugs and add new features more quickly. * Hire (some of) us! Jean-Paul Calderone and possibly others would be happy to consult with your organization. * Buy us a beer on your per diem! Or two. We also accept other beverages. Many of us will be at PyCon -- and no doubt we'll have an open space session -- so please do track us down. Thanks for using Twisted! --Itamar Turner-Trauring, on behalf of the Twisted development team

1 0

Re: [Twisted-Python] web page on doc...
by Kevin Horn 17 Feb '10

17 Feb '10

On Tue, Feb 16, 2010 at 2:18 PM, K. Richard Pixley <rich(a)noir.com> wrote: > Kevin Horn wrote: > > xhtml is processed into html by "lore" not "trial". > > Doh. Thanks. I *knew* that. I haven't gotten either to work yet, but > neither is really on my radar right now either. > > Well, I haven't gotten Lore to work (on Windows at least) in a looooooong time. The only way I can get trial to work on Windows is by using the "Twisted Command Prompt", or some hairy PATH manipulation. > <http://twistedsphinx.funsize.net>Other than that, something very like > what you have described should definitely > be _somewhere_ on the Twisted site. > > (As an aside, I think a lot of things that are currently on the wiki should > be merged > into the documentation proper, but that's a topic for another post.) > > I think I'd need to hear more. > > I think there's a subtle distinction between describing the code, usage, > and interfaces of the twisted library, and describing the people, policies, > and interfaces of the twisted project. Generally I think I'd prefer to keep > them as clearly segregated as possible but there's certainly some grey area > in between. > > --rich > For example, if you want to contribute to Twisted, you need to read (at the very least): http://twistedmatrix.com/documents/current/core/development/index.html but you also need to read: http://twistedmatrix.com/trac/wiki/TwistedDevelopment http://twistedmatrix.com/trac/wiki/ReviewProcess and maybe: http://twistedmatrix.com/trac/wiki/ContributingToTwistedLabs to really get a full picture. A lot of this is incomplete on a given page, and you need to have read the others to really understand what's going on. Some of that stuff should be merged into the actual development docs (like the stuff on commit messages, for example). Some of it certainly belongs in the wiki, but not all of it, and some of that could be condensed into one page. Kevin Horn

2 1

[Twisted-Python] web page on doc...
by K. Richard Pixley 16 Feb '10

16 Feb '10

I've been going around and around on documentation with/for twisted for a few days now. For example, I read things like "the documentation is written in epytext" and "documentation is processed by trial" and conclude that trial processes epytext. I think I'm beginning to understand what's really going on now but two things would have helped me. 1) a brief overview of the documentation. (proposal below). 2) if references to "the documentation" could be clarified to refer to "howto documentation" vs "API documentation". If agreed, then I will volunteer to help find and patch #2. As a proposal for #1, I offer the following as an insertion into http://twistedmatrix.com/trac/wiki/ReviewingDocumentation as a second section: UNDERSTANDING DOCUMENTATION ORGANIZATION Documentation for Twisted comes in four primary parts. The first part is a collection of "Howto" documents which are composed in xhtml, processed by twisted's "trial", into html. The source for the Howto documents lives in subversion under doc in the various "howto" directories. The second part is a collection of examples. These examples are included inline in the "howto" doc. These are written in python and twisted and are included in the various "examples" directories under "doc" in the source. The third part is the API documentation which is produced by pydoctor from the docstrings scattered throughout the twisted source code. The doc strings are written in epytext format. The fourth part are traditional unix man pages. These are written in troff -ms format and the sources are stored in the various "man" directories in the source under "doc". --rich

2 1