Mailman 3 December 2007 - Twisted

[Twisted-Python] How send information (constructor data?) to a derived protocol
by Doug Farrell Dec. 12, 2007

Dec. 12, 2007

Hi all, I'm trying to develop a system to handle Python logging messages via the logging.handlers.sockethandler system. What I want is a to be able to have multiple Python processes logging to the same log file, as well as a global log file. I want to do this by having a local logging server that handles all of the log messages from all of the Python processes per server. I also want to have a global logging server that handles all logging messages from all processes from all servers. I've written a class that derives from Protocol and does the right thing with the log messages generated by the logging.handler.sockethandler, and I'd like to use this class for both the local and global servers. On the global server machine there will be both services running; the local and global server. So here's where I finally get to my question. I'd like to use my derived Protocol class, but I want each instance of it (in the local server and in the global server) to use a separate logging configuration file. How do I pass information to my Protocol class so that it configures itself for the correct logging configuration file? I don't see how to pass information via the constructor, and I don't see how else to get information in there. I'm fairly new to Twisted, so maybe I'm missing something obvious. Any help would be greatly appreciated! Thanks, Doug

2 1

[Twisted-Python] number of Hack Fests reaches an all-time high
by zooko Dec. 11, 2007

Dec. 11, 2007

Folks: There will be a Hack Fest this coming Friday, December 14th, at 17:00 hours Pacific Time, at the offices of allmydata.com in Potrero Hill, San Francisco. This will be the third Hack Fest, making its cardinal number one larger than the cardinal number of any previous Hack Fest! [1, 2] We already have two people lined up to show off their projects, making the number of presenters with prior planning one larger than any previous number of prepared presentations. Those two are Pavel with a Python decompiler ("WATCH as he translates .pyc files into .py files before your very eyes!") and Nejucomo with "Conway's Second Life", which is mattress of cells floating above the ground which spawn new cells and reconfigure themselves in order to play Conway's Life. Since one of these demos will be taking place in Second Life (projected onto the wall of our office in First Life), we thought it would be fun to invite people who can't physically come to San Francisco to virtually join the Hack Fest in Second Life. Please e- mail me zooko(a)zooko.com and nejucomo(a)gmail.com to coordinate. By the way, there will probably be more projects at play at the Hack Fest -- these two are just the only two that are already planned. You are welcome to bring yours! As usual, allmydata.com will generously provide drinks, eats, and funky beats. Please RSVP to zooko(a)zooko.com and secorp(a)allmydata.com. Regards, Zooko [1] http://lists.zooko.com/pipermail/p2p-hackers/2007-November/ 001412.html [2] http://lists.zooko.com/pipermail/p2p-hackers/2007-November/ 001444.html

1 0

[Twisted-Python] stopping MultiService child services
by Don Smith Dec. 11, 2007

Dec. 11, 2007

Hi all, I'm still a bit of a novice but I'm trying to implement MultiService. The trick is that I want my program to be able to change it's config while it is running. Once the app realizes it has a new configuration it needs to stop all services, load the new config and start services accordingly. However, I'm stuck right now with how to stop all services. I'm thinking I need to loop over the children of MultiService and call stopService on them, but I'm not sure how to do that. Or should I just call stopServivce on the MultiService object? Thanks, Don

2 2

[Twisted-Python] Emulating urllib.read()-behaviour with twisted.web.client
by Franz Buchinger Dec. 9, 2007

Dec. 9, 2007

Hello, I'm currently switching from the urllib library to twisted.web.client in my HTTP Download Class. How can I emulate urllib.read(bytes=None) behaviour with twisted.web.client? (only reading n bytes of the ressource, if the bytes parameter is given). The second difficulty (at least from my twisted newbie perspective) is to return the downloaded data to the method caller outside the class. Any hints are appreciated.. Thanks, Franz

1 0

[Twisted-Python] Please help: having problems with the simplest Twisted Server/Client Demo Example from the Tutorial
by Sajit Rao Dec. 9, 2007

Dec. 9, 2007

Hello all, I've been trying the most basic (only a few lines) server/client code example in the Twisted docs and can't get even that to work (code included below). It's very frustrating, because I did get a simple server/client example working without using Twisted. Any help will be greatly appreciated. The problem is that the client never connects with the server. it prints "Started to Connect" and hangs, then times out with an error. I'm trying this on my laptop running WinXP - I first start the server program (from the command line python prompt) and then the client program in another window on the same machine. Please let me know what I'm doing wrong. Server Code: from twisted.internet.protocol import Factory, Protocol from twisted.internet import reactor class QOTD(Protocol): def connectionMade(self): self.transport.write("An apple a day keeps the doctor away\r\n") self.transport.loseConnection() factory = Factory() factory.protocol = QOTD reactor.listenTCP(8007, factory) reactor.run() Client Code: from twisted.internet.protocol import Protocol, ClientFactory from sys import stdout class Echo(Protocol): def dataReceived(self, data): stdout.write(data) class EchoClientFactory(ClientFactory): def startedConnecting(self, connector): print 'Started to connect.' def buildProtocol(self, addr): print 'Connected.' return Echo() def clientConnectionLost(self, connector, reason): print 'Lost connection. Reason:', reason def clientConnectionFailed(self, connector, reason): print 'Connection failed. Reason:', reason from twisted.internet import reactor reactor.connectTCP('localhost', 8007, EchoClientFactory()) reactor.run() Result Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import QOTDServer ..then I start the client in a separate window... Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import EchoClient Started to connect. Connection failed. Reason: [Failure instance: Traceback (failure with no frames) : <class 'twisted.internet.error.TimeoutError'>: User timeout caused connection failure. ][/b]

3 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Dec. 9, 2007

Dec. 9, 2007

1 0

[Twisted-Python] Timeout for Deferred objects
by Mustafa Sakalsiz Dec. 8, 2007

Dec. 8, 2007

Hi, I want set a timeout for Deferred objects. There is a method in Deferred class, setTimeout. It works well for me, but it gives a deprecation warning. If setTimeout is deprecated, what else I should use for timeouts. Thanks Saki

2 1

[Twisted-Python] Altering Distutils to get PyOpenSSL to Work ? Re: Twisted-Python Digest, Vol 44, Issue 32
by Andrew Francis Dec. 5, 2007

Dec. 5, 2007

Hello Zooko: > The error message is in error. Here is the Python > issue ticket in which the message was changed from >stating that you have to use the > same compiler that was used to build Python, to > instead stating that you have to use a compiler that >is compatible with the one that was used to build >Python: > > In my experience, simply setting "compiler=mingw32" > in your distutils config file works for all extension >modules that I have tried *except* for pyOpenSSL. Thanks for the information... I have followed pyOpenSSL's Windows instructions. I downloaded the Microsoft Visual C++ Express kit (what a monster). I still get the same message. I read somewhere that distutils looks at the registry to look for the Microsoft C compiler. I know little about distutils and I have not used Visual Studio in ages. And I really need to get https stuff working soon. If this is the case, what exactly would I have to change and what value do I use? Thanks in advance, Andrew ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs

1 0

[Twisted-Python] TLS echo server and client examples
by Thomas Raschbacher Dec. 5, 2007

Dec. 5, 2007

hi all. With therves help I changed the ssl echoserv and echoclient examples to use tls today. Thought I post this to the list for (possible) inclusion in the examples directory, as it might be useful for other ppl too. Regards -- Thomas Raschbacher http://www.lordvan.com

1 0

[Twisted-Python] @inlineCallbacks eat the traceback. Help?
by Matthias Urlichs Dec. 5, 2007

Dec. 5, 2007

It seems that @inlineCallbacks trashes the error traceback when you yield a Deferred that emits an error. That's not very social behavior. Any ideas how to get the traceback back? This is Twisted 2.5. Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/lib/python2.5/site-packages/twisted/internet/threads.py", line 25, in _putResultInDeferred result = f(*args, **kwargs) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 813, in unwindGenerator return _inlineCallbacks(None, f(*args, **kwargs), Deferred()) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 724, in _inlineCallbacks result = g.throw(result.type, result.value, result.tb) --- <exception caught here> --- File "/var/lib/python-support/python2.5/homevent/worker.py", line 153, in _process r = yield w.process(event=self.event, queue=self) exceptions.TypeError: float() argument must be a string or a number -- Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf(a)smurf.noris.de Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de - - Know when to quit.

2 1

[Twisted-Python] wx and twisted
by Michele Petrazzo - Unipex srl Dec. 3, 2007

Dec. 3, 2007

Hi all, just installed the last wx and twisted and tring my already working app and: Traceback (most recent call last): File "threading.py", line 442, in __bootstrap self.run() File "threading.py", line 422, in run self.__target(*self.__args, **self.__kwargs) File "/usr/lib/python2.4/site-packages/twisted/internet/_threadedselect.py", line 149, in _workerInThread fn(*args) --- <exception caught here> --- File "/usr/lib/python2.4/site-packages/twisted/internet/_threadedselect.py", line 169, in _doSelectInThread [], timeout) exceptions.ValueError: file descriptor cannot be a negative integer (-1) and on shutdown: Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/wx-2.8-gtk2-unicode/wx/_core.py", line 14364, in <lambda> lambda event: event.callable(*event.args, **event.kw) ) File "/usr/lib/python2.4/site-packages/twisted/internet/_threadedselect.py", line 243, in _interleave msg, args = self.toMainThread.get_nowait() File "/usr/lib/python2.4/Queue.py", line 141, in get_nowait return self.get(False) File "/usr/lib/python2.4/Queue.py", line 116, in get raise Empty Queue.Empty How debug it? Debian etch, python 2.4.4, wx 2.8.7.1, twisted 2.5.0 Thanks, Michele

4 5

[Twisted-Python] Advise for heavy concurrency
by Atilla Dec. 3, 2007

Dec. 3, 2007

I'm interested in how well Twisted would perform in situations where there is a very high number of concurrent connections. The problem: I'm working on a feature that would require me to distribute small amounts, both in occurrence and size, of packets to a very large number of connected clients. Think in the terms of a Jabber or IRC server, but with a lot less traffic. What we're trying to do is to maximize the number of simultaneous connections a server can run, to an absolute maximum. It'll be a dedicated system so crippling any system-wide setting is not going to interfere with other applications. So far experimenting with the epoll reactor has shown really nice scalability as numbers grow. Thousands of connections per machine are no problem at all, but we need to scale out heavily. At a certain point the memory requirements for keeping open all those connections in the system grow higher, but those can be minimized by adjusting system buffer sizes, tcp window sizes and certain other settings. Furthermore - more than one instance of the service could be ran per server, dedicating two or three processes on a 4-core CPU system, until exhausting the rest of the limits, in case CPU oevrhead becomes large. I'm also considering advantages that might come through different I/O and/or process schedulers. Is there anything that would be beneficial to my situation, considering the server will be a dedicated machine with little to nothing else to run on it that might suffer performance degradation? Does anyone have any experience with scaling twisted to a very large number of parallel connections? Are there any limitations that I might be missing? We'll start experimenting with a sample service soon to try to identify any specific limitations that will need to be worked on. Any tips are wellcome. Cheers,

7 25

[Twisted-Python] StdioOnnaStick instance has no attribute 'isatty'
by Marcin Kasperski Dec. 3, 2007

Dec. 3, 2007

I just tried to run some trial tests on ubuntu gutsy (twisted 2.5.0), just to get the error as in the title (full traceback below). Does anybody know what is wrong here? Traceback (most recent call last): File "/usr/bin/trial", line 24, in <module> run() File "/usr/lib/python2.5/site-packages/twisted/scripts/trial.py", line 345, in run test_result = trialRunner.run(suite) File "/usr/lib/python2.5/site-packages/twisted/trial/runner.py", line 772, in run result = self._makeResult() File "/usr/lib/python2.5/site-packages/twisted/trial/runner.py", line 712, in _makeResult return self.reporterFactory(self.stream, self.tbformat, self.rterrors) File "/usr/lib/python2.5/site-packages/twisted/trial/reporter.py", line 555, in __init__ if colorizer.supported(): File "/usr/lib/python2.5/site-packages/twisted/trial/reporter.py", line 446, in supported if not sys.stderr.isatty(): AttributeError: StdioOnnaStick instance has no attribute 'isatty' -- ---------------------------------------------------------------------- | Marcin Kasperski | For waterfall, milestones are commitments. | http://mekk.waw.pl | For iterative development, they are decision | | points. (Martin) ----------------------------------------------------------------------

2 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Dec. 2, 2007

Dec. 2, 2007

1 0

[Twisted-Python] implementing a timeout
by J Davis Dec. 1, 2007

Dec. 1, 2007

Greetings, I'm a newcomer to twisted and am struggling a little with the learning curve. I have an idea for an app that I believe is ideally suited for twisted though I could use some help finding the right way to approach it. It performs a status check by getting a page from a web server. It calculates the time it took to download the page and also can check that the content matches a predefined string. That part of it is working fine but I'd like to take it a step further. I need a way to impose a limit on how long it waits to receive a response so that I can detect unresponsive hosts. I must say that I'm at a loss as to how to add such a timeout to my code. Any pointers or examples would be greatly appreciated. Here is a simplified version of the function that I'm working with. --------------------------------------- from twisted.internet import task, reactor from twisted.web import client import time class HTCheck: def handleResponse(self, data, start_time): response_time = time.time() - start_time if data == "It lives!\n": Status = "OK" else: Status = "Test page content mismatch" def check(self): start_time = time.time() d = client.getPage("http://localhost/test.html") d.addCallback(self.handleResponse, start_time) return d l = task.LoopingCall(HTCheck().check) l.start(10.0) reactor.run()

2 2

[Twisted-Python] Automating the release
by Jonathan Lange Dec. 1, 2007

Dec. 1, 2007

Hello all, As you may know, radix and others are working on automating the Twisted release process so that we can move to a more frequent, predictable release cycle. The tickets to do so are here: http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reope… Our goal is to do a release using the automated tools before the end of this year. If you want to help out, there are three things you can do: 1. Help implement one of the release tickets. The release tickets need coding. Some of the tickets have fairly poor descriptions -- but that's ok. All that means is that your first step is hassling radix to clarify what he means. If you are going to hack on these, it's very important that you do test-driven development. We want this code to be first-class. 2. Fix tickets more generally. I *was* going to recommend that you work on some of our tickets with 'high' and 'highest' priority, but it turns out that they represent one quarter of all open tickets. Still, if that doesn't daunt you then please start hacking. I promise that the regular developers will be extra nice and friendly in helping get started with the Twisted code base. 3. Review branches All code that goes into Twisted must be reviewed before it lands. It doesn't really take much arcane knowledge to do a code review. All you have to do is make sure the code conforms to our conventions, check the test coverage and then ask questions about any bit you don't understand. The idea is that if you don't understand it, the code should be clearer. Branches up for review are listed here: http://twistedmatrix.com/trac/report/15 There's general information about Twisted development here: http://twistedmatrix.com/trac/wiki/TwistedDevelopment Happy hacking, jml

1 0