Mailman 3 February 2004 - Twisted

[Twisted-Python] Second try at reporting gtk reactor problem
by David E. Konerding Feb. 27, 2004

Feb. 27, 2004

Hello, since nobody actually responding with a useful message regarding my report, I'm reposting an extremely simple (1 page) example which exhibits the behavior. My example creates a gtkreactor and runs it. A small reporter class is run from a second thread. Under the gtk reactor, the log messages from the second thread are not reported in a timely fashion (only once per second or so). From my understanding the messages should be printed more frequently (about once every 1/10th second). This does work under the wx and plain reactors, so I suspect there is a defect in the gtk reactor. I would simply like to get a confirmation that my example is a valid one, that the results I expect are valid, and that other people can reproduce my experience. If that is the case then I will submit a formal bug report. All of this should be reproducible using Twisted 1.2.0. Things get even weirder if you remove the logging support and replace it with all print statements. import time, sys from twisted.internet import gtk2reactor gtk2reactor.install() from twisted.python import threadable threadable.init() from twisted.internet import reactor from twisted.python import log log.startLogging(sys.stdout) class TestThingy: def __init__(self): print "TestThingy created" def run(self): print "TestThingy run" for i in range(10): print "TestThingy message" time.sleep(0.1) print "TestThingy done" class ThingyManager: def __init__(self): pass def submitThingy(self): log.err("ThingyManager submitting") f = TestThingy() log.err("ThingyManager created") reactor.callInThread(f.run) log.err("ThingyManager calledInThread") def ticker(*args): log.err("ticker: %d" % time.time()) reactor.callLater(1, ticker) t = ThingyManager() reactor.callLater(0, ticker) reactor.callLater(3, t.submitThingy) reactor.run() log.err("Reactor done")

2 2

[Twisted-Python] DeferredList.addDeferred behaviour
by RITA Y/O RODRIGO DIAZ Y/O BENENSON Feb. 27, 2004

Feb. 27, 2004

I have been playing with DeferredList if found that his behaviour was not what I expected, >>> dfl = DeferredList([]) >>> print dfl.called True >>> dfl.addDeferred(d) >>> d.called 0 >>> print dfl.called True The workaround it to first create the list of deferred instance and then instanciate with it the DeferredList. But what sense does have the addDeferred method then ? I this behaviour what is supposed to be ? rodrigob.

4 5

[Twisted-Python] PYCON ATTENDERS!
by Christopher Armstrong Feb. 26, 2004

Feb. 26, 2004

If you plan on coming to a Twisted-related sprint, *PLEASE* add your name to the wiki page at http://www.python.org/cgi-bin/moinmoin/TwistedSprint If you can't do that, at least reply to this message stating your intent to attend. The PyCon organizers need to know how many people are coming. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://radix.twistedmatrix.com/

2 1

[Twisted-Python] Twisted web proxy
by Paul Moore Feb. 26, 2004

Feb. 26, 2004

I'm trying to implement a simple web proxy using Twisted. With 3 or 4 web-based apps on my machine (moin, roundup, PyDS, ...) I'd like to put them all behind a single server on port 80, for ease of access. I could install Apache and use ProxyPass[Reverse], something like this: ProxyPass /moin/ http://localhost:8081/ ProxyPassReverse /moin/ http://localhost:8081/ ProxyPass /roundup/ http://localhost:8082/ ProxyPassReverse /roundup/ http://localhost:8082/ But that seems fairly heavyweight. I tried looking at proxy.py in the Twisted examples, and it sort-of does what I'm after, but seems to give inconsistent results (some proxied pages just don't appear), and the sample code doesn't log anything I can scan for problems (and I can't get it to - wherever I put the print statements seems to be wrong :-() I suspect that I'm mucking up attaching children (the example just proxies everything from the root, I want to hang the remote server "lower" in the tree), but I can't see how. Has anyone got any code that does this sort of thing, that they'd be willing to share? Thanks, Paul.

1 0

[Twisted-Python] ANN: Twisted 1.2.0
by Itamar Shtull-Trauring Feb. 26, 2004

Feb. 26, 2004

Twisted is an event-driven networking framework for server and client applications. For more information, visit http://www.twistedmatrix.com, join the list http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python or visit us on #twisted at irc.freenode.net. The Twisted from Scratch tutorial is a good starting point for learning Twisted: http://twistedmatrix.com/documents/howto/tutorial What's New in 1.2.0 =================== - SFTP server implementation for the SSH server. - Improved wxPython support. - IMAPv4 enhancements and bug fixes. - Allow disabling display of tracebacks in error web pages. - ident protocol implementation (client and server). - Support mapping arbitrary child FDs when running processes on POSIX. - Initial SOAP client support (using SOAPpy). - Partial download support for FTP client. - Web framework now supports different handlers for different methods (e.g. GET or POST). - Coverage support in the trial testing framework. - Bug fixes and documentation and feature enhancements.

1 0

[Twisted-Python] oddness in gtk reactor compred to wx and plain reactor
by David E. Konerding Feb. 26, 2004

Feb. 26, 2004

Hi, I have been experience a weird problem with the gtk reactor. I've got a simple test case which seems to reproduce the problem, so I'm checking with the mailing list to understand if everything is going on properly. Here's the test code. I wasn't sure whether the log. calls are thread-safe, but I've noticed the same problem with Python's logging package. Using the wxPython or plain reactor seems to work as expected: messages from the TestThingy are spewed at about 0.1/sec, while with gtk, messages only come every 2 seconds if at all. import time, sys, logging from twisted.internet import gtk2reactor gtk2reactor.install() ## from wxPython.wx import * ## from twisted.internet import wxsupport from twisted.internet import reactor from twisted.python import threadable threadable.init() from twisted.python import log log.startLogging(sys.stdout) from twisted.internet import app from twisted.manhole import telnet from twisted.application import strports ## For testing with wx reactor ## class myWxApp(wxApp): ## def OnInit(self): ## return True ## mywxAppInstance = myWxApp(0) ## wxsupport.install(mywxAppInstance) class TestThingy: def __init__(self): log.err("TestThingy created") def run(self): log.err("TestThingy run") for i in range(10): log.err("TestThingy message") time.sleep(0.1) log.err("TestThingy done") class ThingyManager: def __init__(self): log.err("ThingyManager") def submitThingy(self): log.err("ThingyManager submitting") f = TestThingy() log.err("ThingyManager created") reactor.callInThread(f.run) log.err("ThingyManager calledInThread") def timer(*args): log.err("Timer: %d" % time.time()) reactor.callLater(1, timer) log.err("Starting") t = ThingyManager() log.err("Starting timer") reactor.callLater(0, timer) log.err("Submitting transfer") reactor.callLater(3, t.submitThingy) log.err("Running reactor") reactor.run() ## for starting the wx reactor ##reactor.run(installSignalHandlers=0) log.err("Reactor done")

3 3

[Twisted-Python] Advice on doing thousands of simultaneous UDP queries with Twisted...
by Mike C. Fletcher Feb. 25, 2004

Feb. 25, 2004

I've written a UDP-based protocol adaptor (TwistedSNMP) where one of the key requirements is the ability to scan thousands of SNMP Agents "simultaneously" (i.e. the requests should be asynchronously sent and retired as the Agents respond). http://members.rogers.com/mcfletch/programming/index.htm#TwistedSNMP Writing a simple asynchronous loop myself (poll on a simple socket, send message from queue when writable, read one into other queue when readable) allowed for doing a few thousand queries simultaneously), with only a few dozen dropped messages. However, with the Twisted equivalent (UDPTransport with my simple protocol object), I was seeing huge drop rates, so, gathering that Twisted isn't queueing up the UDP requests, I wrote a (byzantine) query-throttling mechanism with Twisted defers. Problem is, it's a byzantine, fragile (and *slow*) solution to what would *seem* to be one of the most common requirements in networked development. Worse yet, because I am seeing such high drop rates I wind up having to batch in very small groups, serially (instead of parallel-ly), so the primary purpose of the system (fast querying of thousands of agents) is lost. (Instead of taking 1 or 2 minutes to query 800 or so Agents it will take on the order of 10 minutes.) So, the question: is there a simple way to turn on buffered mode in UDP transports so that they can deal with queueing up a few thousand messages to send, sending them, then having a few thousand computers send a reply (within a few seconds of one another)? Is Twisted just not Queue-reliant via use of some mechanism I haven't discovered yet? Even if I do find a decent queueing mechanism, I'm still left with the problem that timeouts and the like are going to wind up being measured from queueing-time, rather than sending time... not an issue if everything gets sent in a half-second or so, but a real problem if it takes 8 or 9 seconds just to send the original messages out. Looking at the udp.Port class, I'm not seeing anything providing a queue, seems as though there's a non-blocking write or read, but nothing to handle overflows of sends or receives AFAICT, though it looks as though a protocol could do some queueing on incoming in its datagramReceived... just not sure how that would work. Thoughts appreciated, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/

10 11

[Twisted-Python] Re: [Twisted-commits] Preparing for 1.2.0rc3
by Mary Gardiner Feb. 25, 2004

Feb. 25, 2004

3 4

[Twisted-Python] Installation assistance needed
by Alfred Salton Feb. 24, 2004

Feb. 24, 2004

Can anyone point me to instructions for installation on RedHat 9? I can't find any real installation instructions. What am I supposed to do with a 'spec' file? Does an rpm package exist somewhere? I feel defeated before I've started. Alfred

5 4

[Twisted-Python] Problems running Twisted
by Phil Hunt Feb. 23, 2004

Feb. 23, 2004

I'm trying to set up a (very simple) web server using Twisted. I'm using this example from the documentation: ============================================================ # teg1.py from twisted.web import server, resource from twisted.internet import reactor class Simple(resource.Resource): isLeaf = True def render(self, request): return "<html>Hello, world!</html>" site = server.Site(Simple()) reactor.listenTCP(8080, site) reactor.run() ============================================================ I've stored this as <teg1.py>. When I run: $ python teg1.py Python complains that it can't import twisted.web: Traceback (most recent call last): File "teg1.py", line 3, in ? from twisted.web import server, resource ImportError: No module named twisted.web I've previously installed Twisted by running: $ python setup.py install which I thought would install all the modules and put them correctly in the PYTHONPATH (tell me if this isn't true). Am I doing somethnig wrong? I feel I have missed something fundamental, but don't know what. Should I rename my file to <teg1.rpy> or something? Do I need to run at the command-line ``mktap'' instead of ``python''? -- Phil Hunt, phil.hunt(a)tech.mrc.ac.uk

7 15