Mailman 3 March 2007 - Twisted

[Twisted-Python] Can I still run (most of) Twisted on Python 2.2?
by Erik Forsberg March 27, 2007

March 27, 2007

Hi! In TwistedCore/setup.py, I find that python 2.3 is required for Twisted. I'd like to use Twisted for a project that currently requires only Python 2.2, which raises the question - which parts of Twisted requires 2.3, and how much of it can I run on Python 2.2? On the same matter - are there any plans on when to stop supporting Python 2.3, Python 2.4, etc? Thanks, \EF -- Erik Forsberg OpenSource-based Thin Client Technology Systems Analyst/Developer Phone: +46-13-21 46 … [View More]

6 7

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com March 25, 2007

March 25, 2007

1 0

[Twisted-Python] thread pool question
by Rutt, Benjamin March 23, 2007

March 23, 2007

From http://twistedmatrix.com/projects/core/documentation/howto/threading.htm l: I copied the code below and ran it as: $ cat twisted-threadpool.py #!/usr/bin/env python from twisted.internet import reactor def aSillyBlockingMethod(x): import time time.sleep(2) print x # run method in thread reactor.callInThread(aSillyBlockingMethod, "2 seconds have passed") $ ./twisted-threadpool.py 2 seconds have passed But I note that the program never stops. I infer that … [View More]

4 7

[Twisted-Python] Twisted friendly hosting services?
by Vincent Cate March 23, 2007

March 23, 2007

I am working on an application that runs using twisted.web. I want to host it on a machine with plenty of bandwidth as cheaply as possible. Is there a hosting service that uses Twisted.web so that I can just add my resource to the namespace? Or will I have to get a "virtual dedicated server" or a real "dedicated server"? If so, does anyone know a virtual one that uses Ubuntu? Thanks, -- Vince

4 3

[Twisted-Python] Using twisted.conch
by pico March 22, 2007

March 22, 2007

Hi all. I'm struggling a bit with twisted.conch. I've read the howto at http://twistedmatrix.com/projects/conch/documentation/howto/conch_client.ht…. I can understand this example fine. For a simple application, it works fine to subclass SSHChannel and implement all the code in that classes methods. However I would like to be able to implement something as follows: class SSHWrapper: def connect(self,hostname,username,password): #Create SshTransport object def runCommand(self,… [View More]

2 1

[Twisted-Python] Twistd memory usage
by Jarek Zgoda March 22, 2007

March 22, 2007

I have an application, ran as twistd plugin. If I run it as daemon, twistd consumes all available memory (of 4GB available), then after few seconds of swapping twistd is killed by kernel's OOM killer. I cann't reproduce this when I run this application in no-daemon mode (twistd -n), the memory footprint is constant and does not exceed 150MB. Is this normal behaviour? -- Jarek Zgoda "We read Knuth so you don't have to."

2 2

[Twisted-Python] hardcoded max in threadpool or connection pool ?
by Jonathan Vanasco March 21, 2007

March 21, 2007

forgive my ignorance this week - i'm rusty on twisted. I seem to be hitting a limit on threads / db connection ( not sure which one , as db could block connection ) at 11 The only code i have influencing the sizes are as follows: ======= class MY_dbConnectionPool( adbapi.ConnectionPool ): def __init__( self ,): adbapi.ConnectionPool.__init__( self, 'psycopg2' , host="localhost", database="xxxxx", user="xxxxx", password="xxxxx", cp_min= 16, cp_max= 16 ) ======= … [View More]

1 0

[Twisted-Python] suggestion for tracking down cause of 2 bugs sought: possibly threads / defer.DeferredSemaphore ?
by Jonathan Vanasco March 21, 2007

March 21, 2007

I've got 2 bugs in a twisted app that I'm trying to narrow in on. Overview of the app: A twisted service connects to a db and requests about 500 web pages to download and analyze. The service than uses defer.DeferredSemaphore service to handle dispatching each request to a thread via deferToThread . defer.DeferredSemaphore( tokens= MAX_DOWNLOADERS ) reactor.suggestThreadPoolSize( MAX_DOWNLOADERS + 1 ) Each thread runs a python script that is not written in Twisted and has a lot of … [View More]blocking. And to pre-empt many comments that will inevitably pop up: It would be great to convert all the code to twisted , but 90% of the code + underlying modules are used by command line maintenance routines that were already in place. I just don't have the time or resources to rewrite it all in twisted. One day I'd love to have the funding to hire someone to manage them, but I just don't have that time myself. Overview of the bugs: 1) The query for figuring out which pages to download can take quite some time , so I run the getNextDownloads routine in a new thread , so the pool always remains full. It works perfect, except when the number of items queued up to defer.DeferredSemaphore reaches ~ 1000. After ~1000, I seem to only be running with 1-2 semaphores in place not 15 Question: Can anyone suggest a way to test this to see where the error is? Off the top of my head, the following seem plausible: a- hitting 1000 just breaks something in defer.DeferredSemaphore . the module is listed as unstable, so I'm guessing this could be the cause. b- the defer.DeferredSemaphore connection is just coincidental -- its really an issue with threads locking up c- something entirely different Unfortunately, I'm at a loss on trying to figure out whats going on. 2) I'm experiencing an obscene amount of process growth. I'm fairly certain that its from constant errors in urllib2 , which has been nothing of a nightmare with countless known bugs and patches ( downloading takes place via Mechanize , which is wrapped via urllib2 ), and makes me consider just wrapping wget or curl. but this could very well be threading related. Question: Is there any way to profile memory usage by twisted in a granular fashion ? I'd like to figure out where this memory usage is ( it grows steadily to ~1.7gb after 8 hours ). Thanks, // Jonathan Vanasco | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | SyndiClick.com | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | FindMeOn.com - The cure for Multiple Web Personality Disorder | Web Identity Management and 3D Social Networking | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - | RoadSound.com - Tools For Bands, Stuff For Fans | Collaborative Online Management And Syndication Tools | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [View Less]

2 2

[Twisted-Python] Twisted.Names assistance with auth and additional sections
by Sean Leach March 21, 2007

March 21, 2007

Hey all, I have a client I have written using twisted.names that will send a DNS request. Here is the gist of the relevant code: (tfp, tfile) = tempfile.mkstemp() os.write(tfp, 'nameserver %s' % ip) os.close(tfp) resolver = client.createResolver(resolvconf=tfile) deferred = resolver.lookupAddress(somevalue) deferred.addCallback(resultHandler) hacky way to set the IP, but I am having issues with passing in the "servers" list I don't want to troubleshoot until I figure … [View More]

4 9

[Twisted-Python] question about thread and threadpool
by wang wei March 21, 2007

March 21, 2007

the document said: # run method in thread reactor.callInThread(aSillyBlockingMethod, "2 seconds have passed") but after above code, Is that Thread dead? and how can I re-use this thread? and if I create a thread using python thread module, can I use reactor to re-use it? there is a threadpool in twisted, is there any example of how to use it? thanks for any advices.

2 2