Mailman 3 June 2008 - Twisted

[Twisted-Python] Urgently Seeking.......
by Sean Burling June 23, 2008

June 23, 2008

Hi people, My name is Sean Burling and I am an account manager for V-hr, an IT recruitment company based in London. I am currently seeking a Senior Python Developer/ Team Lead (ideally with exposure to either Django or Twisted) for a multimedia company based in the heart of Amsterdam. I am struggling to find candidates with the relevant experience and as such wanted to reach out to the list to find out if any of you or your contacts are interested in new opportunities in Europe. … [View More]

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 22, 2008

June 22, 2008

1 0

[Twisted-Python] reject all open connections
by Irae Hueck Costa June 21, 2008

June 21, 2008

hi all! i just want to clean my reactor - all readers and writer should looseConnection() but how can i do this? reactor.removeAll() dont do it. the best solutions i found was a horrible workaround: a thread is tarted, which constantly calls ``[conn.loseConnection() for conn in reactor.removeAll()]`` is there a better solution? thank you _________________________________________________________________ Conheça o Windows Live Spaces, a rede de relacionamentos do Messenger! http://www.amigosdomessenger.com.br/

1 0

Re: [Twisted-Python] Twisted Trial
by Jake b June 20, 2008

June 20, 2008

On Wed, Jun 18, 2008 at 3:48 AM, Torsten A. <c2k_list(a)gmx.de> wrote: > Thank you all for your replies. > > @Christian: > I should have mentioned that. The current directory is in the pythonpath. > In > the variable PYHTONPATH the dot is set and when I walk through the > filesystem, starting python from different locations, the current path is > always set in sys.path. Hence, I asssume that PythonPath is set correctly. > So your suggestion, what I thought … [View More]

3 2

[Twisted-Python] Twisted, twistd and logging
by Tim Allen June 20, 2008

June 20, 2008

Hi, all. I'm working on a new system to be deployed in our production environment, and having already decided on using Twisted, I'm now trying to figure out how to cleanly and reliably deploy it. I have many questions, but right now I just want to work through the issue of logging. In our existing systems, our code writes messages to syslog, with 'facility' set to LOG_LOCAL0 and 'priority' set according to the severity of the event - debug, info, warn or alert. Anything logged with 'alert' … [View More]

5 7

[Twisted-Python] chaining deferreds
by Andy Hird June 19, 2008

June 19, 2008

Hi all, I've been trying to get a really basic example of chaining deferreds going with no luck. I was hoping someone could give me some help and tell me where I'm going wrong. I've attached the basic code to this message. I'd expect output to be: constructed deferred setup everything printing: hello mr doing something else: printed something finishing up with: done something else But instead the callback chain stops after print_something() has been called (do_something inbetween and … [View More]

2 1

[Twisted-Python] RE: ipython & twisted: no module named threading
by Mike Winter (miwinter) June 19, 2008

June 19, 2008

This is a record of analysis and consequent workaround to enable running ipython and Twisted's trial. Posting to twisted, it probably requires no action from twisted team since I suspect IPython is not dealing with the situation correctly. Ipython needs to resolve the problem w/ a try: import thread except ImportException: print "failed to import thread" import dummy_thread as thread For ipython to 'work' I copy dummy_thread.py thread.py, then when I run trial or twistd I see the … [View More]following exception. If I delete the thread.py this does not show up, but ipython stops working. Exception exceptions.ImportError: 'No module named threading' in <bound method local.__del__ of <_threading_local.local object at 0xb72f6874>> ignored (venv)[miwinter@sawwing test]$ twistd -ny finger.tac shows the same exception (from twisted's finger tutorial). Python -V 2.4.5 and 2.5.1 and 2.5.2 Ipython-0.8.2 * Twisted 8.1.0 virtualenv 1.0 CVSToys 1.0.10 python -m virtualenv venv . venv/bin/activate (venv)# make InstallTwisted (cd Twisted-8.1.0; python setup.py install) (venv)# make InstallBuildbot (easy_install buildbot) (venv)# trial buildbot.test.test_twisted .... Shows the above problem: [skip] PASSED (skips=37, successes=271) Exception exceptions.AttributeError: "'NoneType' object has no attribute 'StringIO'" in <bound method RemoteReference.__del__ of <twisted.spread.pb.RemoteReference instance at 0xb6856aec>> ignored Exception exceptions.ImportError: 'No module named threading' in <bound method local.__del__ of <_threading_local.local object at 0xb72dacfc>> ignored test main-trunk]$ md5sum lib/python2.4/thread*py lib/python2.4/dummy_thread*py 3783492a0969521ff1266ec6c3e570c2 lib/python2.4/threading.py 38db2323ac32ddbcad69888abd30d21f lib/python2.4/dummy_threading.py 52a6d92c98cc5a2eb68a3b1dde16e58b lib/python2.4/thread.py 52a6d92c98cc5a2eb68a3b1dde16e58b lib/python2.4/dummy_thread.py Without thread.py ipython will fail to start, so in my makefile I do: fixThread: bin/python lib/python$(PY_MACRO_VERSION)/thread.py lib/python$(PY_MACRO_VERSION)/thread.py: (cd lib/python$(PY_MACRO_VERSION); cp dummy_thread.py thread.py) This exception (see start of email) is perhaps caused by dummy.py lacking some of the guts that some inappropriately intimate part of threading.py depends upon. w/o target fixThread ipython will not start! Since resolving this by bandaiding Makefile I thought it worthy to report that my patch to my Makefile was to add target unfixThread: to reverse the effects of fixThread which was a hack for ipython to work, better fix ipython somehow. [View Less]

1 0

[Twisted-Python] web client and proxy
by Olivier Fournier June 19, 2008

June 19, 2008

Hi, My english is very bad, sorry... I don't understand how to access to a web server behind a proxy from web.client ? somebody could explain me how to do? thanks Olivier Fournier

2 1

[Twisted-Python] Something like a ThreadPool
by kevin beckford June 18, 2008

June 18, 2008

Is there any sort of job queue or similar , besides AsynQueue, which looked like exactly what i wanted, but is sadly undocumented, native to twisted? What is the twisted way to do such a thing? Is there something already available? -- Kevin Beckford Technical Lead, Lazyweb Construction Company, http://lazyweb.ca kevin(a)lazyweb.ca

4 6

[Twisted-Python] The twisted way for basic file i/o?
by Nathan June 18, 2008

June 18, 2008

I just want to do some basic text-file manipulations without blocking the rest of twisted. I've found "FilePath", which has an open()...which is undocumented: http://twistedmatrix.com/documents/current/api/twisted.python.filepath.File… Is that what I should be using??? ~ Nathan

2 2

Re: AW: [Twisted-Python] Twisted Trial
by Curt June 18, 2008

June 18, 2008

Torsten A. wrote: > I have never really used a version control system I have to admit. I thought > of control systems to log changes which can in case of severe damage or what > ever reason be made undone. But if I can execute different versions of the > project, e.g. in my case the initial version and the last modified, just by > saying run this one or that one, I am glad to use it. Can this be done > (excuse my probably dumb question)? Can someone please give me catchword to … [View More]

1 0

Re: [Twisted-Python] Twisted Trial
by Terry Jones June 18, 2008

June 18, 2008

The various answers recommending version control are just heading off in a direction that wasn't asked about at all! :-) Sorry if I sound grouchy, but the OP hasn't done anything to warrant being curtly told to just use version control, especially when that's not a solution to the problem he's running into. There should be no need to add that people often have multiple versions (branches, often) checked out and want to swap between using them and have the right version picked up by Python. … [View More]

2 1

[Twisted-python] Writing industrial grade twisted servers
by kevin beckford June 18, 2008

June 18, 2008

Hi all, I've been reading through the twisted documentation, trying to get a handle on how to write a real life twisted app. ( I need to use the AsynQueue for my project, btw..) So I've been reading the documentation, and I've been putting together some things.. but i note that much of the docs says that "this is not the way you usually do this" Where do i read to find out how to simply create a production quality twisted application? -- Kevin Beckford Technical Lead, Lazyweb Construction … [View More]

7 6

[Twisted-Python] Executing a number of commands over SSH
by James Brady June 17, 2008

June 17, 2008

Hi all, I'd like to run a few commands on a remote machine over SSH. Looking at the conch packages, I'm not sure what the best approach is for this. The only way I can see to pass the set of commands down into the SSHConnection and SSHChannel objects is through my transport factory, i.e. I can't add or remove commands to be run on an already instantiated SSHConnection. Also, I'm a newcomer to the Twisted way of doing things, so it's not clear to me how I would go about returning the … [View More]

2 2

Re: [Twisted-Python] Re-working a synchronous iterator to use Twisted
by Terry Jones June 17, 2008

June 17, 2008

For the record, here's a followup to my own posting, with working code. The earlier untested code was a bit of a mess. The below runs fine. In case it wasn't clear before, you're pulling "results" (e.g., from a search engine) in off the web. Each results pages comes with an indicator to tell you whether there are more results. I wanted to write a function (see processResults below) that, when called, would call the process function below on each result, all done asynchronously. This solution … [View More]

1 0

Re: [Twisted-Python] Twisted Trial
by Andy Fundinger June 17, 2008

June 17, 2008

It's not quite an answer to your immediate question, but I would suggest setting up Subversion (http://tortoisesvn.tigris.org/) to track your versions. That way you can keep all your versions modified, partially modified, unmodified, etc. and pull back any old version as appropriate. This looks like a tutorial for the kind of setup you'd need for that: http://www.onlamp.com/lpt/a/2820 -Andy On Tue, Jun 17, 2008 at 2:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I … [View More]

4 3

Re: [Twisted-Python] Twisted Trial
by Christian Simms June 17, 2008

June 17, 2008

On Tue, Jun 17, 2008 at 2:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I received a running project using twisted trial, running meaning that all > test units pass successfully, which I am supposed to edit. Now I'd like to > keep two versions of the project: > > a) A backup of the old one, so I can always have it successfully run the > unittests and keep track of its behavious and > b) a copy of the project which I want to modify and test, if the trials … [View More]

1 0

Re: [Twisted-Python] Twisted Trial
by Christopher Armstrong June 17, 2008

June 17, 2008

On Tue, Jun 17, 2008 at 7:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I received a running project using twisted trial, running meaning that all > test units pass successfully, which I am supposed to edit. Now I'd like to > keep two versions of the project: > > a) A backup of the old one, so I can always have it successfully run the > unittests and keep track of its behavious and > b) a copy of the project which I want to modify and test, if the trials … [View More]

1 0

[Twisted-Python] Twisted Trial
by Torsten A. June 17, 2008

June 17, 2008

Hello, I received a running project using twisted trial, running meaning that all test units pass successfully, which I am supposed to edit. Now I'd like to keep two versions of the project: a) A backup of the old one, so I can always have it successfully run the unittests and keep track of its behavious and b) a copy of the project which I want to modify and test, if the trials pass. I have tried to mess with the PYTHONPATH, to run trial in the folder of the copies, but nothing seems to work.… [View More]

1 0

[Twisted-Python] Re-working a synchronous iterator to use Twisted
by Terry Jones June 17, 2008

June 17, 2008

I'm trying to rework some synchronous code to use Twisted and I've hit an interesting case. The following code is my simplification of the situation. I've not run it, but will happily flesh it out if people want this to be more concrete. First, a general helper function for use below: def process(result): # process a single result. pass I have the following simple synchronous function and code that calls it: # SYNCHRONOUS result producer def getResults(uri): … [View More] done = False offset = 0 while not done: results, done, offset = fetchPageViaSynchronousHttp(uri, offset) for result in results: yield result # SYNCHRONOUS calling for result in getResult(uri): process(result) I.e., there are an indeterminate number of results available out there via some web page. The iterator above periodically goes to get more, and yields the new batch one by one to the calling code which processes them one by one. This has the advantage that process() can be called as soon as any results are available. So how to do the above (more or less) in Twisted? Here's one approach: def getResults(page): # Helper func: get results from page, return them in a list pass def needToCallAgain(page): # Helper func: see if there are more results, given page. return bool. pass # ASYNCHRONOUS result producer def getResults(uri, offset=0): def parsePage(page, offset): results = getResults(page) if needToCallAgain(page): d = getResults(uri, offset + len(results)) else: d = None return results, d def returnTheseResults(page, offset): resultIterator, done, offset = parsePage(page, offset) return resultIterator, done from twisted.web import client return client.getPage(uri).addCallback(returnTheseResults, offset) # ASYNCHRONOUS calling def processResults(uri): def cb(resultIterator, deferred): for result in resultIterator: process(result) if deferred is not None: deferred.addCallback(cb) return getResults(uri).addCallback(cb) I'm fairly sure something like this can be made to work. The idea is to have getResults call a callback that takes a current set of results and a new deferred that will be called with the next set of results and a new deferred that will be called with the next set of results and...... so on until the deferred that comes back is None, at which point you're done (i.e., there are no more results). The result is that process() gets called asynchronously with incoming results, and, as with the synchronous approach, we don't have to wait until the whole result set is in before we can begin processing results. But this approach is definitely not simple (or at least it's not, given my beginner-level of sophistication producing and consuming deferreds). Note too that you don't actually use the deferred coming back from processResults. I wanted to have the fun of thinking about this and writing my own pseudo-solution before I posted here, but I imagine that people working with Twisted must have many times dealt with something like the above. How would you handle it? Other approaches are also possible, but I'll stop for now to see what people say. Let me know if you want the above fleshed out to working code. Regards, Terry [View Less]

1 0

[Twisted-Python] Twisted service kit
by Johan Euphrosine June 16, 2008

June 16, 2008

Hi, As part of a freelance job, I'm asked to glue twisted with other technologies together as a proof of concept of what be done (and how), in the context of a service communicating with C++ games. As it may be useful to someone else, I'm pushing the source here: http://proppy.aminche.com/hg/service-test/ This is what the code features/shows so far: - JSON protocol over Int32StringReceived - C++ sample client using boost.asio and Json_Spirit - server and client OpenSSL endpoint - XMLRPC … [View More]

5 6

[Twisted-Python] TX project guidelines (was Re: Twisted service kit)
by Phil Christensen June 15, 2008

June 15, 2008

On Jun 13, 2008, at 1:26 PM, Kevin Horn wrote: > On Fri, Jun 13, 2008 at 12:07 PM, Phil Christensen <phil(a)bubblehouse.org > > wrote: >> The great thing about the Twisted Community Code project is that >> you don't need any particular approval from anyone to add your >> code. If you think what you have might be of use to someone, >> whether it's for production use or just as a learning tool, you >> should consider creating a TX subproject. … [View More]>> >> There are other options as well. If you feel your code doesn't >> warrant the overhead of a full project (e.g., you don't see >> yourself having time to respond to bug requests or create >> blueprints, etc.), you should be able to create a personal Bazaar >> branch to store your code in, which you can then assign to the TX >> superproject. >> >> This probably won't get as many eyeballs, since it won't be on the >> subproject list, but it's a good way to get your code out there in >> an easily accessible place. Plus, given the nature of Bazaar, >> anyone who wishes will be able to work with your code in their own >> branch. >> >> -phil > > BTW, Phil, are there any established guidelines/conventions for the > "TX" packages other than "start your project name with a 'tx'"? Not really. Even the 'tx' prefix is just a suggestion. I think adding the prefix is a good idea if your project has a generic name, like a protocol, but it's up to you. Using it is a nice show of 'community solidarity', but I know I have a couple of projects I hope to add soon where it would just look or sound weird. > Any suggestions or guidelines on how such packages should leverage > setuptools, stuff like that? > > I think it would be very useful to have all of these packages > install in a similar fashion, though of course, it wouldn't be > practical (or advisable) to try and enforce this in any way. But I > do think it would be good to have a set of guidelines posted > someplace saying "here is the recommended way to set up a TX package". Agreed. I think deployment options like this are best decided on a per- project basis, since TX will hopefully be a home for example code and works-in-progress as well as true project releases. Still, I think we can all agree that projects should be installable using distutils at the very least. The key thing here is that for most projects you should be able to "python setup.py install" and be done with it. Does anyone have any other suggestions? If we can come up with a list of suggested criteria, I'll write up a spec to add to the TX blueprints. -phil [View Less]

4 4

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 15, 2008

June 15, 2008

1 0

[Twisted-Python] PerspectiveClientManager.disconnect() - results in Unhandled error in Deferred
by happybrowndog June 12, 2008

June 12, 2008

Hello, I am calling the disconnect() method of an instance of the PerspectiveClientManager class, but when is is called, I get the following error: Unhandled error in Deferred: Traceback (most recent call last): Failure: twisted.spread.pb.PBConnectionLost: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionDone: Connection was closed cleanly. ] I have no idea what to look for as to what is causing this error. Does someone have any idea?

2 1

[Twisted-Python] TSF Sponsored Development
by Jean-Paul Calderone June 12, 2008

June 12, 2008

Hello all, On Friday I finished the first two weeks of funded Twisted development, I'm happy to say the two weeks went great and I got a lot accomplished. The very, very high-level summary is that I started off by emptying the Twisted review queue and then moved on to finishing some twisted.web tasks which had been started previously by other developers and then abandoned for lack of time or interest or whatever other reason. All told, I did 35 reviews on 29 tickets and did development on 8 … [View More]others. I also spent some time cleaning duplicate or invalid tickets out of the Twisted tracker. This collectively resulted in 32 tickets being closed over the past two weeks. Here are the tickets closed as a direct result of this work: #1789 - twisted.conch.test.test_filetransfer.TestOurServerOurClient.testOpenFileAttributes (and perhaps others) leaks file descriptors #3213 - twisted.conch.ssh.filetransfer.FileTransferServer leaks file descriptors #3203 - PortableGtkReactor is broken #3098 - filepath.FilePath.walk loops forever if there is a symlink pointing back up the file tree #3033 - mkdir(a); mkdir(a) logs an unhandled error in SFTP #3054 - Small error in the documentation for pb #2881 - Add a variable in twisted.python.runtime to check for vista #2374 - testOpenSSLBuffering fails intermittently on a dapper machine (running in vmware) because it uses reactor.iterate #2918 - intermittent twisted.test.test_ssl.SpammyTLSTestCase.testUnTLS failure on OS X #3210 - test_pb.NewCredTestCase is insanely long and causes some intermittent failures #3119 - trial leaks memory/objects/something #3052 - twistd logging options needs to be tested #3266 - Provide tools for managing new deprecation policy #3191 - in manhole in gnome-terminal_ lines longer than the terminal is wide result in excessive scrolling and duplication #3105 - symlink support and docstrings in filepath.FilePath.copyTo/moveTo #3267 - Typo in the Asynchronous Programming howto #3184 - Trial minimal reporter prints a negative run time in summary #1410 - spawnProcess on unix doesn't handle case where os.fork fails cleanly #2305 - twisted/internet/process.py leaks file descriptors when os.fork() raises #3272 - sorting the --profile output in trial #3259 - Improved some error messagge in reflect.py #2713 - trial -u can fail doctests on second pass #2698 - Build Bug on Cygwin #914 - xmlrpc.QueryFactory failed to catch some exceptions #1057 - Incorrect and misleading arg name msSinceEpoch #2017 - t.w.client.getPage returning ConnectionDone: Connection was closed cleanly error #1412 - ExpatError on xmlrpc client connection close #393 - [PATCH] HTTP Basic Auth based guard #1108 - web2 needs SOAP support #909 - microdom getElementsByTagName #687 - web.client needs documentation and/or data-checking for method argument #165 - [PATCH]Multiple header with same name support in http Of particular note, I think, are the following: #3213: Prior to this fix, the Twisted SFTP server would leak any file descriptors not closed before the end of an SFTP session. #3203: This fix restores Gtk2 support on Windows. We have also set up a Gtk2 Windows builder which is now passing all but one test. #1410: Prior to this fix, an error when calling os.fork() would cause reactor.spawnProcess to leak any pipes it opened and potentially to continue running in the main process with the wrong UID/GID. #165: This adds support for multiple values for each header to the twisted.web server. Previously only the last value received would be made available to applications. Six more tickets are waiting to be reviewed and will hopefully be closed soon. I want to take this opportunity to thank everyone who worked on tickets I reviewed over the past couple weeks. Twisted development is a collaborative process and I wouldn't have made nearly as big a dent without all of their help. I also want to thank Thomas Hervé in particular, who authored a lot of the code I reviewed and also (if I kept track properly) did every review of code I submitted over the the past two weeks. In addition to doing work on tickets from the tracker, I also addressed two other issues with Twisted's buildbot. First, I took one of the compact build result views and split it into a supported-only view and an everything view to make it easier to find out the status of recent builds on supported platforms. These two views are linked from the main Twisted buildbot page, http://buildbot.twistedmatrix.com/ - see the "Latest builds" link. Second, I improved the error reporting for the setuptools-enabled builder, making it easier to track down the cause of some setuptools-related problems. Thanks to the SFC (<http://conservancy.softwarefreedom.org/>) and all of the sponsors (<http://twistedmatrix.com/trac/wiki/TSF/FoundingSponsors>) who made this possible. Jean-Paul [View Less]

3 2