Mailman 3 June 2008 - Twisted

Re: [Twisted-Python] Twisted Trial
by Andy Fundinger 18 Jun '08

18 Jun '08

It's not quite an answer to your immediate question, but I would suggest setting up Subversion (http://tortoisesvn.tigris.org/) to track your versions. That way you can keep all your versions modified, partially modified, unmodified, etc. and pull back any old version as appropriate. This looks like a tutorial for the kind of setup you'd need for that: http://www.onlamp.com/lpt/a/2820 -Andy On Tue, Jun 17, 2008 at 2:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I received a running project using twisted trial, running meaning that all > test units pass successfully, which I am supposed to edit. Now I'd like to > keep two versions of the project: > > a) A backup of the old one, so I can always have it successfully run the > unittests and keep track of its behavious and > b) a copy of the project which I want to modify and test, if the trials > pass. > > I have tried to mess with the PYTHONPATH, to run trial in the folder of the > copies, but nothing seems to work. Right now it looks like this (running > under WinXP): > > C:\ > some_folder\ > working_project\here_the_module_to_be_checked > modified_project\here_the_module_to_be_checked > > My PYTHONPATH links to C:\some_folder\working_project. > > I hopesome understands what I mean ;) > > Cheers, > Torsten > > > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > -- Blog: http://channel3b.wordpress.com Second Life Name: Ciemaar Flintoff I am a sig Virus. Please put me in your sig so that I can continue to replicate.

4 3

Re: [Twisted-Python] Twisted Trial
by Christian Simms 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 2:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I received a running project using twisted trial, running meaning that all > test units pass successfully, which I am supposed to edit. Now I'd like to > keep two versions of the project: > > a) A backup of the old one, so I can always have it successfully run the > unittests and keep track of its behavious and > b) a copy of the project which I want to modify and test, if the trials > pass. > > I have tried to mess with the PYTHONPATH, to run trial in the folder of the > copies, but nothing seems to work. Right now it looks like this (running > under WinXP): > > C:\ > some_folder\ > working_project\here_the_module_to_be_checked > modified_project\here_the_module_to_be_checked > > My PYTHONPATH links to C:\some_folder\working_project. > > I hopesome understands what I mean ;) > > Cheers, > Torsten > > > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > I think this is a basic Python question. If you set your PYTHONPATH variable to . (period), then when you import a python package it will look in the current working directory for the package, and then the standard system directories. This way, to test both the original and modified versions, you would do: cd \original_project trial package_to_test cd \modified_project trial package_to_test Cheers, Christian

1 0

Re: [Twisted-Python] Twisted Trial
by Christopher Armstrong 18 Jun '08

18 Jun '08

On Tue, Jun 17, 2008 at 7:14 PM, Torsten A. <c2k_list(a)gmx.de> wrote: > Hello, > I received a running project using twisted trial, running meaning that all > test units pass successfully, which I am supposed to edit. Now I'd like to > keep two versions of the project: > > a) A backup of the old one, so I can always have it successfully run the > unittests and keep track of its behavious and > b) a copy of the project which I want to modify and test, if the trials > pass. Dude, use revision control: http://bazaar-vcs.org/ http://subversion.tigris.org/ Keeping copies of stuff manually is so 1970s. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

1 0

[Twisted-Python] Twisted Trial
by Torsten A. 18 Jun '08

18 Jun '08

Hello, I received a running project using twisted trial, running meaning that all test units pass successfully, which I am supposed to edit. Now I'd like to keep two versions of the project: a) A backup of the old one, so I can always have it successfully run the unittests and keep track of its behavious and b) a copy of the project which I want to modify and test, if the trials pass. I have tried to mess with the PYTHONPATH, to run trial in the folder of the copies, but nothing seems to work. Right now it looks like this (running under WinXP): C:\ some_folder\ working_project\here_the_module_to_be_checked modified_project\here_the_module_to_be_checked My PYTHONPATH links to C:\some_folder\working_project. I hopesome understands what I mean ;) Cheers, Torsten

1 0

[Twisted-Python] Re-working a synchronous iterator to use Twisted
by Terry Jones 17 Jun '08

17 Jun '08

I'm trying to rework some synchronous code to use Twisted and I've hit an interesting case. The following code is my simplification of the situation. I've not run it, but will happily flesh it out if people want this to be more concrete. First, a general helper function for use below: def process(result): # process a single result. pass I have the following simple synchronous function and code that calls it: # SYNCHRONOUS result producer def getResults(uri): done = False offset = 0 while not done: results, done, offset = fetchPageViaSynchronousHttp(uri, offset) for result in results: yield result # SYNCHRONOUS calling for result in getResult(uri): process(result) I.e., there are an indeterminate number of results available out there via some web page. The iterator above periodically goes to get more, and yields the new batch one by one to the calling code which processes them one by one. This has the advantage that process() can be called as soon as any results are available. So how to do the above (more or less) in Twisted? Here's one approach: def getResults(page): # Helper func: get results from page, return them in a list pass def needToCallAgain(page): # Helper func: see if there are more results, given page. return bool. pass # ASYNCHRONOUS result producer def getResults(uri, offset=0): def parsePage(page, offset): results = getResults(page) if needToCallAgain(page): d = getResults(uri, offset + len(results)) else: d = None return results, d def returnTheseResults(page, offset): resultIterator, done, offset = parsePage(page, offset) return resultIterator, done from twisted.web import client return client.getPage(uri).addCallback(returnTheseResults, offset) # ASYNCHRONOUS calling def processResults(uri): def cb(resultIterator, deferred): for result in resultIterator: process(result) if deferred is not None: deferred.addCallback(cb) return getResults(uri).addCallback(cb) I'm fairly sure something like this can be made to work. The idea is to have getResults call a callback that takes a current set of results and a new deferred that will be called with the next set of results and a new deferred that will be called with the next set of results and...... so on until the deferred that comes back is None, at which point you're done (i.e., there are no more results). The result is that process() gets called asynchronously with incoming results, and, as with the synchronous approach, we don't have to wait until the whole result set is in before we can begin processing results. But this approach is definitely not simple (or at least it's not, given my beginner-level of sophistication producing and consuming deferreds). Note too that you don't actually use the deferred coming back from processResults. I wanted to have the fun of thinking about this and writing my own pseudo-solution before I posted here, but I imagine that people working with Twisted must have many times dealt with something like the above. How would you handle it? Other approaches are also possible, but I'll stop for now to see what people say. Let me know if you want the above fleshed out to working code. Regards, Terry

1 0

[Twisted-Python] Twisted service kit
by Johan Euphrosine 16 Jun '08

16 Jun '08

Hi, As part of a freelance job, I'm asked to glue twisted with other technologies together as a proof of concept of what be done (and how), in the context of a service communicating with C++ games. As it may be useful to someone else, I'm pushing the source here: http://proppy.aminche.com/hg/service-test/ This is what the code features/shows so far: - JSON protocol over Int32StringReceived - C++ sample client using boost.asio and Json_Spirit - server and client OpenSSL endpoint - XMLRPC with credential checking using twisted.web and twisted.cred - packaging as a twisted plugin - connection round robbin using pydirector - inter node communication using twisted.spread perspective broker - db persistance using storm ORM - RPC registry using python decorator - testing using twisted.trial and inlineCallbacks I wonder if it is worth contributing on its own, to the newly created TX project. Maybe some parts of it could be extracted, and made into a more submittable (and useful) shape. Any thoughts ? Thanks to irc://irc.freenode.net/#twisted peoples for 24h/day help and advises. -- Johan Euphrosine <proppy(a)aminche.com>

5 6

[Twisted-Python] TX project guidelines (was Re: Twisted service kit)
by Phil Christensen 16 Jun '08

16 Jun '08

On Jun 13, 2008, at 1:26 PM, Kevin Horn wrote: > On Fri, Jun 13, 2008 at 12:07 PM, Phil Christensen <phil(a)bubblehouse.org > > wrote: >> The great thing about the Twisted Community Code project is that >> you don't need any particular approval from anyone to add your >> code. If you think what you have might be of use to someone, >> whether it's for production use or just as a learning tool, you >> should consider creating a TX subproject. >> >> There are other options as well. If you feel your code doesn't >> warrant the overhead of a full project (e.g., you don't see >> yourself having time to respond to bug requests or create >> blueprints, etc.), you should be able to create a personal Bazaar >> branch to store your code in, which you can then assign to the TX >> superproject. >> >> This probably won't get as many eyeballs, since it won't be on the >> subproject list, but it's a good way to get your code out there in >> an easily accessible place. Plus, given the nature of Bazaar, >> anyone who wishes will be able to work with your code in their own >> branch. >> >> -phil > > BTW, Phil, are there any established guidelines/conventions for the > "TX" packages other than "start your project name with a 'tx'"? Not really. Even the 'tx' prefix is just a suggestion. I think adding the prefix is a good idea if your project has a generic name, like a protocol, but it's up to you. Using it is a nice show of 'community solidarity', but I know I have a couple of projects I hope to add soon where it would just look or sound weird. > Any suggestions or guidelines on how such packages should leverage > setuptools, stuff like that? > > I think it would be very useful to have all of these packages > install in a similar fashion, though of course, it wouldn't be > practical (or advisable) to try and enforce this in any way. But I > do think it would be good to have a set of guidelines posted > someplace saying "here is the recommended way to set up a TX package". Agreed. I think deployment options like this are best decided on a per- project basis, since TX will hopefully be a home for example code and works-in-progress as well as true project releases. Still, I think we can all agree that projects should be installable using distutils at the very least. The key thing here is that for most projects you should be able to "python setup.py install" and be done with it. Does anyone have any other suggestions? If we can come up with a list of suggested criteria, I'll write up a spec to add to the TX blueprints. -phil

4 4

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 15 Jun '08

15 Jun '08

1 0

[Twisted-Python] PerspectiveClientManager.disconnect() - results in Unhandled error in Deferred
by happybrowndog 12 Jun '08

12 Jun '08

Hello, I am calling the disconnect() method of an instance of the PerspectiveClientManager class, but when is is called, I get the following error: Unhandled error in Deferred: Traceback (most recent call last): Failure: twisted.spread.pb.PBConnectionLost: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionDone: Connection was closed cleanly. ] I have no idea what to look for as to what is causing this error. Does someone have any idea?

2 1

[Twisted-Python] TSF Sponsored Development
by Jean-Paul Calderone 12 Jun '08

12 Jun '08

Hello all, On Friday I finished the first two weeks of funded Twisted development, I'm happy to say the two weeks went great and I got a lot accomplished. The very, very high-level summary is that I started off by emptying the Twisted review queue and then moved on to finishing some twisted.web tasks which had been started previously by other developers and then abandoned for lack of time or interest or whatever other reason. All told, I did 35 reviews on 29 tickets and did development on 8 others. I also spent some time cleaning duplicate or invalid tickets out of the Twisted tracker. This collectively resulted in 32 tickets being closed over the past two weeks. Here are the tickets closed as a direct result of this work: #1789 - twisted.conch.test.test_filetransfer.TestOurServerOurClient.testOpenFileAttributes (and perhaps others) leaks file descriptors #3213 - twisted.conch.ssh.filetransfer.FileTransferServer leaks file descriptors #3203 - PortableGtkReactor is broken #3098 - filepath.FilePath.walk loops forever if there is a symlink pointing back up the file tree #3033 - mkdir(a); mkdir(a) logs an unhandled error in SFTP #3054 - Small error in the documentation for pb #2881 - Add a variable in twisted.python.runtime to check for vista #2374 - testOpenSSLBuffering fails intermittently on a dapper machine (running in vmware) because it uses reactor.iterate #2918 - intermittent twisted.test.test_ssl.SpammyTLSTestCase.testUnTLS failure on OS X #3210 - test_pb.NewCredTestCase is insanely long and causes some intermittent failures #3119 - trial leaks memory/objects/something #3052 - twistd logging options needs to be tested #3266 - Provide tools for managing new deprecation policy #3191 - in manhole in gnome-terminal_ lines longer than the terminal is wide result in excessive scrolling and duplication #3105 - symlink support and docstrings in filepath.FilePath.copyTo/moveTo #3267 - Typo in the Asynchronous Programming howto #3184 - Trial minimal reporter prints a negative run time in summary #1410 - spawnProcess on unix doesn't handle case where os.fork fails cleanly #2305 - twisted/internet/process.py leaks file descriptors when os.fork() raises #3272 - sorting the --profile output in trial #3259 - Improved some error messagge in reflect.py #2713 - trial -u can fail doctests on second pass #2698 - Build Bug on Cygwin #914 - xmlrpc.QueryFactory failed to catch some exceptions #1057 - Incorrect and misleading arg name msSinceEpoch #2017 - t.w.client.getPage returning ConnectionDone: Connection was closed cleanly error #1412 - ExpatError on xmlrpc client connection close #393 - [PATCH] HTTP Basic Auth based guard #1108 - web2 needs SOAP support #909 - microdom getElementsByTagName #687 - web.client needs documentation and/or data-checking for method argument #165 - [PATCH]Multiple header with same name support in http Of particular note, I think, are the following: #3213: Prior to this fix, the Twisted SFTP server would leak any file descriptors not closed before the end of an SFTP session. #3203: This fix restores Gtk2 support on Windows. We have also set up a Gtk2 Windows builder which is now passing all but one test. #1410: Prior to this fix, an error when calling os.fork() would cause reactor.spawnProcess to leak any pipes it opened and potentially to continue running in the main process with the wrong UID/GID. #165: This adds support for multiple values for each header to the twisted.web server. Previously only the last value received would be made available to applications. Six more tickets are waiting to be reviewed and will hopefully be closed soon. I want to take this opportunity to thank everyone who worked on tickets I reviewed over the past couple weeks. Twisted development is a collaborative process and I wouldn't have made nearly as big a dent without all of their help. I also want to thank Thomas Hervé in particular, who authored a lot of the code I reviewed and also (if I kept track properly) did every review of code I submitted over the the past two weeks. In addition to doing work on tickets from the tracker, I also addressed two other issues with Twisted's buildbot. First, I took one of the compact build result views and split it into a supported-only view and an everything view to make it easier to find out the status of recent builds on supported platforms. These two views are linked from the main Twisted buildbot page, http://buildbot.twistedmatrix.com/ - see the "Latest builds" link. Second, I improved the error reporting for the setuptools-enabled builder, making it easier to track down the cause of some setuptools-related problems. Thanks to the SFC (<http://conservancy.softwarefreedom.org/>) and all of the sponsors (<http://twistedmatrix.com/trac/wiki/TSF/FoundingSponsors>) who made this possible. Jean-Paul

3 2