Mailman 3 June 2010 - Twisted

[Twisted-Python] qt4 reactor status
by Steve Castellotti 30 Jun '10

30 Jun '10

Hello- I'm wondering if anyone can fill me in on the current status and plans for Qt support under Twisted? I've been working with PyGTK for most of my cross-platform applications over the past decade or so, but mainly for license reasons and to be honest GTK+ still looks pretty much the same visually as it did back in the early 2000's (especially on Windows). It feels time to move on to something a little less tired and a little better suited to mobile deployment (starting with the N900 at least). With Qt4 being LGPL'd and PySide offering the same as an alternative to PyQT's GPL-or-commercial-only choices, it seems like there shouldn't be any barriers to supporting it, save interest and time from developers of course. The last piece of information I could dig up was this trac ticket: http://twistedmatrix.com/trac/ticket/3977 It sounds like Dr. Glenn Tarbox is happy with a potential license change on the original qt4reactor code, but that still needs confirmation from Itamar Shtull-Trauring and Gabe Rudy (with no further comment in the last nine months). Whether the code itself is compatible with the current releases of Qt, PyQt, or PySide isn't clear (there is certainly no "stable" release suitable for serious re-deployment that I can find). The source itself is located here: https://launchpad.net/qt4reactor ...but that code is now two years old, with the only update from Alan Franzoni a little over a year ago doing some initial cleanup work "in order to support better testing." I've really only begun getting familiar with Qt development, but as of this moment it seems like I should focus on learning the QtNetwork component and start porting over my Twisted libraries to that, as at least support is present and maintained - plus the licensing is clear. In the meantime, can anyone please update me if there has been any further work or consideration towards official Qt support for Twisted? Cheers Steve Castellotti

4 9

Re: [Twisted-Python] [Twisted] #3977: Reintegrate qt4reactor into twisted
by Glenn Tarbox, PhD 29 Jun '10

29 Jun '10

On Tue, Jun 29, 2010 at 7:28 PM, Twisted <trac(a)twistedmatrix.com> wrote: > #3977: Reintegrate qt4reactor into twisted > > -----------------------------+---------------------------------------------- > Reporter: jknight | Owner: ghtdak > Type: enhancement | Status: new > Priority: normal | Milestone: > Component: core | Keywords: > Branch: | Branch_author: > Launchpad_bug: | > > -----------------------------+---------------------------------------------- > > Comment(by sc): > > Replying to [comment:2 ghtdak]: > > I'm cool with changing to whatever license makes sense. IIRC, the other > contributers would include Therve, Itmar Shtull-Trauring and Gabe Rudy > > > > Maintainer: U{Glenn H Tarbox, PhD<mailto:glenn@tarbox.org>} > > Previous maintainer: U{Itamar Shtull- > Trauring<mailto:twisted@itamarst.org>} > > Original port to QT4: U{Gabe Rudy<mailto:rudy@goldenhelix.com>} > > Following up on this mailing list thread as to the status of the > qt4reactor and its licensing: > > http://twistedmatrix.com/pipermail/twisted-python/2010-June/022457.html > > > It would seem PySide has reached the point where it should be a viable > alternative to PyQt, without the license complications. > > It is still necessary to attain permission to re-release the qt4reactor > code under Twisted's MIT license however. > Is there something specific I need do to the code to "make it so" or is this message intended for the previous maintainer(s)? -glenn > > -- > Ticket URL: <http://twistedmatrix.com/trac/ticket/3977#comment:4> > Twisted <http://twistedmatrix.com/trac/> > Engine of your Internet > -- Glenn H. Tarbox, PhD || 206-274-6919 || glenn(a)tarbox.org - xmpp || ghtdak - aim,jabber,IRC,yahoo

1 0

[Twisted-Python] runInteraction function returning a deferred
by Landreville 29 Jun '10

29 Jun '10

I am working with a third-party web-scraping application that also uses twisted (Scrapy) and would like for it to do its thing in a transaction along with a database action. This can be done with runInteraction, but Scrapy uses a non-blocking call when it does its thing. This actually doesn't even return a deferred, but you pass in the callback function you would like it to call when finished. Is there some way for runInteraction to wait for this deferred to be finished before ending the transaction? Or could something like this work, where the thread gets paused until it completes?: def interaction(txn): finished = {'test':False, 'failed': False} def _cb(r): finished['test'] = True def _err(failure): finished['failed'] = True Spider.crawl(_cb, _err) # do database stuff here while not finished['test'] and not finished['failed']: time.sleep(0.25) if finished['failed']: raise Exception('Web scraping failed.') d = dbpool.runInteraction(interaction) Thanks for a look, Landreville

2 2

[Twisted-Python] ANNOUNCING Tahoe, the Least-Authority File System, v1.7.0
by Zooko O'Whielacronx 29 Jun '10

29 Jun '10

Dear people of twisted-python: We just released Tahoe-LAFS v1.7. The major new feature is an SFTP server. This means that (with enough installing software and tinkering with your operating system configuration) you can have a normal-looking mount point backed by a Tahoe-LAFS grid. Regards, Zooko ANNOUNCING Tahoe, the Least-Authority File System, v1.7.0 The Tahoe-LAFS team is pleased to announce the immediate availability of version 1.7.0 of Tahoe-LAFS, an extremely reliable distributed storage system. Tahoe-LAFS is the first distributed storage system which offers "provider-independent security"—meaning that not even the operator of your storage server can read or alter your data without your consent. Here is the one-page explanation of its unique security and fault-tolerance properties: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html Tahoe-LAFS v1.7.0 is the successor to v1.6.1, which was released February 27, 2010 [1]. v1.7.0 is a major new release with new features and bugfixes. It adds a fully functional SFTP interface, support for non-ASCII character encodings, and a new upload algorithm which guarantees that each file is spread over multiple servers for fault-tolerance. See the NEWS file [2] for details. WHAT IS IT GOOD FOR? With Tahoe-LAFS, you distribute your filesystem across multiple servers, and even if some of the servers are compromised by by an attacker, the entire filesystem continues to work correctly, and continues to preserve your privacy and security. You can easily share specific files and directories with other people. In addition to the core storage system itself, volunteers have built other projects on top of Tahoe-LAFS and have integrated Tahoe-LAFS with existing systems. These include frontends for Windows, Macintosh, JavaScript, iPhone, and Android, and plugins for Hadoop, bzr, mercurial, duplicity, TiddlyWiki, and more. See the Related Projects page on the wiki [3]. We believe that strong cryptography, Free and Open Source Software, erasure coding, and principled engineering practices make Tahoe-LAFS safer than RAID, removable drive, tape, on-line backup or "cloud storage" systems. This software is developed under test-driven development, and there are no known bugs or security flaws which would compromise confidentiality or data integrity under recommended use. (For all currently known issues please see the known_issues.txt file [4].) COMPATIBILITY This release is fully compatible with the version 1 series of Tahoe-LAFS. Clients from this release can write files and directories in the format used by clients of all versions back to v1.0 (which was released March 25, 2008). Clients from this release can read files and directories produced by clients of all versions since v1.0. Servers from this release can serve clients of all versions back to v1.0 and clients from this release can use servers of all versions back to v1.0. This is the ninth release in the version 1 series. This series of Tahoe-LAFS will be actively supported and maintained for the forseeable future, and future versions of Tahoe-LAFS will retain the ability to read and write files compatible with Tahoe-LAFS v1. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [5] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1 or, at your option, any later version. (The Transitive Grace Period Public Licence has requirements similar to the GPL except that it allows you to wait for up to twelve months after you redistribute a derived work before releasing the source code of your derived work.) See the file "COPYING.TGPPL.html" [6] for the terms of the Transitive Grace Period Public Licence, version 1. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe-LAFS works on Linux, Mac OS X, Windows, Cygwin, Solaris, *BSD, and probably most other systems. Start with "docs/quickstart.html" [7]. HACKING AND COMMUNITY Please join us on the mailing list [8]. Patches are gratefully accepted -- the RoadMap page [9] shows the next improvements that we plan to make and CREDITS [10] lists the names of people who've contributed to the project. The Dev page [11] contains resources for hackers. SPONSORSHIP Tahoe-LAFS was originally developed by Allmydata, Inc., a provider of commercial backup services. After discontinuing funding of Tahoe-LAFS R&D in early 2009, they have continued to provide servers, bandwidth, small personal gifts as tokens of appreciation, and bug reports. Thank you to Allmydata, Inc. for their generous and public-spirited support. Google, Inc. is sponsoring Tahoe-LAFS development as part of the Google Summer of Code 2010. Google suggested that we should apply for the Summer of Code program, and when we did they generously awarded four sponsorships to students from around the world to hack on Tahoe-LAFS this summer. Thank you to Google, Inc. for their generous and public-spirited support. HACK TAHOE-LAFS! If you can find a security flaw in Tahoe-LAFS which is serious enough that feel compelled to warn our users and issue a fix, then we will award you with a customized t-shirts with your exploit printed on it and add you to the "Hack Tahoe-LAFS Hall Of Fame" [12]. ACKNOWLEDGEMENTS This is the fifth release of Tahoe-LAFS to be created solely as a labor of love by volunteers. Thank you very much to the team of "hackers in the public interest" who make Tahoe-LAFS possible. In this release we especially owe thanks to David-Sarah Hopwood, who has dedicated many late nights to the project and displayed superb software engineering skills. Zooko Wilcox-O'Hearn on behalf of the Tahoe-LAFS team June 18, 2010 Boulder, Colorado, USA [1] http://tahoe-lafs.org/trac/tahoe/browser/relnotes.txt?rev=4244 [2] http://tahoe-lafs.org/trac/tahoe/browser/NEWS?rev=4254 [3] http://tahoe-lafs.org/trac/tahoe/wiki/RelatedProjects [4] http://tahoe-lafs.org/trac/tahoe/browser/docs/known_issues.txt [5] http://tahoe-lafs.org/trac/tahoe/browser/COPYING.GPL [6] http://tahoe-lafs.org/source/tahoe/trunk/COPYING.TGPPL.html [7] http://tahoe-lafs.org/source/tahoe/trunk/docs/quickstart.html [8] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev [9] http://tahoe-lafs.org/trac/tahoe/roadmap [10] http://tahoe-lafs.org/trac/tahoe/browser/CREDITS?rev=4511 [11] http://tahoe-lafs.org/trac/tahoe/wiki/Dev [12] http://tahoe-lafs.org/hacktahoelafs/

5 10

[Twisted-Python] reactor.spawnProcess() and file descriptors
by Carlos Valiente 28 Jun '10

28 Jun '10

Hi! Is there any way of telling reactor.spawnProcess() not to close all open file descriptors? I've tried passing the following as ``childFDs``: childFDs = {0: "w", 1: "r", 2: "r"} max_fds = resource.getrlimit(resource.RLIMIT_NOFILE)[1] + 1 for i in xrange(3, max_fds): childFDs[i] = i but of course it doesn't work, because most of those file descriptors are not open, and then the call to clear the close-on-exec flag fails. In case there's no way of avoiding the close, I could try writing a patch for reactor.spawnProcess(): Something like adding an extra keyword argument to enable or disable the close, for instance. Would such a patch have any chance of being accepted? Thanks, Carlos

3 5

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 26 Jun '10

26 Jun '10

1 0

[Twisted-Python] Twisted 10.1.0pre1 released -- please test!
by Jonathan Lange 25 Jun '10

25 Jun '10

>From the dank and grey depths of a London summer, a thin, reedy voice wanders across the waves of the Internet, saying that it's jolly well pleased to announce the first 10.1.0 pre-release. Tarballs for Twisted 10.1.0pre1 are now available at: http://people.canonical.com/~jml/Twisted/10.1.0pre1/ Highlights include: * Deferreds now support cancellation * A new "endpoint" interface which can abstractly describe stream transport endpoints such as TCP and SSL * inotify support for Linux, which allows monitoring of file system events. * AMP supports transferring timestamps For more information, see the NEWS file. Please download the tarballs and test them as much as possible. Thanks, jml

6 10

[Twisted-Python] twistedmatrix.com renewal
by Reza Lotun 25 Jun '10

25 Jun '10

It appears that the twistedmatrix.com domain has expired! -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun(a)gmail.com work: reza(a)tweetdeck.com twitter: @rlotun

2 2

[Twisted-Python] in a factory: assert not self.usedUp
by Jean Daniel 23 Jun '10

23 Jun '10

Hello, I have three questions on a code examples, and actually I have seen this pattern in several places in Twisted. The code are copied from twisted/doc/mail/examples/imap4client.py 1. In the code below, what is the purpose of an assert checking that the member attribute promptDeferred is empty in the prompt method? Is this some kind of documentation saying "the attribute is ALWAYS empty when this function is called"? 2. It seems that there is the protocol's deferred and the protocol are unlinked before the deferred is fired. What would happen if the defered was fired while it is still held in an attribute of the protocol? class TrivialPrompter(basic.LineReceiver): from os import linesep as delimiter promptDeferred = None def prompt(self, msg): assert self.promptDeferred is None # 1. self.display(msg) self.promptDeferred = defer.Deferred() return self.promptDeferred def display(self,msg): self.transport.write(msg) def lineReceived(self, line): if self.promptDeferred is None: return d, self.promptDeferred = self.promptDeferred, None # 2. d.callback(line) 3. Third question, in the factory below, there is an attribute usedUp set at False by default and set to True late in the buildprotocol method. There is an assert making sure the factory is only used once, checking that usedUp is False at the beginning of buildprotocol(). Why it is so important that the factory must be used only once? What would happens otherwise? class SimpleIMAP4ClientFactory(protocol.ClientFactory): usedUp = False protocol = SimpleIMAP4Client def __init__(self, username, onConn): self.ctx = ssl.ClientContextFactory() self.username = username self.onConn = onConn def buildProtocol(self, addr): assert not self.usedUp # 3. self.usedUp = True p = self.protocol(self.ctx) p.factory = self p.greetDeferred = self.onConn auth = imap4.CramMD5ClientAuthenticator(self.username) p.registerAuthenticator(auth) return p Thanks for your help,

2 1

[Twisted-Python] Review process, news fragments
by exarkun＠twistedmatrix.com 21 Jun '10

21 Jun '10

Hello Twisted developers, Please take a few minutes to refresh your memory of the contents of <http://twistedmatrix.com/trac/wiki/ReviewProcess>. In particular, I'd like to draw everyone's attention to the requirements for news fragments. Since we introduced this part of the workflow, the review requirements for these has been a little unclear. Many reviewers haven't required that the news fragment be reviewed (that is, if it was missing, they would say "Please add a news fragment and then merge.") or haven't kept the guidelines these files in mind when reviewing them. I'd like for this to change. Informative, consistently written news fragments result better information being available to users when we do a release. This is a very simple part the development process but effort here has a disproportionately large effect on the perception of Twisted. If the guidelines for news fragment content on the review process wiki page are unclear or insufficient, please complain and we can work on improving the weaknesses. Thanks, and congratulations to everyone on the (now inevitable :) 10.1 release. Jean-Paul

5 5