Mailman 3 September 2010 - Twisted

[Twisted-Python] Handling too many open file descriptors
by Landreville 27 Sep '10

27 Sep '10

I'm running an application that makes about 1300 snmp connections every minute; I'm using utils.getProcessOutput with snmpget because pysnmp throws an error when I try to run it. Now of course I get the Too many open files error, but is the best way to handle this increasing the limit on Linux or by implementing some sort of queue so that there are only x number of the snmpget processes at a time? Is there such a queue feature in twisted? Thanks, Jason

3 3

[Twisted-Python] Announcing Twisted 10.1.0pre2!
by Thomas Hervé 27 Sep '10

27 Sep '10

It's this time of the year again! From the dangerous and intrepid land of Montréal, Canada, I have the pleasure to announce the first 10.2.0 pre-release. Tarballs for Twisted 10.2.0pre1 are now available at: http://people.canonical.com/~therve/Twisted/10.2.0pre1/ Highlights include: * A new implementation of the Deferred callbacks firing, which now handles chains iteratively instead of recursively, and should prevent recursion errors. * Deferred.setTimeout has finally been removed. And the usual load of bug fixes. For more information, see the NEWS file. Please download the tarballs and test them as much as possible. I intend to do a final release within the next 2 weeks, if I didn't die by ingestion of poutine. Thanks, Thomas

4 7

[Twisted-Python] how to abort a Protocol connection
by Weikai Xie 27 Sep '10

27 Sep '10

Folks, I have a usage case where the server need to abort a client TCP connection (implemented by Protocol) immediately. However, the normal Protocol.transport.loseConnection() will try to send pending data to the client first before close the connection. Currently I used an ugly hack to work around it, where I set Protocol.transport._writeDisconnected=True before call Protocol.transport.loseConnection() , which can fool the loseConnection to give up writing pending data. But I think there should be some more elegant way to do it. Any idea? Regards. Weikai Xie

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 26 Sep '10

26 Sep '10

1 0

[Twisted-Python] How to exit a twistd program with a status code ?
by David Cournapeau 24 Sep '10

24 Sep '10

Hi, I have a program run through twistd, and would like to exit with a non zero status code on some error. I have a callback which stops the reactor and then sys.exit(some_value), but twistd still seems to catch the SystemExit exception after calling for reactor.stop(): Unhandled error in Deferred: Traceback (most recent call last): File "/home/david/local/lib/python2.6/site-packages/twisted/internet/base.py", line 1031, in connectionFailed self.factory.clientConnectionFailed(self, reason) File "/home/david/local/lib/python2.6/site-packages/twisted/web/client.py", line 350, in clientConnectionFailed self.deferred.errback(reason) File "/home/david/local/lib/python2.6/site-packages/twisted/internet/defer.py", line 307, in errback self._startRunCallbacks(fail) File "/home/david/local/lib/python2.6/site-packages/twisted/internet/defer.py", line 354, in _startRunCallbacks self._runCallbacks() --- <exception caught here> --- File "/home/david/local/lib/python2.6/site-packages/twisted/internet/defer.py", line 371, in _runCallbacks self.result = callback(self.result, *args, **kw) File "scripts/tests/run_tests.py", line 69, in exit_on_error sys.exit(1) exceptions.SystemExit: 1 What's the correct way of doing this ? cheers, David

5 4

[Twisted-Python] ANNOUNCING Tahoe, the Least-Authority File System, v1.8.0
by Zooko O'Whielacronx 24 Sep '10

24 Sep '10

ANNOUNCING Tahoe, the Least-Authority File System, v1.8.0 The Tahoe-LAFS team is pleased to announce the immediate availability of version 1.8.0 of Tahoe-LAFS, an extremely reliable distributed storage system. Get it here: http://tahoe-lafs.org/source/tahoe/trunk/docs/quickstart.html Tahoe-LAFS is the first distributed storage system to offer "provider-independent security" — meaning that not even the operators of your storage servers can read or alter your data without your consent. Here is the one-page explanation of its unique security and fault-tolerance properties: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html The previous stable release of Tahoe-LAFS was v1.7.1, which was released July 18, 2010 [1]. v1.8.0 offers greatly improved performance and fault-tolerance of downloads and improved Windows support. See the NEWS file [2] for details. WHAT IS IT GOOD FOR? With Tahoe-LAFS, you distribute your filesystem across multiple servers, and even if some of the servers fail or are taken over by an attacker, the entire filesystem continues to work correctly, and continues to preserve your privacy and security. You can easily share specific files and directories with other people. In addition to the core storage system itself, volunteers have built other projects on top of Tahoe-LAFS and have integrated Tahoe-LAFS with existing systems, including Windows, JavaScript, iPhone, Android, Hadoop, Flume, Django, Puppet, bzr, mercurial, perforce, duplicity, TiddlyWiki, and more. See the Related Projects page on the wiki [3]. We believe that strong cryptography, Free and Open Source Software, erasure coding, and principled engineering practices make Tahoe-LAFS safer than RAID, removable drive, tape, on-line backup or cloud storage. This software is developed under test-driven development, and there are no known bugs or security flaws which would compromise confidentiality or data integrity under recommended use. (For all important issues that we are currently aware of please see the known_issues.txt file [4].) COMPATIBILITY This release is compatible with the version 1 series of Tahoe-LAFS. Clients from this release can write files and directories in the format used by clients of all versions back to v1.0 (which was released March 25, 2008). Clients from this release can read files and directories produced by clients of all versions since v1.0. Servers from this release can serve clients of all versions back to v1.0 and clients from this release can use servers of all versions back to v1.0. This is the eleventh release in the version 1 series. This series of Tahoe-LAFS will be actively supported and maintained for the forseeable future, and future versions of Tahoe-LAFS will retain the ability to read and write files compatible with this series. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [5] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1 or, at your option, any later version. (The Transitive Grace Period Public Licence has requirements similar to the GPL except that it allows you to delay for up to twelve months after you redistribute a derived work before releasing the source code of your derived work.) See the file "COPYING.TGPPL.html" [6] for the terms of the Transitive Grace Period Public Licence, version 1. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe-LAFS works on Linux, Mac OS X, Windows, Cygwin, Solaris, *BSD, and probably most other systems. Start with "docs/quickstart.html" [7]. HACKING AND COMMUNITY Please join us on the mailing list [8]. Patches are gratefully accepted -- the RoadMap page [9] shows the next improvements that we plan to make and CREDITS [10] lists the names of people who've contributed to the project. The Dev page [11] contains resources for hackers. SPONSORSHIP Tahoe-LAFS was originally developed by Allmydata, Inc., a provider of commercial backup services. After discontinuing funding of Tahoe-LAFS R&D in early 2009, they continued to provide servers, bandwidth, small personal gifts as tokens of appreciation, and bug reports. Google, Inc. sponsored Tahoe-LAFS development as part of the Google Summer of Code 2010. They awarded four sponsorships to students from around the world to hack on Tahoe-LAFS that summer. Thank you to Allmydata and Google for their generous and public-spirited support. HACK TAHOE-LAFS! If you can find a security flaw in Tahoe-LAFS which is serious enough that feel compelled to warn our users and issue a fix, then we will award you with a customized t-shirts with your exploit printed on it and add you to the "Hack Tahoe-LAFS Hall Of Fame" [12]. ACKNOWLEDGEMENTS This is the fifth release of Tahoe-LAFS to be created solely as a labor of love by volunteers. Thank you very much to the team of "hackers in the public interest" who make Tahoe-LAFS possible. David-Sarah Hopwood and Zooko Wilcox-O'Hearn on behalf of the Tahoe-LAFS team September 23, 2010 Rainhill, Merseyside, UK and Boulder, Colorado, USA [1] http://tahoe-lafs.org/trac/tahoe/browser/relnotes.txt?rev=4579 [2] http://tahoe-lafs.org/trac/tahoe/browser/NEWS?rev=4732 [3] http://tahoe-lafs.org/trac/tahoe/wiki/RelatedProjects [4] http://tahoe-lafs.org/trac/tahoe/browser/docs/known_issues.txt [5] http://tahoe-lafs.org/trac/tahoe/browser/COPYING.GPL [6] http://tahoe-lafs.org/source/tahoe/trunk/COPYING.TGPPL.html [7] http://tahoe-lafs.org/source/tahoe/trunk/docs/quickstart.html [8] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev [9] http://tahoe-lafs.org/trac/tahoe/roadmap [10] http://tahoe-lafs.org/trac/tahoe/browser/CREDITS?rev=4591 [11] http://tahoe-lafs.org/trac/tahoe/wiki/Dev [12] http://tahoe-lafs.org/hacktahoelafs/

1 0

[Twisted-Python] Graceful/Hitless restart
by ruslan usifov 23 Sep '10

23 Sep '10

Hello! Sorry for my bad English! Can anybody provide some samples how to implement Graceful restart in twisted? Thanks

2 4

[Twisted-Python] State of snmp
by Landreville 22 Sep '10

22 Sep '10

Hello, What is the current recommended (or most popular) way of using SNMP with Twisted? I have tried TwistedSNMP, but it seems to only work with an old version of pysnmp that I can't find -- the TwistedSNMP website says pysnmp version 3.x, but pysnmp's website says the latest version is 2.x. I saw that there is a package in Ubuntu called python-pynetsnmp which contains bindings for netsnmp and somehow integrates it into twisted, but I cannot find this outside of Ubuntu's packages (I don't use Ubuntu) . The pynetsnmp website on sourceforge has no files to download, so I don't think that is source for the package. Thanks, Landreville

2 2

[Twisted-Python] High traffic connection issue
by Ted Pederson 22 Sep '10

22 Sep '10

Trying to do some scalability testing on my first twisted server. When I have hundreds of connections and moments of very high server activity I am seeing that clients are unable to connect. Here are some numbers: 250 open TCP connections. I see 857 operations in my log during the second where a new client tries to connect. In the client log I see that connectionMade() is called. I am pretty sure I don't see a corresponding connectionMade() on the server side (could this be right?). Then, about a second later I get a connectionLost() on the client with 'Connection to the other side was lost in a non-clean fashion'. Is this to be expected? Should the client just be expected to retry? Thanks

2 1

[Twisted-Python] file descriptor cannot be a negative integer (-1)
by ruslan usifov 22 Sep '10

22 Sep '10

Hello! I have use kqreactor from this branch http://twistedmatrix.com/trac/browser/branches/kqreactor-1918-3/. In the follow code we connect to unix socket. Sometimes we got the error: "file descriptor cannot be a negative integer (-1)" what does this mean? I haven't any ideas l_client = protocol.ClientCreator(reactor, Proxy); l_client.connectUNIX(self.factory.proxysock).addCallback(connectionMade).addErrback(cantConnect); PS: my uname -a is: FreeBSD 8.0-RELEASE-p2 FreeBSD 8.0-RELEASE-p2 #2: Wed Jan 13 17:50:25 UTC 2010 root@DTG1383:/usr/obj/usr/src/sys/Z-AMD64 amd64

3 6