Mailman 3 March 2008 - Twisted

[Twisted-Python] announcing allmydata.org "Tahoe" v0.9
by zooko March 17, 2008

March 17, 2008

ANNOUNCING: Allmydata.org "Tahoe" version 0.9 We are pleased to announce the release of version 0.9 of allmydata.org "Tahoe". Allmydata.org "Tahoe" is a secure, decentralized, fault-tolerant filesystem. All of the source code is available under a Free Software, Open Source licence (or two). This filesystem is encrypted and distributed over multiple peers in such a way that it continues to work correctlly even when some of the peers are unavailable, malfunctioning, or malicious. A one-page explanation of the novel properties of this filesystem is visible at: http://allmydata.org/source/tahoe/trunk/docs/about.html This is the successor to Allmydata.org "Tahoe" v0.8, which was released February 15, 2008 [1]. This release is backwards-compatible with v0.8. This is a major release of allmydata.org "Tahoe" -- the first release which is considered stable and functional enough to serve as a permanent store of valuable data. New versions of Tahoe are expected to maintain backwards compatibility with this version for the forseeable future. This release of allmydata.org "Tahoe" will form the basis of the next consumer backup product from Allmydata, Inc. -- http://allmydata.com . This release adds extensive "visibility into the grid" -- the web user interface now shows detailed information about the nodes comprising the grid and fine-grained statistics about the time and space used for the storage and retrieval operations. Since v0.8 we've made the following changes: * Added user-friendly Windows application and package, thanks especially to Rob Kinninmont and Mike Booker (tickets #195, 242, 243, 321, 335). * Display information about uploads and downloads (ticket #39). * Improve performance (including tickets #304, 321). * Improve packaging and testing (including ticket #145). * Add even more extensive automated tests (including ticket #207). * Updated documentation. WHAT IS IT GOOD FOR? With Tahoe, you can distribute your filesystem across a set of computers, such that if some of the computers fail or turn out to be malicious, the filesystem continues to work from the remaining computers. You can also share your files with other users, using a strongly encrypted, capability-based access control scheme. This release is targeted at hackers and smart users who are willing to use a web user interface, a command-line user interface, or a FUSE interface. (Or a RESTful API. Just telnet to localhost and type HTTP requests to get started.) Because this software is the product of less than a year and a half of active development, we do not categorically recommend it for the storage of data which is extremely confidential or precious. However, we believe that the combination of erasure coding and careful engineering makes the use of this software a much safer alternative than common alternatives, such as RAID or traditional backup onto a remote hard drive, removable drive, or tape. This software comes with extensive unit tests [2], and there are no known security flaws which would compromise confidentiality or data integrity. (For all currently known security issues please see the Security web page: [3].) This release of Tahoe is suitable for the "friendnet" use case [4] -- it is easy to create a filesystem spread over the computers of you and your friends so that you can share files and disk space with one another. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1.0. The Transitive Grace Period Public Licence says that you may distribute proprietary derived works of Tahoe without releasing the source code of that derived work for up to twelve months, after which time you are obligated to release the source code of the derived work under the Transitive Grace Period Public Licence. See the file "COPYING.TGPPL.html" for the terms of the Transitive Grace Period Public Licence, version 1.0. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe works on Linux, Mac OS X, Windows, Cygwin, and Solaris. For installation instructions please see "docs/install.html" [5]. HACKING AND COMMUNITY Please join us on the mailing list [6] to discuss uses of Tahoe. Patches that extend and improve Tahoe are gratefully accepted -- the RoadMap page [7] shows the next improvements that we plan to make and CREDITS [8] lists the names of people who've contributed to the project. The wiki Dev page [9] contains resources for hackers. SPONSORSHIP Tahoe is sponsored by Allmydata, Inc. [10], a provider of consumer backup services. Allmydata, Inc. contributes hardware, software, ideas, bug reports, suggestions, demands, and money (employing several allmydata.org Tahoe hackers and allowing them to spend part of their work time on the next-generation, free-software project). We are eternally grateful! Zooko O'Whielacronx on behalf of the allmydata.org team March 13, 2008 Boulder, Colorado, USA [1] http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=2183 [2] http://allmydata.org/tahoe-figleaf/current/ [3] http://allmydata.org/trac/tahoe/wiki/Security [4] http://allmydata.org/trac/tahoe/wiki/UseCases [5] http://allmydata.org/source/tahoe/trunk/docs/install.html [6] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev [7] http://allmydata.org/trac/tahoe/roadmap [8] http://allmydata.org/trac/tahoe/browser/CREDITS?rev=2307 [9] http://allmydata.org/trac/tahoe/wiki/Dev [10] http://allmydata.com P.S. Someday, people might search for some of the earliest occurrences of the name "LAUGFS", which stands for "Least AUthority Grid FileSystem". Hello, searchers from the future!

3 3

[Twisted-Python] Bug in options howto?
by Eric Smith March 16, 2008

March 16, 2008

I might be mistaken, but I think there's a bug in: http://twistedmatrix.com/projects/core/documentation/howto/options.html In the section "Inheritance, Or: How I Learned to Stop Worrying and Love the Superclass", it shows two classes, one which inherits from the other. It then shows the same thing, but as a single class. In the single class example, I think the base class should be usage.Options, not BaseOptions. I'll open a ticket if this is a bug, otherwise sorry for wasting your time.

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com March 16, 2008

March 16, 2008

1 0

[Twisted-Python] twisted names - how to add answers to a message and send replies
by Chris Dew March 14, 2008

March 14, 2008

I have a small script which answers dns queries by returning the original message. I have spent a couple of hours RTFMing, but I still cannot find out how to correctly add answers to the message, before returning it. I want to add an answer, either a Record_CNAME or Record_A, so as to produce a trivial authoritative-only DNS server. Adding a Record_XXX object to message.answers using the Array's 'append' method produces a MalformedPacket (according to wireshark), though the server does not throw any exceptions. (In this situation, the answer count, as shown by dig, does rise to 1, even though no answer is displayed [due to malformed packet?].) Perhaps there is a completely different way to do this? Could I craft a message with answers from scratch? Any help appreciated, Chris. #!/usr/bin/python from twisted.internet.protocol import Protocol, Factory from twisted.internet import reactor from twisted.names.server import DNSServerFactory from twisted.names.dns import Query, DNSDatagramProtocol, RRHeader, Record_CNAME, Record_A, Message from twisted.names import dns from twisted.internet.interfaces import IAddress class Controller(object): def messageReceived(self, m, protocol, addr): print m, dir(m), m.queries, m.answers # tried all sorts of things here protocol.transport.write(m.toStr(), addr) controller = Controller() if __name__ == "__main__": reactor.listenUDP(53, DNSDatagramProtocol(controller)) reactor.run()

2 4

[Twisted-Python] Help with BananaError
by Iago Toral March 14, 2008

March 14, 2008

Hi all, I'm working on the gnome build brigade (http://build.gnome.org/) using Buildbot, which is a continuous integration tool based on Twisted. I have several Twisted services (buildbot masters) running behind a firewall (say ports 8001, 8002, mand 8003) and only one opened port for external connections (say 9000). I'm writting a small Twisted tool to allow me to use these services which port is not open directly for external connections through that opened port to other external Twisted clients (buildbot slaves): +----------------+ |server-1:8001<->| |server-2:8002<->|<->:9000<--->Internet<-->clients |server-3:8003<->| +----------------+ This tool has two parts, the client side (Mux) wraps the application protocol by adding a header that identifies the client and also the server it wants to connect. The server side (Demux) listens on the opened port (9000) for incomming messages, extracts the client header with information identifying the client and uses that information to properly redirect the wrapped message to the appropriate internal port. This tool works mostly ok, except that after some minutes working properly redirecting several dozens of messages correctly, the Demux breaks (on reception of a normal incoming message) with the following error message: exception: File "/home/master/devel/lib/python/twisted/spread/banana.py", line 176, in dataReceived raise BananaError("Security precaution: List too long.") The messages do not exceed the maximum size defined in banana.py, actually these messages are rather small, usually less than 1KB. Does anyone know what that error message means and when it happens? What might an application be doing wrong for that to happen? Attached is the code of this tool, both the Mux and the Demux, it is a very small file (~100 lines of code). Also, take into account that this was written as a protoype and it is my first TWisted program so it might be very badly designed. Thanks in advance for any feedback you can give!! Iago

2 1

Re: [Twisted-Python] Re: question about deferred lists tutorial description
by Terry Jones March 12, 2008

March 12, 2008

Hi Josef > Apologies for the previous premature email; it seems I cannot read. :-) > The apparent contradiction I mentioned was the result of my conflating > 'callback' with 'addCallback'. Right. > However, I guess I'd still appreciate some discussion on the nuances of > these two functions. It's just a trivial non-real-world example. The use of callback is to supply a return value, which is passed to the function added with addCallback (and similarly with errback). In a non-trivial example, some other code (associated with the function that returned the deferred) would be what called callback (or errback). The example just shows all the code in one place. Does that make sense? Terry

1 0

[Twisted-Python] question about deferred lists tutorial description
by Josef Novak March 12, 2008

March 12, 2008

Hi, I have been using twisted intermittently for a couple of months, but still don't feel like I have a good intuitive understanding of how to get things done quickly, and efficiently with the framework. Thus I've been revisiting the tutorial documents and resolved to go through them with a fine-toothed comb in an effort to really get my noggin around things. In the deffered discussion: http://twistedmatrix.com/projects/core/documentation/howto/defer.html there is a section about defer.DeferredList, which contains code examples and a subsequent note which -seems- to contradict the code above it: ------------begin code from tutorial---------- The callback will be called with a list of the results of the Deferreds it contains, like so: def printResult(result): print result deferred1 = defer.Deferred() deferred2 = defer.Deferred() deferred3 = defer.Deferred() dl = defer.DeferredList([deferred1, deferred2, deferred3]) dl.addCallback(printResult) deferred1.callback('one') deferred2.errback('bang!') deferred3.callback('three') # At this point, dl will fire its callback, printing: # [(1, 'one'), (0, 'bang!'), (1, 'three')] # (note that defer.SUCCESS == 1, and defer.FAILURE == 0) ------------end code from tutorial---------- Ok, so far, so good. You can build a series of deferreds, stick them in a DeferredList, add a callback to the list, *subsequently* add some callbacks/errbacks to the individual deferreds in the DeferredList, fire things up, and have the results of the *subsequently added* callbacks/errbacks sent to the DeferredList callback and printed out. However, the following Note regarding callbacks for DeferredList objects seems to contradict the promised results of the above codeblock, stating that: """If you want to apply callbacks to the individual Deferreds that go into the DeferredList, you should be careful about when those callbacks are added. The act of adding a Deferred to a DeferredList inserts a callback into that Deferred (when that callback is run, it checks to see if the DeferredList has been completed yet). The important thing to remember is that it is this callback which records the value that goes into the result list handed to the DeferredList's callback. Therefore, if you add a callback to the Deferred after adding the Deferred to the DeferredList, the value returned by that callback will not be given to the DeferredList's callback. To avoid confusion, we recommend not adding callbacks to a Deferred once it has been used in a DeferredList.""" Ok, so in fact, any callbacks/errbacks I need to add to deferreds which will be added to a DeferredList object, need to be added Prior to my adding the individual deferreds to the DeferredList object. Or so it would appear to my limited understanding. I would greatly appreciate it if someone could resolve this apparent contradiction for me, or if it is a mistake in the documentation, please point out which one of the code block or the note is actually correct. Cheers!

1 1

[Twisted-Python] mutex in twisted?
by Gabriel Rossetti March 11, 2008

March 11, 2008

Hello, Is there some sort of mutex/semaphore code that can be used in twisted? I need to run an external subprocess from twisted when I get some data from a client and I'd like to make sure that while one client's data spawned the process to run, some other client will not do the same at the same time, thus my need for a mutex. Maybe there is another way to do this in a more twisted sort of way (pun not intended)? If so I'd love to hear how! Thanks, Gabriel

2 3

[Twisted-Python] parsing url query parameters and values
by Vasil Vangelovski March 11, 2008

March 11, 2008

Is there any method in the Twisted or Twisted web api I can use to parse url query parameters and values. This isn't a very naive algorithm since there are more complicated query strings that can be constructed, for example: http://www.example.com/path?param1=val1&param2=val2&&param3=val3 http://www.example.com/path?param1=val1&param2=val2&som=|=val&param3=val3 Ideally I could use something that bulds a dictionary of parameters and values

3 3

[Twisted-Python] xml library in twisted
by Raj kumar March 11, 2008

March 11, 2008

Hi all, I'm dealing with xml files in a twisted application, can any one suggest me the best xml library in twisted? Previously i used twisted.web.microdom and twisted.web.minidom but they don't have much of the functionalities.... Is there any other xml library in twisted which makes it easy with lot of options? I wanted to create a xml file like this... <game> <player1> <stuff> </stuff> </player1> <player2> <stuff> </stuff> </player2> </game> and want to access that player stuff directly..(like a tree structure) Thank you in advance. Now you can chat without downloading messenger. Go to http://in.messenger.yahoo.com/webmessengerpromo.php

3 2