[Twisted-Python] ANNOUNCING Tahoe, the Least-Authority File System, v1.7.0
Dear people of twisted-python: We just released Tahoe-LAFS v1.7. The major new feature is an SFTP server. This means that (with enough installing software and tinkering with your operating system configuration) you can have a normal-looking mount point backed by a Tahoe-LAFS grid. Regards, Zooko ANNOUNCING Tahoe, the Least-Authority File System, v1.7.0 The Tahoe-LAFS team is pleased to announce the immediate availability of version 1.7.0 of Tahoe-LAFS, an extremely reliable distributed storage system. Tahoe-LAFS is the first distributed storage system which offers "provider-independent security"—meaning that not even the operator of your storage server can read or alter your data without your consent. Here is the one-page explanation of its unique security and fault-tolerance properties: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html Tahoe-LAFS v1.7.0 is the successor to v1.6.1, which was released February 27, 2010 [1]. v1.7.0 is a major new release with new features and bugfixes. It adds a fully functional SFTP interface, support for non-ASCII character encodings, and a new upload algorithm which guarantees that each file is spread over multiple servers for fault-tolerance. See the NEWS file [2] for details. WHAT IS IT GOOD FOR? With Tahoe-LAFS, you distribute your filesystem across multiple servers, and even if some of the servers are compromised by by an attacker, the entire filesystem continues to work correctly, and continues to preserve your privacy and security. You can easily share specific files and directories with other people. In addition to the core storage system itself, volunteers have built other projects on top of Tahoe-LAFS and have integrated Tahoe-LAFS with existing systems. These include frontends for Windows, Macintosh, JavaScript, iPhone, and Android, and plugins for Hadoop, bzr, mercurial, duplicity, TiddlyWiki, and more. See the Related Projects page on the wiki [3]. We believe that strong cryptography, Free and Open Source Software, erasure coding, and principled engineering practices make Tahoe-LAFS safer than RAID, removable drive, tape, on-line backup or "cloud storage" systems. This software is developed under test-driven development, and there are no known bugs or security flaws which would compromise confidentiality or data integrity under recommended use. (For all currently known issues please see the known_issues.txt file [4].) COMPATIBILITY This release is fully compatible with the version 1 series of Tahoe-LAFS. Clients from this release can write files and directories in the format used by clients of all versions back to v1.0 (which was released March 25, 2008). Clients from this release can read files and directories produced by clients of all versions since v1.0. Servers from this release can serve clients of all versions back to v1.0 and clients from this release can use servers of all versions back to v1.0. This is the ninth release in the version 1 series. This series of Tahoe-LAFS will be actively supported and maintained for the forseeable future, and future versions of Tahoe-LAFS will retain the ability to read and write files compatible with Tahoe-LAFS v1. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [5] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1 or, at your option, any later version. (The Transitive Grace Period Public Licence has requirements similar to the GPL except that it allows you to wait for up to twelve months after you redistribute a derived work before releasing the source code of your derived work.) See the file "COPYING.TGPPL.html" [6] for the terms of the Transitive Grace Period Public Licence, version 1. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe-LAFS works on Linux, Mac OS X, Windows, Cygwin, Solaris, *BSD, and probably most other systems. Start with "docs/quickstart.html" [7]. HACKING AND COMMUNITY Please join us on the mailing list [8]. Patches are gratefully accepted -- the RoadMap page [9] shows the next improvements that we plan to make and CREDITS [10] lists the names of people who've contributed to the project. The Dev page [11] contains resources for hackers. SPONSORSHIP Tahoe-LAFS was originally developed by Allmydata, Inc., a provider of commercial backup services. After discontinuing funding of Tahoe-LAFS R&D in early 2009, they have continued to provide servers, bandwidth, small personal gifts as tokens of appreciation, and bug reports. Thank you to Allmydata, Inc. for their generous and public-spirited support. Google, Inc. is sponsoring Tahoe-LAFS development as part of the Google Summer of Code 2010. Google suggested that we should apply for the Summer of Code program, and when we did they generously awarded four sponsorships to students from around the world to hack on Tahoe-LAFS this summer. Thank you to Google, Inc. for their generous and public-spirited support. HACK TAHOE-LAFS! If you can find a security flaw in Tahoe-LAFS which is serious enough that feel compelled to warn our users and issue a fix, then we will award you with a customized t-shirts with your exploit printed on it and add you to the "Hack Tahoe-LAFS Hall Of Fame" [12]. ACKNOWLEDGEMENTS This is the fifth release of Tahoe-LAFS to be created solely as a labor of love by volunteers. Thank you very much to the team of "hackers in the public interest" who make Tahoe-LAFS possible. In this release we especially owe thanks to David-Sarah Hopwood, who has dedicated many late nights to the project and displayed superb software engineering skills. Zooko Wilcox-O'Hearn on behalf of the Tahoe-LAFS team June 18, 2010 Boulder, Colorado, USA [1] http://tahoe-lafs.org/trac/tahoe/browser/relnotes.txt?rev=4244 [2] http://tahoe-lafs.org/trac/tahoe/browser/NEWS?rev=4254 [3] http://tahoe-lafs.org/trac/tahoe/wiki/RelatedProjects [4] http://tahoe-lafs.org/trac/tahoe/browser/docs/known_issues.txt [5] http://tahoe-lafs.org/trac/tahoe/browser/COPYING.GPL [6] http://tahoe-lafs.org/source/tahoe/trunk/COPYING.TGPPL.html [7] http://tahoe-lafs.org/source/tahoe/trunk/docs/quickstart.html [8] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev [9] http://tahoe-lafs.org/trac/tahoe/roadmap [10] http://tahoe-lafs.org/trac/tahoe/browser/CREDITS?rev=4511 [11] http://tahoe-lafs.org/trac/tahoe/wiki/Dev [12] http://tahoe-lafs.org/hacktahoelafs/
On Mon, 21 Jun 2010 10:45 -0700, "Zooko O'Whielacronx" <zooko@zooko.com> wrote:
Dear people of twisted-python:
We just released Tahoe-LAFS v1.7. The major new feature is an SFTP server. This means that (with enough installing software and tinkering with your operating system configuration) you can have a normal-looking mount point backed by a Tahoe-LAFS grid.
Splendid! I downloaded it and typed "python setup.py build", and it went off and started building a copy of Twisted 10.0! Will that be installed on my system, or will it only affect the Tahoe build? I can't afford to move to 10.0 until I've had time to track down a certain bug that appeared between 2.5.0 and version 8. Peter.
On 6/22/10 10:35 AM, Peter Westlake wrote:
I downloaded it and typed "python setup.py build", and it went off and started building a copy of Twisted 10.0! Will that be installed on my system, or will it only affect the Tahoe build?
Using "setup.py build" should not affect anything outside of your source directory: it should create a Twisted .egg and store/use it in the support/lib subdir of your Tahoe source checkout. Tahoe should be the only thing that sees the new Twisted. In contrast, I suspect that "setup.py install" *would* install all the dependencies (as well as Tahoe itself) onto your system. In general, just don't run anything as root, and you should be ok. cheers, -Brian
On Fri, Jun 25, 2010 at 12:04 AM, Brian Warner <warner@lothar.com> wrote:
On 6/22/10 10:35 AM, Peter Westlake wrote:
I downloaded it and typed "python setup.py build", and it went off and started building a copy of Twisted 10.0! Will that be installed on my system, or will it only affect the Tahoe build?
Using "setup.py build" should not affect anything outside of your source directory: it should create a Twisted .egg and store/use it in the support/lib subdir of your Tahoe source checkout. Tahoe should be the only thing that sees the new Twisted.
In contrast, I suspect that "setup.py install" *would* install all the dependencies (as well as Tahoe itself) onto your system.
In general, just don't run anything as root, and you should be ok.
Well there's also virtualenv — won't it help?
On Tue, Jun 22, 2010 at 10:35 AM, Peter Westlake <peter.westlake@pobox.com> wrote:
I downloaded it and typed "python setup.py build", and it went off and started building a copy of Twisted 10.0!
That's funny. Could you please run: python -c "import pkg_resources;print pkg_resources.require('Twisted')"
Will that be installed on my system, or will it only affect the Tahoe build?
(Brian Warner answered this.)
I can't afford to move to 10.0 until I've had time to track down a certain bug that appeared between 2.5.0 and version 8.
Do you have a ticket # for that bug? Regards, Zooko
[resending after the glitch with the domain] On Thu, 24 Jun 2010 18:34 -0700, "Zooko O'Whielacronx" <zooko@zooko.com> wrote:
On Tue, Jun 22, 2010 at 10:35 AM, Peter Westlake <peter.westlake@pobox.com> wrote:
I downloaded it and typed "python setup.py build", and it went off and started building a copy of Twisted 10.0!
That's funny. Could you please run:
python -c "import pkg_resources;print pkg_resources.require('Twisted')"
It says: allmydata-tahoe-1.7.0$ python -c "import pkg_resources;print pkg_resources.require('Twisted')" [Twisted 10.0.0 (/localhome/packages/allmydata-tahoe-1.7.0/Twisted-10.0.0-py2.5-linux-i686.egg), zope.interface 3.4.0 (/usr/lib/python2.5/site-packages)]
Will that be installed on my system, or will it only affect the Tahoe build?
(Brian Warner answered this.)
Yes - thanks to Brian.
I can't afford to move to 10.0 until I've had time to track down a certain bug that appeared between 2.5.0 and version 8.
Do you have a ticket # for that bug?
No, because I can't provide a simple test case for it. My only hope is to get the source of Twisted and bisect, then see what changed in that revision. I fully intend to do that as soon as I can spare the time. In fact, it's becoming more urgent since 10 fixes some other problems. Really I should be doing that instead of tryin Tahoe-LAFS, but you know how it is... Peter.
On 10:04 am, peter.westlake@pobox.com wrote:
No, because I can't provide a simple test case for it. My only hope is to get the source of Twisted and bisect, then see what changed in that revision.
If it helps at all, there's now a Git mirror of Twisted so you can actually use "git bisect" for this (although I'd love to hear that someone has written a general multi-vcs bisect tool and that git is no longer the easiest way to do this sort of thing). Jean-Paul
On Sun, Jun 27, 2010 at 4:04 AM, Peter Westlake <peter.westlake@pobox.com> wrote:
python -c "import pkg_resources;print pkg_resources.require('Twisted')"
It says:
allmydata-tahoe-1.7.0$ python -c "import pkg_resources;print pkg_resources.require('Twisted')" [Twisted 10.0.0 (/localhome/packages/allmydata-tahoe-1.7.0/Twisted-10.0.0-py2.5-linux-i686.egg), zope.interface 3.4.0 (/usr/lib/python2.5/site-packages)]
Well, okay, that says that the one in your current working directory, in a .egg, would satisfy that requirement. But suppose you cd to a different directory and run it again. :-) Or rm that .egg and run it again. Thanks! The thing is, it should have been satisfied with the Twisted 8 that was already installed in your system, instead of creating that local Twisted 10.0.0 egg. Regards, Zooko
On Mon, 28 Jun 2010 21:41 -0600, "Zooko O'Whielacronx" <zooko@zooko.com> wrote:
On Sun, Jun 27, 2010 at 4:04 AM, Peter Westlake <peter.westlake@pobox.com> wrote:
python -c "import pkg_resources;print pkg_resources.require('Twisted')"
It says:
allmydata-tahoe-1.7.0$ python -c "import pkg_resources;print pkg_resources.require('Twisted')" [Twisted 10.0.0 (/localhome/packages/allmydata-tahoe-1.7.0/Twisted-10.0.0-py2.5-linux-i686.egg), zope.interface 3.4.0 (/usr/lib/python2.5/site-packages)]
Well, okay, that says that the one in your current working directory, in a .egg, would satisfy that requirement. But suppose you cd to a different directory and run it again. :-) Or rm that .egg and run it again. Thanks!
Here it is: allmydata-tahoe-1.7.0$ rm -r Twisted-10.0.0-py2.5-linux-i686.egg allmydata-tahoe-1.7.0$ python -c "import pkg_resources;print pkg_resources.require('Twisted')" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 524, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: Twisted
The thing is, it should have been satisfied with the Twisted 8 that was already installed in your system, instead of creating that local Twisted 10.0.0 egg.
Mystery solved! Though not in the way that I expected: all the individual Twisted packages are installed, like python-twisted-bin, python-twisted-core and so on, but the top-level python-twisted itself was not! Installing that gets the response: allmydata-tahoe-1.7.0$ python -c "import pkg_resources;print pkg_resources.require('Twisted')" [Twisted 8.1.0 (/usr/lib/python2.5/site-packages)] The python-twisted package is described as "transitional", which suggests that it might one day go away. Someone here is bound to know if it will. The reason this wasn't the explanation I expected, BTW, is that I expected to have 2.5.0 installed, not 8.1.0; but the bug only affects production machines, not the desktop where I'm installed Tahoe-LAFS. One last thing: this is on Debian Lenny, so I added the APT repository, but "apt-get update" gives these errors: Hit http://volatile.debian.org stable/volatile/non-free Packages Hit http://security.debian.org stable/updates/non-free Sources Ign http://allmydata.org lenny/main Packages Ign http://allmydata.org lenny/tahoe Packages Ign http://allmydata.org lenny/main Sources Ign http://allmydata.org lenny/tahoe Sources Ign http://allmydata.org lenny/main Packages Ign http://allmydata.org lenny/tahoe Packages Ign http://allmydata.org lenny/main Sources Ign http://allmydata.org lenny/tahoe Sources Err http://allmydata.org lenny/main Packages 404 Not Found Err http://allmydata.org lenny/tahoe Packages 404 Not Found Err http://allmydata.org lenny/main Sources 404 Not Found Err http://allmydata.org lenny/tahoe Sources 404 Not Found W: Failed to fetch http://allmydata.org/debian/dists/lenny/main/binary-i386/Packages 404 Not Found W: Failed to fetch http://allmydata.org/debian/dists/lenny/tahoe/binary-i386/Packages 404 Not Found W: Failed to fetch http://allmydata.org/debian/dists/lenny/main/source/Sources 404 Not Found W: Failed to fetch http://allmydata.org/debian/dists/lenny/tahoe/source/Sources 404 Not Found E: Some index files failed to download, they have been ignored, or old ones used instead. Thanks for your help, and apologies if this is straying off-topic for the list. Peter.
I'm not sure if we can or should do anything about the Debian python-twisted-core package not providing the Python "Twisted" distribution. I don't know much about the operation of the Tahoe-LAFS apt repository. Maybe it was temporarily down when you tried to apt-get update? Let's take it to the tahoe-dev@tahoe-lafs.org list. Thanks! Regards, Zooko
On Tue, 29 Jun 2010 09:14 -0600, "Zooko O'Whielacronx" <zooko@zooko.com> wrote:
I'm not sure if we can or should do anything about the Debian python-twisted-core package not providing the Python "Twisted" distribution.
I don't know much about the operation of the Tahoe-LAFS apt repository. Maybe it was temporarily down when you tried to apt-get update?
Let's take it to the tahoe-dev@tahoe-lafs.org list.
Okay - see you there! Thanks, Peter.
participants (5)
-
Brian Warner -
exarkun＠twistedmatrix.com -
Peter Westlake -
Yaroslav Fedevych -
Zooko O'Whielacronx