Mailman 3 July 2010 - Twisted

[Twisted-Python] ANNOUNCING Tahoe, the Least-Authority File System, v1.7.1
by Zooko O'Whielacronx 19 Jul '10

19 Jul '10

ANNOUNCING Tahoe, the Least-Authority File System, v1.7.1 The Tahoe-LAFS team is pleased to announce the immediate availability of version 1.7.1 of Tahoe-LAFS, an extremely reliable distributed storage system. Tahoe-LAFS is the first distributed storage system which offers "provider-independent security"—meaning that not even the operators of your storage servers can read or alter your data without your consent. Here is the one-page explanation of its unique security and fault-tolerance properties: http://tahoe-lafs.org/source/tahoe/trunk/docs/about.html Tahoe-LAFS v1.7.1 is the successor to v1.7.0, which was released June 18, 2010 [1]. v1.7.1 is a stable minor release which adds several bugfixes and improvements in security, forward-compatibility, packaging, usability, and portability. See the NEWS file [2] for details. WHAT IS IT GOOD FOR? With Tahoe-LAFS, you distribute your filesystem across multiple servers, and even if some of the servers are compromised by by an attacker, the entire filesystem continues to work correctly, and continues to preserve your privacy and security. You can easily share specific files and directories with other people. In addition to the core storage system itself, volunteers have built other projects on top of Tahoe-LAFS and have integrated Tahoe-LAFS with existing systems. These include frontends for Windows, Macintosh, JavaScript, iPhone, and Android, and plugins for Hadoop, bzr, mercurial, duplicity, TiddlyWiki, and more. See the Related Projects page on the wiki [3]. We believe that strong cryptography, Free and Open Source Software, erasure coding, and principled engineering practices make Tahoe-LAFS safer than RAID, removable drive, tape, on-line backup or "cloud storage" systems. This software is developed under test-driven development, and there are no known bugs or security flaws which would compromise confidentiality or data integrity under recommended use. (For all currently known issues please see the known_issues.txt file [4].) COMPATIBILITY This release is fully compatible with the version 1 series of Tahoe-LAFS. Clients from this release can write files and directories in the format used by clients of all versions back to v1.0 (which was released March 25, 2008). Clients from this release can read files and directories produced by clients of all versions since v1.0. Servers from this release can serve clients of all versions back to v1.0 and clients from this release can use servers of all versions back to v1.0. This is the tenth release in the version 1 series. This series of Tahoe-LAFS will be actively supported and maintained for the forseeable future, and future versions of Tahoe-LAFS will retain the ability to read and write files compatible with this series. LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [5] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1 or, at your option, any later version. (The Transitive Grace Period Public Licence has requirements similar to the GPL except that it allows you to delay for up to twelve months after you redistribute a derived work before releasing the source code of your derived work.) See the file "COPYING.TGPPL.html" [6] for the terms of the Transitive Grace Period Public Licence, version 1. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe-LAFS works on Linux, Mac OS X, Windows, Cygwin, Solaris, *BSD, and probably most other systems. Start with "docs/quickstart.html" [7]. HACKING AND COMMUNITY Please join us on the mailing list [8]. Patches are gratefully accepted -- the RoadMap page [9] shows the next improvements that we plan to make and CREDITS [10] lists the names of people who've contributed to the project. The Dev page [11] contains resources for hackers. SPONSORSHIP Tahoe-LAFS was originally developed by Allmydata, Inc., a provider of commercial backup services. After discontinuing funding of Tahoe-LAFS R&D in early 2009, they have continued to provide servers, bandwidth, small personal gifts as tokens of appreciation, and bug reports. Thank you to Allmydata, Inc. for their generous and public-spirited support. Google, Inc. is sponsoring Tahoe-LAFS development as part of the Google Summer of Code 2010. Google suggested that we should apply for the Summer of Code program, and when we did they generously awarded four sponsorships to students from around the world to hack on Tahoe-LAFS this summer. Thank you to Google, Inc. for their generous and public-spirited support. HACK TAHOE-LAFS! If you can find a security flaw in Tahoe-LAFS which is serious enough that feel compelled to warn our users and issue a fix, then we will award you with a customized t-shirts with your exploit printed on it and add you to the "Hack Tahoe-LAFS Hall Of Fame" [12]. ACKNOWLEDGEMENTS This is the fifth release of Tahoe-LAFS to be created solely as a labor of love by volunteers. Thank you very much to the team of "hackers in the public interest" who make Tahoe-LAFS possible. David-Sarah Hopwood and Zooko Wilcox-O'Hearn on behalf of the Tahoe-LAFS team July 18, 2010 Rainhill, Merseyside, UK and Boulder, Colorado, USA [1] http://tahoe-lafs.org/trac/tahoe/browser/relnotes.txt?rev=4514 [2] http://tahoe-lafs.org/trac/tahoe/browser/NEWS?rev=4577 [3] http://tahoe-lafs.org/trac/tahoe/wiki/RelatedProjects [4] http://tahoe-lafs.org/trac/tahoe/browser/docs/known_issues.txt [5] http://tahoe-lafs.org/trac/tahoe/browser/COPYING.GPL [6] http://tahoe-lafs.org/source/tahoe/trunk/COPYING.TGPPL.html [7] http://tahoe-lafs.org/source/tahoe/trunk/docs/quickstart.html [8] http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev [9] http://tahoe-lafs.org/trac/tahoe/roadmap [10] http://tahoe-lafs.org/trac/tahoe/browser/CREDITS?rev=4567 [11] http://tahoe-lafs.org/trac/tahoe/wiki/Dev [12] http://tahoe-lafs.org/hacktahoelafs/

1 0

[Twisted-Python] WebSocket Support #4173
by Reza Lotun 18 Jul '10

18 Jul '10

Hi All, As suggested by therve (Thomas) I just wanted to kick off a discussion about the current state of the WebSocket ticket (http://http://twistedmatrix.com/trac/ticket/4173). At the moment there are two patches linked against the ticket - mine and progrium's (Jeff?). Based on Jeff's comments it seems that while his patch may contain a more elegant implementation (to be honest, I haven't looked carefully at it yet), my patch seems to work against the dev version of chromium. Is there any way we can work to combine the patches, and extend the unit tests? Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun(a)gmail.com work: reza(a)tweetdeck.com twitter: @rlotun

3 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 18 Jul '10

18 Jul '10

1 0

[Twisted-Python] Twisted HTTP client supporting failover for multiple A records?
by Luke Marsden 16 Jul '10

16 Jul '10

Hi all, Is there any existing support for any Twisted HTTP client to simulate the behaviour of all modern browsers in that -- if an address returns multiple A records -- and if one IP fails (connection refused, etc) then the client attempts a number of the other IPs before giving up? If not, where should I start? I understand that client.Agent is more modern than client.getPage. Thanks for an awesome framework! -- Best Regards, Luke Marsden Hybrid Logic Ltd. Web: http://www.hybrid-cluster.com/ Hybrid Web Cluster - cloud web hosting based on FreeBSD and ZFS Mobile: +447791750420

5 10

[Twisted-Python] Deferred instance has no attribute '__getitem__' ??????
by Ken MacDonald 16 Jul '10

16 Jul '10

I have a function and two convenience functions, like this: @defer.inlineCallbacks def round_value_and_percent(id, value, percent): daily = 1. params = dict(nutidin=id, valuein=value, dvin=daily) query = "ROUND_NUT_DV" q_result = yield named_query_param(ctx, "nut", query, params) // db query rounded = {} rounded["value"] = q_result[0][0] rounded["percent"] = q_result[0][1] defer.returnValue((rounded['value'], rounded['percent'])) @defer.inlineCallbacks def round_val(id, value, rule): """ Return first element of round_value_and_percent """ defer.returnValue(round_value_and_percent(id, value, 0, rule)[0]) @defer.inlineCallbacks def round_per(id, percent, rule): """ Return second element of round_value_and_percent. """ defer.returnValue(round_value_and_percent(id, 0, percent, rule)[1]) When I do: result = yield round_val(1, 2, 3) I get this: Deferred instance has no attribute '__getitem__' Clues appreciated! Ken

3 3

[Twisted-Python] twisted.protocols.basic.NetstringReceiver: Some errors are not visible
by Carlos Valiente 14 Jul '10

14 Jul '10

>From twisted/protocols/basic.py: class NetstringReceiver(protocol.Protocol): def doLength(self): ... raise NetstringParseError, "netstring too long" ... def dataReceived(self, data): ... try: ... self.doLength() ... except NetstringParseError: self.transport.loseConnection() self.brokenPeer = 1 The error message "netstring too long" is shadowed by the except block. I'd like to see it in my log files. Would it make sense to emit logging messaged for those errors, or is there any particular reason why they are silenced? Cheers, Carlos

2 3

[Twisted-Python] Knowing whether spawnProcess did anything useful
by Dominic van Berkel 14 Jul '10

14 Jul '10

Hiya, I'm looking for a way to know whether spawnProcess() did anything useful. That is, whether anything has actually been executed. In my specific case the executables are all persistent (pluggable modules, they stick around for a while) so based on the following snippet I thought I could check whether the resulting ProcessTransport has a pid: "From before IProcessProtocol.makeConnection is called to before IProcessProtocol.processEnded is called, pid is an int giving the platform process ID of this process. pid is None at all other times."[1] However, a quick tour on the REPL shows otherwise (than I expected, not necessarily otherwise than what the docs say): >>> from twisted.internet import reactor >>> from twisted.internet import protocol >>> p = protocol.ProcessProtocol() >>> f = reactor.spawnProcess(p, "fail", ["fail"]) >>> f <Process pid=21053 status=-1> >>> f.pid 21053 I thought status might be of some relevance, but it didn't really change when I gave it a shot with actual executables. Something tells me I'm being massively stupid here, but I'm genuinely stuck. Can't really believe that there's no way to do this, though. Thanks. [1] http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.… -- Dominic van Berkel "You don't have conversations with microprocessors. You tell them what to do, then helplessly watch the disaster when they take you literally." - David Brin, Startide Rising

2 2

[Twisted-Python] Email problem
by Alvin Wang 13 Jul '10

13 Jul '10

Problem send email through gmail with an attachment Any help is appreciated Thanks Alvin """ Does not work """ from twisted.internet.ssl import ClientContextFactory from twisted.internet.defer import Deferred from twisted.mail.smtp import ESMTPSenderFactory from twisted.internet import reactor import smtplib, sys, MimeWriter, mimetypes, mimetools, base64 import StringIO import time import os sender = 'acct(a)gmail.com' senderpw = 'password' to = 'alvinwang(a)gmail.org' subject = 'gc' + str(time.time()) text = 'Message Text Body' attachments = ["/home/alvin/Pictures/alvin.jpg", "/home/alvin/Pictures/badlogo.gif"] contextFactory = ClientContextFactory() result = Deferred() message = StringIO.StringIO() writer = MimeWriter.MimeWriter(message) writer.addheader('To', to) writer.addheader('From', sender) writer.addheader('Subject', subject) writer.addheader('MIME-Version', '1.0') writer.startmultipartbody('mixed') part = writer.nextpart() body = part.startbody('text/plain') part.flushheaders() body.write(text) # now add the attachments if attachments is not None: for a in attachments: filename = os.path.basename(a) ctype, encoding = mimetypes.guess_type(a) if ctype is None: ctype = 'application/octet-stream' encoding = 'base64' elif ctype == 'text/plain': encoding = 'quoted-printable' else: encoding = 'base64' part = writer.nextpart() part.addheader('Content-Transfer-Encoding', encoding) body = part.startbody("%s; name=%s" % (ctype, filename)) mimetools.encode(open(a, 'rb'), body, encoding) # that's all falks writer.lastpart() m = message.getvalue() message.close() factory = ESMTPSenderFactory(sender, senderpw, sender, to, m, result, contextFactory=contextFactory) reactor.connectTCP('smtp.gmail.com', 25, factory) result.addCallback(lambda ign: reactor.stop()) reactor.run() /usr/lib/python2.6/dist-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead import MimeWriter, tempfile, rfc822 Traceback (most recent call last): File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) --- <exception caught here> --- File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite why = getattr(selectable, method)() File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 137, in doRead return Connection.doRead(self) File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 460, in doRead return self.protocol.dataReceived(data) File "/usr/lib/python2.6/dist-packages/twisted/protocols/basic.py", line 251, in dataReceived why = self.lineReceived(line) File "/usr/lib/python2.6/dist-packages/twisted/mail/smtp.py", line 1069, in lineReceived why = self._okresponse(self.code,'\n'.join(self.resp)) File "/usr/lib/python2.6/dist-packages/twisted/mail/smtp.py", line 1134, in smtpState_data self.getMailData(), self.transport, self.transformChunk) File "/usr/lib/python2.6/dist-packages/twisted/protocols/basic.py", line 493, in beginFileTransfer self.consumer.registerProducer(self, False) File "/usr/lib/python2.6/dist-packages/twisted/internet/abstract.py", line 313, in registerProducer producer.resumeProducing() File "/usr/lib/python2.6/dist-packages/twisted/protocols/basic.py", line 499, in resumeProducing chunk = self.file.read(self.CHUNK_SIZE) exceptions.AttributeError: 'str' object has no attribute 'read' Unhandled error in Deferred: Traceback (most recent call last): Failure: exceptions.Exception: Consumer asked us to stop producing Traceback (most recent call last): File "postprob.py", line 71, in <module> reactor.run() File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1170, in run self.mainLoop() File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1182, in mainLoop self.doIteration(t) File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 140, in doSelect _logrun(selectable, _drdw, selectable, method, dict) --- <exception caught here> --- File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.6/dist-packages/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/lib/python2.6/dist-packages/twisted/internet/selectreactor.py", line 156, in _doReadOrWrite self._disconnectSelectable(selectable, why, method=="doRead") File "/usr/lib/python2.6/dist-packages/twisted/internet/posixbase.py", line 197, in _disconnectSelectable selectable.connectionLost(failure.Failure(why)) File "/usr/lib/python2.6/dist-packages/twisted/internet/tcp.py", line 678, in connectionLost self.connector.connectionLost(reason) File "/usr/lib/python2.6/dist-packages/twisted/internet/base.py", line 1039, in connectionLost self.factory.clientConnectionLost(self, reason) File "/usr/lib/python2.6/dist-packages/twisted/mail/smtp.py", line 1713, in clientConnectionLost self._processConnectionError(connector, err) File "/usr/lib/python2.6/dist-packages/twisted/mail/smtp.py", line 1721, in _processConnectionError self.file.seek(0, 0) exceptions.AttributeError: 'str' object has no attribute 'seek'

2 1

[Twisted-Python] asynchronous python generator
by Jean Daniel 12 Jul '10

12 Jul '10

Hello, I wrote a small client protocol which connects to a notification server. The client role is to connect, and then to print the notification which comes from the server until the server says "stop" (a netcat server do just fine, I use "nc -C -l 6789"). I would like the API for this client protocol to be compatible with the "for" python loop. Here is the client class Notif(basic.LineReceiver): def lineReceived(self, data): self.d.callback(data) def __iter__(self): return self @defer.inlineCallbacks def next(self): self.d = defer.Deferred() notif = yield self.d if notif=="stop": defer.returnValue(Failure(StopIteration())) else: defer.returnValue(notif) Here is an example of how to use this client: @defer.inlineCallbacks def gotConnection(conn): for notif in conn: print notif reactor.stop() c = protocol.ClientCreator(reactor, Notif) c.connectTCP("localhost", 6789).addCallback(gotConnection) reactor.run() Except that is does not work: I think that the next() method of the generator is indeed automatically called by the "for" machinery but its return value is not yielded as the inlineCallbacks requires it. This works for instance, and it is very close to a for loop: @defer.inlineCallbacks def gotConnection(conn): while True: try: print (yield conn.next()) except StopIteration: break Does someone knows how to make the for loop work with data coming from network requests?

3 4

[Twisted-Python] https server example with client certs for twisted
by Gelonida 11 Jul '10

11 Jul '10

H, I'd like to implement a small https twisted server, which can wrap a wsgi module and which can verify client certificates I am able to do this with apache, but would like to do the same with twisted. What I have running is following. The #!/bin/sh certfile=mycert.crt keyfile=mycert.key wsgi_module=tw_wrapper.application twistd -n web --https 4433 \ --certificate $certfile --privkey $keyfile \ --wsgi $wsgi_module I wondered where I could find any good example code implementing a web server with client certificate authentication ( and ideally with a subsequent wsgi module) TIA

2 1