Mailman 3 May 2008 - Twisted

[Twisted-Python] the sequence of executing defers' callbacks
by Xian Chen 23 May '08

23 May '08

Hi All, This is a fundamental question about Twisted. In the normal case, all the callbacks belonging to one defer object will be triggered following the chain sequence. ( Let's assume no errback will be fired.) such as callback1--->callbac2--->callbackN. However, if I want to trigger callback1 when MSG1 is ready, callback2 when MSG2 is ready, and so on, how can I do that? Of course, the MSGs are coming at random sequence. Thanks in advance, Xian Chen

4 4

Re: [Twisted-Python] the sequence of executing defers' callbacks
by Terry Jones 22 May '08

22 May '08

>>>>> "Xian" == Xian Chen <hoganxian(a)gmail.com> writes: Xian> However, if I want to trigger callback1 when MSG1 is ready, callback2 Xian> when MSG2 is ready, and so on, how can I do that? Xian> Of course, the MSGs are coming at random sequence. Hi Xian Do you have multiple functions that are returning these MSGx results? If so, it sounds like you might be looking for DeferredList. See the docs at http://twistedmatrix.com/projects/core/documentation/howto/defer.html Terry

1 0

[Twisted-Python] Foolscap-0.2.7 released, Foolscap vs. debian openssl bug
by Brian Warner 22 May '08

22 May '08

I released Foolscap-0.2.7 the other week, just a few days after the release of 0.2.6 . This fixes a brown-paper-bag bug in 0.2.6 that prevented the 'flogtool' utility from running at all. Tarballs can be obtained from foolscap.lothar.com and from PyPI. All releases are signed by my GPG public key: 0x1514A7BD. http://pypi.python.org/packages/source/f/foolscap/foolscap-0.2.7.tar.gz#md5… http://pypi.python.org/packages/source/f/foolscap/foolscap-0.2.7.zip#md5=95… For documentation, bugs, and patches, please visit http://foolscap.lothar.com/trac . == Debian OpenSSL Bugs and Foolscap == As you've probably heard[1], last week the debian project published a security advisory that described how the openssl package has been using a 18-ish-bit random number generator for the last two years. As a result, keys and certificates generated by the affected versions (or used for DSA signatures by affected versions) should be considered compromised and immediately replaced. This affects the last 3 or 4 Ubuntu releases too (Dapper was correct, Edgy was broken). All affected releases now have fixed versions available for download, and obviously everyone using an affected release needs to upgrade immediately. [1]: http://lists.debian.org/debian-security-announce/2008/msg00152.html Since the bug affected key generation, keys which were created by the broken versions will remain broken even after openssl itself is upgraded. The Debian and Ubuntu projects have released tools to detect weak ssh keys that were generated by these versions and disable them. Other projects (including Foolscap) use generated keys for various purposes, so the effects of this bug will linger even after openssl has been fixed. How does this affect Foolscap? The short answer: Weak keys generated by the broken versions of openssl will enable a man-in-the-middle attack, which (if successful) would reveal the private swissnums that protect access to Referenceables. Foolscap users who have created Tubs while they had the broken versions installed should stop using the associated FURLs and generate new certificates/FURLs. The longer answer: There are three things in Foolscap that use random numbers: 1: TubIDs [COMPROMISED]. These are a hash of the public key that corresponds to a Tub's private key, and are included in FURLs (as the base32 part immediately following the pb:// prefix). This provides protection against a man-in-the-middle attack: when connecting to a FURL, the certificate of the remote end is compared against the TubID in the FURL, and the connection is dropped with extreme prejudice if they do not match. This validation is performed before any secrets are passed over the wire. 2: SwissNums [SAFE]. These are a randomly-generated name for a reference, created when tub.registerReference(obj) is called without a name= argument, and when a Referenceable is sent over the wire without being explicitly registered. They appear at the right-hand end of the FURL, after the last slash. Most Referenceables are given these unguessable names, because that limits access to parties that have been explicitly granted access. (when registerReference() is called with a guessable name, anyone who can guess that name will get reference to the given object). The capability-based security of foolscap Referenceables depends upon these swissnums remaining secret. 3: SSL transport sessions [VULNERABLE]. Foolscap runs over SSL, which uses random numbers to generate the Diffie-Hellman key agreement parameters. An eavesdropper who can guess one of these parameters will be able to determine the session key, and can decrypt the remainder of the conversation. TubIDs are derived from an SSL certificate that OpenSSL generates on demand, and if the broken version of OpenSSL was used for this purpose, than that certificate will be guessable. (at some point it would be great to write a tool that would generate the 128000-ish TubIDs that can be generated by the broken SSL, to create a blacklist like the ones created by debian/ubuntu for ssh keys). So if you've generated a TubID with the broken version of openssl, it is pretty easy to guess the corresponding private key. This allows an attacker to mount an undetectable man-in-the-middle attack: if they can convince you to make your TCP connection to their server instead of the right one, then they can impersonate the real server, and they'll get to see the unencrypted conversation (revealing the private swissnums). Swissnums are generated by calling os.urandom(), which was not affected by the openssl bug. As long as one of the other problems (man-in-the-middle attack) doesn't reveal your swissnums to an attacker, access to Referenceables will remain limited to the parties to whom you've explicitly granted it (by telling them the swissnum, in a FURL). Foolscap uses SSL for its encrypted transport, and SSL frequently uses Diffie-Hellman key negotiation to establish a session key. This key negotiation process depends upon random numbers, generated by the same buggy RNG that is used to create keys. So if one of the two ends of an encrypted session was using the buggy version of openssl, then an attacker may be able to guess the session key, and thus decrypt the conversation. (note, I'm not a cryptographer, and I'm not as confident about this attack as I am about the others). So like the MitM attack, it seems likely that swissnums used by sessions established during the buggy openssl era may be compromised: an eavesdropper who recorded that encrypted conversation may be able to decrypt it after the fact. Of course, the whole idea of capabilities is to implement the Principle Of Least Authority: each FURL gives access to a specific Referenceable that will only do some specific limited task. If an MitM/eavesdropper manages to guess your supposedly-unguessable swissnums, they'll still only be able to do whatever the Referenceable will let them do. So to summarize: if you've been using the buggy openssl libraries, here's what you need to do: 1: upgrade openssl 2: throw out your old FURLs (tell everyone that you've given them to to stop using them): this protects clients from the MitM attack. 3: throw out your old Tub certificates: the certFile= argument to Tub() provides a persistent location for this.. just delete that file. This prevents clients from ever successfully using the old FURLs to connect to your server. 4: restart the Tub, allowing it to create a new certificate 5: regenerate and distribute new FURLs You can also just upgrade openssl (and keep using your old certs and FURLs) if you're willing to remain vulnerable to man-in-the-middle attacks or eavesdroppers who have recorded your earlier conversations. cheers, -Brian

1 0

[Twisted-Python] Re: Twisted-Python Digest, Vol 50, Issue 26
by ixafei 22 May '08

22 May '08

hi

5 4

[Twisted-Python] Memory size of Deferreds
by Martin Geisler 21 May '08

21 May '08

Hello, I am working on a project called VIFF (http://viff.dk/) which does secure multiparty computations. Twisted, and in particular the Deferred class, has been a great tool for this! Each variable in a computation is stored as a Deferred, and the computation builds an expression tree based on these Deferreds, which is then evaluated as the Deferreds call their callbacks. What worries me is the size of a single Deferred. One user wanted to do computations on very large lists of Deferreds and reported that his programs used more memory than expected. I tried to measure the size of a Deferred by creating large lists of them and found that a single empty Deferred takes up about 200 bytes. Adding a callback brings that up to about 500 bytes. The discussion can be found here: http://thread.gmane.org/gmane.comp.cryptography.viff.devel/232 I might be measuring this the wrong way -- I simply looked at how the memory size grew in 'top' when I created more and more lists. As noted in the thread above, it helps defining the __slots__ attribute -- it saves the 144 bytes needed for the default object dictionary. Maybe that could be done for Deferred? Has anybody measured how much memory the C implementation users per Deferred? I have not tried it out yet. http://twistedmatrix.com/trac/ticket/2245 -- Martin Geisler VIFF (Virtual Ideal Functionality Framework) brings easy and efficient SMPC (Secure Multi-Party Computation) to Python. See: http://viff.dk/.

7 11

回复： Re: [Twisted-Python] Re: Memory size of D eferreds
by arlenecc 21 May '08

21 May '08

原信息主题： Re: [Twisted-Python] Re: Memory size of Deferreds 发件人： Tristan Seligmann <mithrandi(a)mithrandi.net> 日期： 2008-05-21 11:38 * Martin Geisler <mg(a)daimi.au.dk> [2008-05-21 13:03:30 +0200]: > In particular, I guess you need to catch the QueueUnderflow exceptions > and then somehow know when it is time to call get() again. DeferredSemaphore may be more appropriate. -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <<signature.asc>>

1 0

回复： Re: [Twisted-Python] Re: Memory size of D eferreds
by arlenecc 21 May '08

21 May '08

原信息主题： Re: [Twisted-Python] Re: Memory size of Deferreds 发件人： Tristan Seligmann <mithrandi(a)mithrandi.net> 日期： 2008-05-21 11:38 * Martin Geisler <mg(a)daimi.au.dk> [2008-05-21 13:03:30 +0200]: > In particular, I guess you need to catch the QueueUnderflow exceptions > and then somehow know when it is time to call get() again. DeferredSemaphore may be more appropriate. -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <<signature.asc>>

1 0

回复： Re: [Twisted-Python] Re: Memory size of D eferreds
by arlenecc 21 May '08

21 May '08

原信息主题： Re: [Twisted-Python] Re: Memory size of Deferreds 发件人： Tristan Seligmann <mithrandi(a)mithrandi.net> 日期： 2008-05-21 11:38 * Martin Geisler <mg(a)daimi.au.dk> [2008-05-21 13:03:30 +0200]: > In particular, I guess you need to catch the QueueUnderflow exceptions > and then somehow know when it is time to call get() again. DeferredSemaphore may be more appropriate. -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <<signature.asc>>

1 0

回复： Re: [Twisted-Python] Re: Memory size of D eferreds
by arlenecc 21 May '08

21 May '08

原信息主题： Re: [Twisted-Python] Re: Memory size of Deferreds 发件人： Tristan Seligmann <mithrandi(a)mithrandi.net> 日期： 2008-05-21 11:38 * Martin Geisler <mg(a)daimi.au.dk> [2008-05-21 13:03:30 +0200]: > In particular, I guess you need to catch the QueueUnderflow exceptions > and then somehow know when it is time to call get() again. DeferredSemaphore may be more appropriate. -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <<signature.asc>>

1 0

回复： Re: [Twisted-Python] Re: Memory size of D eferreds
by arlenecc 21 May '08

21 May '08

原信息主题： Re: [Twisted-Python] Re: Memory size of Deferreds 发件人： Tristan Seligmann <mithrandi(a)mithrandi.net> 日期： 2008-05-21 11:38 * Martin Geisler <mg(a)daimi.au.dk> [2008-05-21 13:03:30 +0200]: > In particular, I guess you need to catch the QueueUnderflow exceptions > and then somehow know when it is time to call get() again. DeferredSemaphore may be more appropriate. -- mithrandi, i Ainil en-Balandor, a faer Ambar _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <<signature.asc>>

1 0