Mailman 3 March 2014 - Twisted

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com March 22, 2014

March 22, 2014

1 0

[Twisted-Python] git?
by Chris Withers March 20, 2014

March 20, 2014

Hi All, Is there a git mirror of the twisted repo anywhere? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

10 25

[Twisted-Python] Twisted 14.0.0-pre1 Announcement
by HawkOwl March 19, 2014

March 19, 2014

I am pleased to announce that the first prerelease of Twisted 14.0 is hot off the code presses for testing! Tarballs for this prerelease can be found at <http://twistedmatrix.com/Releases/pre/14.0.0pre1>, with the changelog at <http://twistedmatrix.com/Releases/pre/14.0.0pre1/NEWS.txt>. Highlights include: -Twisted Positioning (`twisted.positioning`) makes its entry into Twisted! It comes ready to talk with common GPS devices, and will supersede `twisted.protocols.gps`. - A wealth of SSL/TLS improvements, including ECDHE support, a stronger default set of ciphers, and strengthening against attacks such as CRIME. A Twisted Web server with pyOpenSSL 0.14 is capable of getting an A in Qualys SSL Labs tests. - Python 3 improvements, including the ability for `pip install` to install all ported modules. - Twisted Pair’s TUN/TAP support has been overhauled, with documentation and full test coverage. - Significant documentation improvements, including more API documentation for Twisted Mail & Twisted Names, narrative documentation for Twisted Names, and a migration to Sphinx for building Twisted narrative docs. For more information and a full list of changes, check the NEWS.txt file. Please download the tarballs and test them with your applications, so we can make sure we’re all ready for release! -hawkowl

1 0

[Twisted-Python] Twisted 14.0.0-pre1 Announcement
by HawkOwl March 19, 2014

March 19, 2014

I am pleased to announce that the first prerelease of Twisted 14.0 is hot off the code presses for testing! Tarballs for this prerelease can be found at <http://twistedmatrix.com/Releases/pre/14.0.0pre1>, with the changelog at <http://twistedmatrix.com/Releases/pre/14.0.0pre1/NEWS.txt>. Highlights include: -Twisted Positioning (`twisted.positioning`) makes its entry into Twisted! It comes ready to talk with common GPS devices, and will supersede `twisted.protocols.gps`. - A wealth of SSL/TLS improvements, including ECDHE support, a stronger default set of ciphers, and strengthening against attacks such as CRIME. A Twisted Web server with pyOpenSSL 0.14 is capable of getting an A in Qualys SSL Labs tests. - Python 3 improvements, including the ability for `pip install` to install all ported modules. - Twisted Pair’s TUN/TAP support has been overhauled, with documentation and full test coverage. - Significant documentation improvements, including more API documentation for Twisted Mail & Twisted Names, narrative documentation for Twisted Names, and a migration to Sphinx for building Twisted narrative docs. For more information and a full list of changes, check the NEWS.txt file. Please download the tarballs and test them with your applications, so we can make sure we’re all ready for release! -hawkowl

1 0

[Twisted-Python] Getting the options object from the service
by Jonas Brunsgaard March 18, 2014

March 18, 2014

Hello Everybody What is the prefered way to get the options object from the servicemaker when using twistd plugins. I guess i could access the object via from twisted.plugins.my_plugin import myServiceMaker options = myServicemMaker.options But is there a better way to do it? Somthing like from somwhere import get_service_config() options = get_service_config() All the best -- Brunsgaard

2 2

[Twisted-Python] object identify and lifecycle in pb
by Daniel Sank March 17, 2014

March 17, 2014

Dear Twisted users, I have been writing a developer library under pb and I've come to a problem that I don't know how to solve. I suspect folks here will have dealt with similar problems so I want to get your advice. This is a long post so I've put specific questions on their own lines. Suppose we're part of a society of spies. We desire to possess top secret Widgets. Widgets can be purchased from a black market, or obtained from other spies. Suppose we implement this by having Widget inheret from pb.Cacheable. When I buy a Widget from the black market I am passed a pb.RemoteCache of that Widget. We use Cacheable so that when the Widget receives a top secret message, my RemoteCache is notified of the message. Now suppose I give my Widget to my trusted co-spy Glyph. On the network this is implemented by passing Glyph a RemoteCache of that Widget and by remembering to stop updating my RemoteCache, since I'm no longer in posession of the Widget. This could be done simply by keeping a .owner attribute on the Widget so that it knows whos RemoteCache to update when secret messages come in. Now suppose it turns out Glyph is a traitor and secretly gives the Widget to Mr. Evil. Again, the Widget's .owner attribute changes and everything seems hunky dory. The problem comes in when I infiltrate Mr. Evil's castle and steal the Widget. The server passes me a reference to the Widget which, because of the way pb works, I can immediately identify as the same widget I originally gave Glyph, allowing me to identify him as a traitor. This is bad because my ability to trace the Widget comes from pb, not from my own decision to give (or not give) the Widgets distinguishing characteristics. Q1: Is there a simple solution to this problem? My idea is to have each Widget implemented as a vanilla object that knows nothing about pb or the network, but which can (through extension in a subclass) create pb objects as users need them. The problem I'm having with that idea is that I haven't figured out a reasonable to distribute the network layer objects, particularly in cases in which objects "contain" one another. Let's try an example. First we have a vanilla Widget class Widget(object): def __init__(self): owner = None def receiveMessage(self, msg): print("Widget received message: %s"%(msg,)) Next we need to extend this object so that it can spawn and interact with network-layer objects. To do this we add two features 1) We use a decorator that allows methods to notify other observing methods when they are run. For a possible implementation see the "obsub" module on PyPI. I have my own implementation for reasons. 2) We add a method that spawns network layer objects. class FeaturefulWidget(Widget): networkClass = WidgetNetwork @event #Allows other bound methods to observe this method def receiveMessage(self, msg): return Widget.receiveMessage(self, msg) def makeNetworkModel(self): return self.networkClass(self) def addCallback(self, methodName, callback): """ Sign up a callback to be invoked when a method decorated by @event is called methodName: the name of the method to be observed on the present object. callback: The thing to call when methodName is invoked. """ <just trust me that this works> Now we can make the actual cacheable object, class WidgetNetworkModel(pb.Cacheable): def __init__(self, obj): self.obj = weakref.proxy(obj) # Sign up our receiveMessage method to be called whenever # obj.receiveMessage is called. obj.addCallback("receiveMessage", self.receiveMessage) # The usual set of observers needed by a pb.Cacheable self.observers = weakref.WeakKeyDictionary{} #perspective -> observer def getStateToCacheAndObserveFor(self, perspective, observer): self.observers[p] = observer # Get state data from the actual object return {'some key': self.obj.someData, etc.} def receiveMessage(self, msg): for p in self.observers: <possibly decide whether p should know about this message> self.observers[p].callRemote("receiveMessage", msg) and its remote cache, class WidgetRemoteCache(pb.RemoteCache): def setCopyableState(self, state): for k, v in state.items(): setattr(self, k, v) def observe_receiveMessage(self, msg): print("Top secret message received: %s"%(msg,)) In an application we would work most directly with FeaturefulWidget. When a user gets its hands on a particular instance of FeaturefulWidget called myWidget, we would either pass or return myWidget.makeNetworkModel(). Q2: Is there a good mental framework we can use to decide when to make new network models and when to pass multiple remote references to the same one? Q3: We had a line in getStateToCacheAndObserveFor where we put self.obj.someData in a dict to pass over the wire. In many cases objects will contain other objects that should be networkified before they can be passed over the wire. Certainly we can manually put self.obj.someData.makeNetworkModel() into the dict as necessary, but I wonder if there's a nice way to handle this sort of thing. Thanks! Daniel

1 0

[Twisted-Python] Contributing documentation inside CONTRIBUTING repo file
by Adi Roiban March 17, 2014

March 17, 2014

Hi. Right now CONTRIBUTING file from twisted repo sends developers to: https://twistedmatrix.com/trac/wiki/ContributingToTwistedLabs What do you think about having basic contributing documentation inside the repo? I am talking about these 2 pages: https://twistedmatrix.com/trac/wiki/BasicGuideToContributingCode https://twistedmatrix.com/trac/wiki/TwistedDevelopment Commiters Check list can be left on the wiki page: https://twistedmatrix.com/trac/wiki/CommitterCheckList -------- I think that those pages are very important to assist new developers in creating a successful patch (ie one that is merged without to much ping-pong) Even if I now have wiki edit page, I don't have the courage to change the wiki page (for example to add information about twistedchecker or how to build the new docs). Changing a file and submitting for review feels more comfortable. Also, developers could always refer to that file from inside their preferred text editor. --------- Is there someone review-ul changes on the wiki pages? Thanks! -- Adi Roiban

6 10

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com March 15, 2014

March 15, 2014

1 0

[Twisted-Python] ANN: Crochet 1.1.0
by Itamar Turner-Trauring March 14, 2014

March 14, 2014

Crochet is an MIT-licensed library that makes it easier for blocking or threaded applications like Flask or Django to use the Twisted networking framework. Crochet provides the following features: * Runs Twisted's reactor in a thread it manages. * The reactor shuts down automatically when the process' main thread finishes. * Hooks up Twisted's log system to the Python standard library logging framework. Unlike Twisted's built-in logging bridge, this includes support for blocking Handler instances. * A blocking API to eventual results (i.e. Deferred instances). This last feature can be used separately, so Crochet is also useful for normal Twisted applications that use threads. Downloads: https://pypi.python.org/pypi/crochet Documentation: https://crochet.readthedocs.org/en/latest/ * What's New in 1.1.0* Bug fixes: * EventualResult.wait() can now be used safely from multiple threads, thanks to Gavin Panella for reporting the bug. * Fixed reentrancy deadlock in the logging code caused by http://bugs.python.org/issue14976, thanks to Rod Morehead for reporting the bug. * Crochet now installs on Python 3.3 again, thanks to Ben Cordero. * Crochet should now work on Windows, thanks to Konstantinos Koukopoulos. * Crochet tests can now run without adding its absolute path to PYTHONPATH or installing it first. Documentation: * EventualResult.original_failure is now documented. **

1 0

[Twisted-Python] Freezing Ampoule or the Invasion of Infinite Zombie Processes
by Burak Nehbit March 13, 2014

March 13, 2014

Hi everyone, It seems that I ran into a rather inextricable problem, would love to see what you think. As you might remember, I have been working on a distributed, anonymous community platform(s) called Aether ( www.getaether.net ) which is largely a Twisted codebase in its networking system. After releasing the app, I ended up with performance problems, in some part due to my programming incompetence, and some part because I was also using it to drive a GUI with qtreactor, which ran its own Javascript interpreter. The codebase is largely cleaned up now and the current architecture of the program involves two processes that talk to each other via Ampoule: a default reactor core, which handles network events, and a qtreactor which is solely concerned with the user interface. It works very well. Stress induced by networking does not slow down the UI and vice versa. The problem here is that I can’t freeze and package the app via PyInstaller to release it. What happens is that when I run the packaged application, the application starts to spawn an infinite amount of copies of itself. This seems to be the default behaviour of Ampoule, in when a worker process (my GUI) fails, it spawns a new one. 1) How can I disable this feature? I want ampoule to not restart a failed worker process. 2) How can I get information about what’s going wrong in the worker process? I am using a custom bootstrap code, and it seems that even if I deliver an empty bootstrap code to make it a noop, the process still fails and a new one is spawned. Or to be more accurate, just a new process is spawned, I don’t know whether if it is failing, but nothing happens on spawned processes, none of them start to show GUI or any other signs of life. If I don’t force quit my frozen app, it will create an increasing number of zombie processes. I have Twisted logging set up in the GUI process, but it doesn’t even hit that line, so I get no exceptions or any message of any kind. It fails even with empty bootstrap code. Any idea on how to approach this problem? I have nothing to start with , so anything would be appreciated. I’m reading the source code of Ampoule but found nothing useful so far. The non-frozen version of the app works fine. I also have a prior version of the app that can be packaged, so there’s nothing wrong with my Twisted code that can’t be packaged. It’s Ampoule that is newly added, and so far resisting the refrigerator. Best regards, Burak

3 7