Mailman 3 June 2013 - Twisted

[Twisted-Python] Twisted 13.1.0pre1 is here!
by Ashwini Oruganti 17 Jun '13

17 Jun '13

Hi everyone, I'm pleased to announce that the pre-release for the upcoming Twisted 13.1 is now available! You can download the tarballs here: http://twistedmatrix.com/~ashfall/13.1.0pre1/ Highlights for this release include: * trial now has an --exitfirst flag which stops the test run after the first error or failure. * twisted.internet.ssl.CertificateOptions now supports chain certificates. * twisted.conch.endpoints.SSHCommandClientEndpoint is a new IStreamClientEndpoint which supports connecting a protocol to the stdio of a command running on a remote host via an SSH connection. * twisted.web.xmlrpc.QueryProtocol now generates valid Authorization headers for long user names and passwords. * self.transport.resumeProducing() will no longer raise an AssertionError if called after self.transport.loseConnection() For the complete list, see the NEWS file. Please download the tarballs and test them! -Ashwini

2 1

[Twisted-Python] 13.1.0 and #6499
by Hynek Schlawack 15 Jun '13

15 Jun '13

Hi, #6499 (Teach chain certificates to SSL server endpoints string syntax) has been pronounced ready for merge but since I’m no committer, I can’t do that myself. It would be *great* if it could be merged before Ashwini cuts the release branch! Thanks & sorry for the noise, —h

2 1

[Twisted-Python] Announcing twisted-dev-tools
by Tom Prince 15 Jun '13

15 Jun '13

http://labs.twistedmatrix.com/2013/06/announcing-twisted-dev-tools.html I'd like to announce the release of twisted-dev-tools. It is a project that collects various python scripts useful for developer working on twisted itself. Right now, it contains the following tools. - force-build: This is an updated version of force-builds.py from twisted-trac-integration. It has a different (more flexible) syntax. If run from a git repository, where the current commit has been pushed to svn, running it with no arguments will automatically build the corresponding branch. - mkbranch: A helper for thos use git: it creates a branch in svn, with a standard commit message. Eventually, this should be enhanced to automatically fetch that commit, and switch to the branch locally. - review-tickets: Command-line list of tickets currently in review - fetch-ticket: Command-line tool to view a ticket - get-attachemnt: Tool for interacting with trac attachments. - list: list all attachments on a given ticket - get: gets a gien ticket (defaults to the lat) - apply: applies that last attachment to the current git repository, and commits it with an appropriate message Most of the functionality is also exposed as a python library, so custom scripts are possible as well. The code is available at https://github.com/twisted/twisted-dev-tools and on pypi https://pypi.python.org/pypi/twisted-dev-tools

2 1

[Twisted-Python] Simple bidirectional Socket Client
by Raketenschnitzel 14 Jun '13

14 Jun '13

Hi there, i'm new to twisted, but i want to use it in a project for dealing with the socket connection to a server for receiving and sending data. I already did some projects in Delphi and there you have an build-in eventdriven socket class. Python itself lacks at this feature (afaik) and so i stumpled upon twisted. Since its only a small part of my task, im looking for an easy solution but after consulting google for two days and testing around im still failing at get it working. What i got so far: class ControllerProtocol(Protocol): def connectionMade(self): # send Greeting self.factory.isConnected = True self.transport.write("hello, world!") def dataReceived(self, data): "Parse received data and send corresponding Message." self.transport.write(self.factory.reactOnReceive(data)) def connectionLost(self, reason): #try to reconnect self.factory.isConnected = False print "connection lost" def sendMessage(self, data): print 'Send:', data self.transport.write(data) class Communicator(ClientFactory): protocol = ControllerProtocol def __init__(self, Host, Port): if isinstance(Host, str) and isinstance(Port, int) : self.myHost = Host self.myPort = Port self.isConnected = False print 'Going To connect' self.connectToMother() def connectToMotherShip(self): reactor.connectTCP(self.myHost, self.myPort, self) reactor.run() def buildProtocol(self, addr): p = ClientFactory.buildProtocol(self, addr) self.connectedProtocol = p return p def sendMessage(self, Message): self.connectedProtocol.SendMessage(Message) def reactOnReceive(self, String): #here i parse the received string for different requests after converting it to json and update values of the operator pass class Operator: def __init__(self): self.myCommunicator = Communicator("192.168.1.12", 815) do other stuff ... I want an Operator Object that builds the Communicator. The communicator updates values of the Operator asynchronously if it receives messages. Each 10 secs the Operator will initiate a Communicator.sendMessage to send a status information to the server. Two questions: 1) Is it possible to solve my task like this with twisted and if so, how do I get my Communicator to not block my Operator? 2) If twisted is not the easiest way for this approach, do i have alternatives? I want at least something event driven. I also read about the asyncore packages, but people often recommend twisted over asyncore. I simply want to send and receive message with a "threaded" communicator class, while doing other stuff in my Operator. Thanks a lot for your help, Toni

2 1

[Twisted-Python] Difficulty restarting/shutting down a twisted server - ports not closing
by Paul Sajna 12 Jun '13

12 Jun '13

I'm having a bit of trouble setting up an ssh-type server with twisted. When I shutdown or restart the server, the port remains open. Please have a look at my stackoverflow question. Thank you. http://stackoverflow.com/questions/16379504/port-object-has-no-attribute-so…

6 12

[Twisted-Python] HybridCluster raises $1.5m; we are hiring great Twisted devs
by Luke Marsden 10 Jun '13

10 Jun '13

1 0

[Twisted-Python] github, again
by Glyph 09 Jun '13

09 Jun '13

Hi Twisted developers, This weekend I had a discussion with many Twisted developers, both local to and visiting San Francisco. The topic came up of how to get more long-term contributors to participate more regularly in the project - particularly, doing code reviews, but also, developing and contributing to complex fixes and features that new contributors might not be able to tackle. One suggestion that almost everybody made immediately was: we should use Github for code reviews. In the past, I've heard this suggestion given mainly as a way to contribute more code, which does not appeal to me, since we are already swamped reviewing all the code that is currently being contributed. This time, however, it's been pitched as a way to get people to do more reviews, which I'm keenly interested in. Why would people do more reviews on Github? In a nutshell, it's a lot less work. Here are some reasons why: Instead of having to run 'force-builds' on the command line, or load a buildbot status page, Github has a way for a build system to report build success automatically, so you can see immediately within a pull request if the changes that it proposes are "good to merge". You can see this at work with Travis here: <https://github.com/twisted/klein/pull/11>. Originally I thought that this was a Travis-CI feature, but have since learned that this is apparently easy - trivial even - to hook up to Buildbot, since it's a simple HTTP API to invoke when a build completes, and there is even some existing buildbot infrastructure (deployed by Django, among others) to automate it. Instead of having to describe each patch location so that you can comment on it in a single message, if you want to put a comment on a particular part of a diff in a Github pull request, you can just click on it and start typing. In addition to the diff, it's reasonably easy to see the code in context on the web, which is faster than getting it into one's local development environment. If a review is successful, instead of having to have a local development environment, a committer can just hit the "approve" button and it's landed immediately. Instead of having to read through all history ever to see what's still relevant, a pull request will hide comments that address outdated diffs, allowing the change author to easily see what remains. These advantages are not comprehensive, but they're the more significant ones I remember from this discussion. A prerequisite for using Github for code reviews would be using Git rather than Subversion. Luckily there's not much work to do in this area, thanks to Tom's excellent work on the Git import and automatic Github mirror. As a bonus, by using Git instead of Subversion, we can start properly recording merge metadata. In this discussion, Alex Gaynor pointed out that Django has a hybrid workflow where they still use Trac for bug tracking, and Github for code review. We would therefore not need to come up with a way to migrate all of our tickets to Github issues (which seems, oddly, to be fairly unpopular even among those who like github a lot). What would need to happen in order for this to take place? We'd need some consensus (hence this message). We'd need to update the release process <http://twistedmatrix.com/trac/wiki/ReleaseProcess> and our development documentation <http://twistedmatrix.com/trac/wiki/BasicGuideToContributingCode> to refer to the relevant Git commands rather than Subversion commands. We'd need a redirect from <http://twistedmatrix.com/trac/browser/> and <http://twistedmatrix.com/trac/changeset/> that would point at <https://github.com/twisted/twisted> and <https://github.com/twisted/twisted/commits/> respectively. We'd need a Github web hook that could poke Buildbot to kick off commits. We'd need Buildbot integration to update Github pull requests with build results when builds complete. We'd need someone to install git rather than bzr on all the buildbots, and update the configuration of the builders to get the code from a git rather than Subversion URL. Someone will need to convert every open ticket in review to a pull request. I do anticipate some objections. One objection is that each of the above tasks is going to take some work. I am fairly confident that some of the people who have educated me here will step forward to volunteer to do it. Please reply to this message if you'd like to volunteer, saying what you'd like to volunteer to do. If not, then I guess that objection stands :-). Another is that this might not be worth that investment of effort. This is why it was nice to have Alex contributing to the discussion: Django did basically this very change (right down to the "Trac for tickets / Github for pull requests" distinction), at a much higher scale than we have, and as he described it the change was *well* worth it. Another objection is that Github is proprietary software, and an externally-maintained service that we'd be depending upon. One solution to the "proprietary software" thing is the availability of the MIT-licensed <http://gitlab.org>. It's a largely feature-complete clone of Github; if, for some reason, we need to migrate away from Github in a hurry, it will be relatively painless to set up Gitlab instead, and the fact that Git is a DVCS means every contributor will serve as a backup. The main reason I would not suggest just deploying it is that it creates another sticky infrastructure-management problem, and while Braid is great, I'd prefer to avoid creating more work in that area. Github also has APIs for literally all of their features, so we can create a backup script. (Also worth noting: Gitlab is an open-source competitor to Github, but they still trust Github enough to <https://github.com/gitlabhq/> host their own development there.) Finally, my own minor concern: Github has no notion of a "code review" as a unit of work. A pull request is just "open" until it is "closed". Closing pull requests to request changes would be jarring to the cultural norms associated with Github's UI. All the github users I've spoken with, even those who follow processes which are effectively identical to Twisted's, have assured me that this is not really an issue. A code review is "accepted" when you merge it; it's "rejected" if the pull request is still open but has some comments on it. This will make porting over <http://twistedmatrix.com/highscores/> a bit challenging, but I think it would be worth letting that break for the time being. -glyph

16 23

[Twisted-Python] A bug of the DNSDatagramProtocol
by yuyan zhang 07 Jun '13

07 Jun '13

when i run the code: df = Deferred() name = Name('version.bind') axf = AXFRController(name,df) dns = DNSDatagramProtocol(axf) d = dns.query((ip,53),[Query('version.bind',TXT,CH)]) it raise the error : Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite why = selectable.doRead() --- <exception caught here> --- File "/usr/local/lib/python2.7/site-packages/twisted/internet/udp.py", line 149, in doRead self.protocol.datagramReceived(data, addr) File "/usr/local/lib/python2.7/site-packages/twisted/names/dns.py", line 1936, in datagramReceived self.controller.messageReceived(m, self, addr) exceptions.TypeError: messageReceived() takes exactly 3 arguments (4 given) Unhandled Error and the I set the DNSDatagramProtocol.datagramReceived's last row self.controller.messageReceived(m, self, addr) to self.controller.messageReceived(m, self) it'ok! And this is a bug????

2 1

[Twisted-Python] on contributions
by Glyph 05 Jun '13

05 Jun '13

A few times recently (most recently in my email on Github) I have mentioned that I am more interested in getting more people to review Twisted tickets than in getting more people to write new patches to Twisted. Based on some feedback that other contributors have sent me privately, I'm afraid that this may have given the wrong impression. It's not that I don't want people to write patches for Twisted. I do. I want everybody to write patches for Twisted all the time! It's also not that I don't appreciate the time and effort that went into those patches. I do. In fact, it is because I appreciate that time and effort that I have said I want to optimize for reviews. It bothers me a lot that someone, having gone through the effort to craft a patch to Twisted, might have to wait for weeks or months for a reviewer to even thank them for that patch, let alone do a full review and give them the feedback they need to get it landed. I am constantly looking for ways to give quicker, feedback and better experiences to new contributors. I think the tools Github offers might help, but we (the core Twisted team) will still need to think of better ways to motivate ourselves and to induct others into the reviewer community. (If this has inspired you to do a review, there are plenty waiting: <http://twistedmatrix.com/trac/report/15>.) So, please, go on contributing patches to Twisted; I thank you for your contribution and I thank you doubly for your patience. Thanks, -glyph

2 2

[Twisted-Python] Twisted on PyPI
by Donald Stufft 03 Jun '13

03 Jun '13

Hey there! Recently there has been a lot of work around packaging in Python. On my end of things I've been trying to clean up PyPI. Recently I've deployed a change that ensures that all *new* package names on PyPI conform to a strict set of rules. These rules are: - Must begin and end with an ASCII alphanumeric - Must contain only ASCII letters, digits, underscores, hyphens, and periods Now we are left with about 235 total projects out of ~31k who do not conform to these guidelines. Out of those 235 projects Twisted is one of the major ones (Not Twisted itself, but Twisted Web, Twisted Core, etc). Now the plan that we've come up with is to migrate any name that the only invalid characters are internal whitespace by replacing the whitespace with a -. This would Make Twisted Web Twisted-Web. This reflects already what is required to install Twisted Web via pip or easy_install. So I'm emailing you all to warn you about this upcoming change since Twisted is one of the major projects affected and also to see if the Twisted developers would prefer a different rename than Twisted-Web. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

5 12

Twisted June 2013