Mailman 3 May 2014 - Twisted

[Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here
by HawkOwl May 7, 2014

May 7, 2014

Hi everyone, I’m sure that some of you have been following the past seven or so weeks of Twisted 14.0 release shenanigans, and this email hopes to explain what went wrong, what we can do better next time, and where we can go from here. Problem 1: Twisted 14.0.0pre1 had a regression. This was not noticed in the prerelease stage because it was not marked as a regression, where the RM does a check for open regressions on the milestone. What we can do better next time: Tickets that are … [View More]regressions need to be marked as regressions and applied to the release milestone. If you think it might be a regression - even slightly - mark it as such, and comment that you are not sure. It’s easier to find the ticket later and decide it is not actually a regression than have to abort a release because it’s come up after a prerelease. Problem 2: The fix for the regression was not merged into pre1, the release was rerolled from trunk. This meant some pyOpenSSL and TLS improvements got into the 14.0 release from pre2 onwards, but introduced new regressions. What we can do better next time: Do not reroll from trunk to get bug fixes - merge them into the release branch. Problem 3: The fixes for the regressions were finished after some delay, since the fixes had to be written and reviewed. This introduced delays into the 14.0 release cycle. What we can do better next time: Rather than fix regressions introduced, the ticket that introduced them should be reverted. Problem 4: The fixes for the regressions did not merge cleanly with the release branch. Some 35+ tickets were merged between pre1 and the release of the regression fix into trunk. What we can do better next time: Bug fixes should be based off the release branch, not trunk. This reduces the likelihood of code churn or unknown dependencies causing problems during the merge. Problem 5: There was mixed communication whether one of the regression fixes was to be introduced in 14.0 or in a bug fix release (14.0.1). What we can do better: If a fix is intended for merging in to a prerelease, it should be raised on the mailing list, so that there is more visibility for its intentions. Problem 6: I personally made several mistakes along the way - from screwing up svn merges to interpreting the “abort the release and incorporate the bugfix” to apply the initial regression fix. Since the TLS changes were topical, I decided that having them out ASAP would be better than not. What we can do better: Improved docs/automation to reduce the margin for RM error, and better automation to make a new release to get out important features really easy. These are the major problems which I have identified - I’m sure there’s plenty more, and I would like people to list them if I have not - even if they make me look like an idiot ;). We can learn from it, I’m sure. So, this leaves where to from now. I see a few options, with my estimates for work and risk that it’ll explode: 1 - Most work, high risk - Work on making the regression fixes merge cleanly with 14.0.0pre5. This is big-ish task with room for error, since there was some underlying code churn. 2 - Some work, medium risk - Release 14.0.0pre5 as 14.0 final, and I (or another RM if I’m no longer trusted ;) ) initiate the 14.1 release immediately. 3 - Least work, highish risk - Scrap 14.0, begin the 14.1 release immediately. since 14.0 tags become 14.1 tags, and we just have to hope that there’s no regressions in the 39 tickets fixed between pre1 and now. This may introduce issues (since 14.0 is an un-release, and there are questions about what this does to our deprecation windows). If I am to be honest, I much prefer option #3, but I would like opinions from other developers, before I go causing more problems than I already have :) Regards, HawkOwl [View Less]

5 6

[Twisted-Python] qt4reactor on PySide (Was: Serial interface for software program)
by anatoly techtonik May 7, 2014

May 7, 2014

On Mon, May 5, 2014 at 10:06 PM, <exarkun(a)twistedmatrix.com> wrote: > On 06:49 pm, sank.daniel(a)gmail.com wrote: >>> >>> There's a 3rd party event loop for Twisted that runs on top of Qt >>> (qt4reactor), which would >>> allow Twisted and Qt code to run in the same thread. >> >> Speaking of which, why isn't that thing officially part of twisted? Super >> useful. > > The PyQt team insists that the PyQt license is … [View More]

2 1

[Twisted-Python] ReactorBuilder doesn't support external reactors ticket reminder (Was: Serial interface for software program)
by anatoly techtonik May 7, 2014

May 7, 2014

On Tue, May 6, 2014 at 4:02 AM, Itamar Turner-Trauring <itamar(a)itamarst.org> wrote: > On 05/05/2014 03:06 PM, exarkun(a)twistedmatrix.com wrote: >> >> However, I'm not even sure if the GUI reactors *should* be distributed as >> part of the main Twisted package. They are already first-class reactors via >> the plugin system. Independent releases let them track changes in the GUI >> library they're each concerned with more quickly than they could if they … [View More]

1 0

[Twisted-Python] Serial interface for software program
by anatoly techtonik May 7, 2014

May 7, 2014

Hello, network hackers, Recently I've got a Raspberry Pi and a friend showed me its serial interface and helped to setup it. I was stunned. What I saw - you open serial terminal, and you jump in into the middle of boot section. Just open a terminal and you see what's going on inside of R.Pi. Any time. That's was an intro. =) For a long time I wanted the same interface for debugging software programs. When I was hacking on Spyder I found that my human brain is too limited to squeeze details of … [View More]dynamic of interaction of objects at run-time. Spyder is written in Python with Qt, and it is a parallel application much like any Twisted app is. But I wanted to push the progress, I have to deal with complexity, so I badly needed to be able to draw a video of interaction at run-time. That was the problem I tried to solve. And for the first step I needed a way to connect to Spyder at run-time and see what's going on. I didn't know how to call that way before, but now I have a rather accurate wording - "a serial interface for an application". That is the background for the question I am trying to ask. What is the best way to implement such interface considering the following properties: 1. it needs to be simple (so that you can quickly create python script that reads the info) 2. it needs to be universal (so that script for one app worked for other) 3. it should be asynchronous (connected terminal should not degrade performance) 4. it needs to be reliable (at least detecting missing packets) 5. and cross-platform (and pure python) Why Twisted? For the first I think that this problem is actual for people who are dealing with complexity of dynamic and interconnected systems built with the help of Twisted. For the second I believe it can only be solved or analyzed by people skilled in interprocess and internetwork communications. One button test: 1. program behaves weird 2. user hits the button 3. terminal pops up 4. shows a lot of data for a running program I am interested to know. 1. how the button should discover the program network port seems cool, but what about two parallel programs? what about different programs that provide this interface? 2. how terminal receives the data i see it as line based - one line - one event, everything is text 3. how to make it fast TCP clearly won't here 4. how to send the data what if two threads (or parallel code) produce serial data? who should run the server that serves connected terminals, and how other thread will send message to this server? you can not run two TCP servers on the same port on the app side, so is there an alternative? how to detect missing data in a stream? how to avoid dealing with incomplete packets and low level network details? Thanks. I'd like to get back to hacking on Spyder one day, but this is a stumbling block for me every time I think about it or any program of compared complexity. -- anatoly t. [View Less]

6 8

[Twisted-Python] overly specific ImportError assert for python 3?
by Glyph May 5, 2014

May 5, 2014

cyli was working on <https://twistedmatrix.com/trac/ticket/7144> and encountered this peculiar failure on her branch (and not on trunk): <https://buildbot.twistedmatrix.com/builders/python-3.3-tests/builds/2212/st…>. The interesting bit of the failure is that in one case, the ImportError's message is "import of 'service_identity' halted; None in sys.modules" and on trunk it's apparently 'No module named service_identity'. The branch in question changes conch, which isn't even … [View More]

3 3

[Twisted-Python] Log level using Twistd
by Juan Antonio Ibañez Santorum May 4, 2014

May 4, 2014

Hello, I am trying to modify loglevel withou result. I configure my app logging inside the tac file as: logging.basicConfig(level=logging.ERROR, format="%(asctime)s --- %(name)s - %(levelname)s - %(message)s") logfile = DailyLogFile("gdumper.log", "/var/log") application.setComponent(ILogObserver, FileLogObserver(logfile).emit) logging.basicConfig call seems do not apply as I get both log.msg and log.err messages into the log. Is there any other way to set up log level? Regards

2 2

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com May 4, 2014

May 4, 2014

1 0

[Twisted-Python] Some comments regarding #5190 - ``RFC 6125 ("Service Identity") implementation´´
by exarkun＠twistedmatrix.com May 3, 2014

May 3, 2014

Hello, I've just noticed that the changeset for #5190 included some untested code. Specifically, there are no tests for the code which detects missing dependencies and emits warnings about them. I'd previously noticed that this code was broken but hadn't realized this was because it was untested. I don't think there's any disagreement whatsoever over Twisted's testing requirements. All code must have full line and branch coverage (as reported by the coverage.py tool). Developers, … [View More]

3 3

[Twisted-Python] adding endpoint plugins
by David Stainton May 2, 2014

May 2, 2014

Greetings, I wrote a simple IStreamClientEndpointStringParser for using the SOCKS5ClientEndpoint with Tor: https://gist.github.com/david415/26a4ed59078d2e27376f I modified a txsocksx code example for testing the new endpoint descriptor: https://gist.github.com/david415/7c6040117319cc3b0230 ...but I am not sure how to "register" it such that calls to clientFromString can return this endpoint when the descriptor prefix matches. This doc: https://twistedmatrix.com/documents/13.2.0/api/twisted.… [View More]

6 7

[Twisted-Python] pb objects unexpectedly change identity
by Daniel Sank May 2, 2014

May 2, 2014

Dear twisted users, I think I have found some surprising behavior in perspective broker. I define a subclass of pb.RemoteCache which has a method managed by a descriptor. The descriptor keeps track of the instances it manages in a set. When the RemoteCache is first received by the client, it accesses the descriptor-ified method inside setCopyableState. At that time the RemoteCache's id is a certain value X. Then later when the RemoteCache is notified of a change by the server side Cacheable, … [View More]

1 2

[Twisted-Python] Twisted 14.0.0pre3 Announcement
by HawkOwl May 1, 2014

May 1, 2014

Hi everyone, here’s a new Twisted 14.0 prerelease for you! Tarballs for this prerelease can be found at <http://twistedmatrix.com/Releases/pre/14.0.0pre3>, with the changelog at <http://twistedmatrix.com/Releases/pre/14.0.0pre3/NEWS.txt>. Changes from the previous prerelease include: - Making Twisted Agent do HTTPS hostname verification. For more information and a full list of changes, check the NEWS.txt file. Please download the tarballs and test them with your applications, … [View More]

6 15