Mailman 3 May 2010 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 30 May '10

30 May '10

1 0

[Twisted-Python] Shared resource manioulation example
by vitaly＠synapticvision.com 28 May '10

28 May '10

Hi, I'm looking for a Twisted based example how it should be properly done update/delete of global dictionary's SAME key:value pair from different functions all together. Or by another words, two functions are trying to update the same key:value pair of the same dictionary at the same time, than how to do it properly with Twisted? Thanks a lot.

7 10

[Twisted-Python] twisted + wx + pypubsub = progress bar update problem
by Gabriel Rossetti 28 May '10

28 May '10

Hello everyone, I am using wx (wxreactor) and pypubsub with twisted and I'm having some issues. I have a loop that updates values and I'm trying to show the user the progress. The loop uses pypubsub to update the progress bar but even if I explicitly refrech the progress bar widget it doesn't do it until the loop exits. I tried calling ProcessPendingEvents() but it doesn't solve the problem. Does anyone have an idea on how to solve this? Thank you, Gabriel

1 0

[Twisted-Python] can the reactor be told to stop processing events and to just queue them?
by Gabriel Rossetti 28 May '10

28 May '10

Hello everyone, is there a way to tell the reactor to just queue the events and not to process them for a specific protocol? I have two protocols running on my reactor, one that monitors the network and one for a usb device, I'd like to pause the event processing for the network side when certain msgs arrive on the usb protocol and then unpause it later on. I know I can create a queue and add the network msgs and process them later but I find it redundant to do so when twisted already has an event queue. Thank you, Gabriel

2 3

[Twisted-Python] Looking for Twisted programmers at Fluidinfo
by Terry Jones 28 May '10

28 May '10

Apologies for a job posting, but we're looking to hire Twisted people at Fluidinfo (http://fluidinfo.com) . We wrote FluidDB using Twisted from top to bottom. We love Twisted. In fact, we closed a funding deal a week ago and the company's first check is being written to the Twisted Software Foundation. We wrote and use txAMQP, txRDQ, Twisted interfaces to Avro, Zookeeper, and Amazon S3 & EC2. We give talks on Twisted (we have a couple in the upcoming EuroPycon) and I am personally addicted to the myriad slippery beauties of Twisted deferreds :-) So there you go. Want to work (preferably!) in New York in a cool startup with a team of hackers who love Twisted? Of course you do! See http://fluidinfo.com/jobs & google FluidDB or Fluidinfo for more. [We now return you to regular programming.] Terry

3 2

[Twisted-Python] major changes, release engineering, and learning cost
by Glyph Lefkowitz 28 May '10

28 May '10

The nice thing about Twisted's compatibility policy is that developers, and even users, very rarely have problems when installing a new version of Twisted. While this is a nice benefit, the current strategy of developing features in a compatible way does have a couple of costs, and I'd like to see if we can address them without giving up the benefit. I have a suggestion for a process tweak which would hopefully mitigate some of the difficulties which arise due to the compatibility policy. When we add a new feature that supersedes an older one, or fix a bug in Twisted that involves changing behavior, the developer fixing it has to come up with a new name. If we have several behavior-changing bugfixes in the same subsystem, that means that developers using Twisted may have to learn about 3 different symbol names. Since we tend to avoid just suffixing names with numbers (for good reason, I think), they won't have to learn Bicycle, Bicycle2, Bicycle3, they'll have to learn Bicycle, then Footcycle, and finally Velocipede, and somehow infer that Velocipede is the newest/best name that they should be using, by reading the (hopefully clear, concise) warnings that come out of their unit tests. This came up again recently on a ticket about URLPath, <http://twistedmatrix.com/trac/ticket/2625#comment:16>, where a contributor suggested that it would be better to make a whole new module because it's easier for external developers to learn about that then learn about an individual method change. This of course raises the question: if we're going to have a whole new URL class, shouldn't it fix the (numerous) *other* bugs that we know about in URLPath? Up until now the objection to doing things this way is that it results in gigantic branches which are intractable to review. That's a good objection, but it leaves us with a false dichotomy; reliable software and painless upgrades with a random scattershot of new features that are hard to understand, or coherent iterations of technology which can't be effectively reviewed, and therefore can't be effectively quality controlled. I propose that we get the best of both worlds by changing the way we do reviews slightly. Right now every code review needs to be done on an entire feature going to trunk, and _all_ of the code going to trunk needs to be reviewed at once. I suggest that instead, we create "integration branches" for sensible constellations of features, and have a two-stage review process. For example, let's say I want to work on making URLPath good. There are several tickets addressing it: <http://twistedmatrix.com/trac/ticket/2093> <http://twistedmatrix.com/trac/ticket/2094> <http://twistedmatrix.com/trac/ticket/2625> For the sake of argument, let's all pretend these are all deeply interrelated and involve changes to behavior of existing methods. I think that is sort of true of most of these, but it would be far too verbose to talk about *how*, and get bogged down in that discussion. First, I'd make an integration ticket, let's call it #ABCD, describing how these features are related and a brief outline of the new API I propose which resolves them. Then I'd create an integration branch from trunk, for that ticket. From the #ABCD branch, I'd create a branch for #2093, and put it up for review. The reviewer would review #2093 as usual, citing any test coverage issues, documentation issues, etc. After the usual review process, when I get an "OK to merge", I would merge #2093 *to the #ABCD branch*, not trunk. I would repeat this process for #2094 and #2625, merging each to the #ABCD branch as they passed review. Finally, I'd put #ABCD itself up for review. At this point the process would differ a bit. Reviewers would be able to assume, at this point, that the potentially large body of code in #ABCD had already been reviewed, that the test cases were good, the documentation was reasonably clear, and the logic made sense. This final review would be a quick sanity check, to make sure the tests still pass and that there are no conflicts. I would like to strongly emphasize that this point in the process would be an inappropriate time for reviewers to start arguing with each other over what is required for the branch to land, disputing the original specification, etc; this is just an opportunity to spot potential regressions before it lands. Each ticket review for a component of the larger feature should be an opportunity to draw attention to the direction of the larger feature development and prompt discussion. This *might* be an appropriate point to note that some other behavior-changing feature had been left out, though. In the case that there *were* conflicts, this would be an opportunity to review the conflict resolution itself. (We saw a nascent version of this approach on some stuff related to <http://twistedmatrix.com/trac/ticket/886> and it was hugely painful because nobody was really sure what the process was supposed to be. So let's not do it like that again.) So: thoughts? Does this make sense as a policy change for facilitating the development of major new features or consolidating behavior-changing fixes into easier-to-understand units?

6 13

[Twisted-Python] turning off chunked transfer-encoding
by Michael P. Soulier 25 May '10

25 May '10

Hi, I noticed that when using twisted.internet.http and returning a Content-type of application/json, the server responds with chunked transfer encoding. I'm debugging a client problem and I'd like to turn chunked transfer encoding off. Is this possible? Thanks, Mike -- Michael P. Soulier <msoulier(a)digitaltorque.ca> "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." --Albert Einstein

2 2

[Twisted-Python] Reactor Spinning?
by Moxie Marlinspike 24 May '10

24 May '10

Hey Twisted folks, I'm using the twisted.web framework for a high performance HTTP proxy server that very closely resembles the HTTP proxy server example that comes with the twisted package. Under heavy load, I occasionally run into a problem where the reactor appears to start spinning on an "empty" write. An strace of the process reveals these system calls: epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1 write(81, "", 0) = 0 gettimeofday({1272980323, 567613}, NULL) = 0 gettimeofday({1272980323, 567696}, NULL) = 0 epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1 write(81, "", 0) = 0 gettimeofday({1272980323, 567988}, NULL) = 0 gettimeofday({1272980323, 568071}, NULL) = 0 epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1 write(81, "", 0) = 0 gettimeofday({1272980323, 568501}, NULL) = 0 gettimeofday({1272980323, 568613}, NULL) = 0 epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1 write(81, "", 0) = 0 gettimeofday({1272980323, 568978}, NULL) = 0 gettimeofday({1272980323, 569084}, NULL) = 0 epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 6) = 1 write(81, "", 0) = 0 ...it seems to be waiting for write availability on a FD, trying to write an empty buffer, seeing that it wrote zero bytes, and so waiting for availability again. I've confirmed that when this occurs, whatever FD it is spinning on does correspond with a network socket connected to the proxy. I've seen this with both Twisted 8.2 and 9.0 running on Linux using both the epoll and select reactors. If anyone has any thoughts about this problem or has seen it before, I'd appreciate any insight that anyone might have. Thanks, - moxie -- http://www.thoughtcrime.org

5 10

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 23 May '10

23 May '10

1 0

[Twisted-Python] Test, plz ignore
by James Y Knight 21 May '10

21 May '10

test mail.

1 0