Mailman 3 January 2010 - Twisted

[Twisted-Python] lore2sphinx themeing
by Kevin Horn Jan. 5, 2010

Jan. 5, 2010

Thanks to a quick patch from Michael Thompson, the current output of the Sphinx conversion [1] now matches the color scheme and typographic style of the main twisted site. It's using the layout of the 'default' sphinx theme (previously was using the 'sphinxdoc' theme). I'd like to know what people out there want/expect the final theme of the Sphinx version of the Twisted docs to look like. The floor is open...please offer your suggestions! [1] http://twistedsphinx.funsize.net/

3 3

Re: [Twisted-Python] A pseudo-deferred class that can be canceled
by Terry Jones Jan. 5, 2010

Jan. 5, 2010

As a final followup before bed... I chose not to have a 'cancel' method on my class because that seemed misleading. You're not really canceling anything. You're just asking that a deferred that you got earlier be fired right now with a value of your choosing. So I made 2 methods, and named them 'callback' and 'errback'. The Subject above is reasonably accurate, it's like a pseudo-Deferred class. Everyone works with regular deferreds, as usual, but if you know you might want to "cancel" a … [View More]

1 0

Re: [Twisted-Python] A pseudo-deferred class that can be canceled
by Terry Jones Jan. 5, 2010

Jan. 5, 2010

>>>>> "Glyph" == Glyph Lefkowitz <glyph(a)twistedmatrix.com> writes: Glyph> I already lost you at the first sentence. :-) Sorry. Glyph> The class below never appears to use 'self._f' Oops, that should have been a self._f in the call method. Glyph> and ... Deferreds are things that fire, I don't see how the callable Glyph> (f) can fire. Can you rephrase your intent? Yes. I wanted a way to be able to call something (anything) that returns a deferred and … [View More]then, at some later point, to be able to decide that I no longer want the deferred to fire in the way the original function would have done it. Instead I want to fire it myself, to in some sense cancel the outstanding call (by ignoring its result and instead returning a result of my choosing, right now). E.g., I call twisted.web.client.getPage and get a deferred (d) back. I add a callback to d. If sometime later (perhaps just due to too much time elapsing, but perhaps for another reason - e.g., maybe I'm shutting down a service, who knows) I want to effectively cancel that deferred, what choice do I have? The deferred was made elsewhere (in HTTPClientFactory in this case), which is presumably going to fire it at some point. Somewhere else in my code I've now got a callback function that's going to get the result, though I now don't want that to happen. I could set a flag somewhere to indicate to my own callback(s) that it(they) should ignore any incoming result and/or fail, but that's pretty messy. Instead I'd like to just manually cancel the deferred myself - where by "cancel" I mean fire it with a value of my choosing. Most convenient, probably, is to call its errback with a TimeoutError of some kind (just as defer.timeout does) which I can just absorb or log etc. I can't do this though with the deferred I got back from getPage. So the (untested) class I posted sits in the middle. If the original deferred (from getPage, in our example) fires first, it passes (via chainDeferred) the result along to the deferred it gave me. If instead I trigger the deferred myself (by calling 'callback' or 'errback' *on the CancellableDeferred instance*) then it fires the deferred it gave me back and will later ignore the result from getPage (if any). Glyph> I'm glad you're thinking about this, because it is an *extremely* Glyph> thorny issue which I would really like to address one day. Many of Glyph> the issues you're talking about were brought up, and various Glyph> solutions suggested, then found problematic, then modified ad Glyph> nauseam on <http://twistedmatrix.com/trac/ticket/990>. If you can Glyph> read that discussion and make some sense of it, perhaps you can post Glyph> a recommendation there, or at least a summary of the discussion so Glyph> far so that I don't have to read the whole discussion again to Glyph> remember what I think should happen next? :) OK, I'll take a look (in my copious spare time, etc). >> I don't know why defer.Deferred.setTimeout is deprecated, but I guess >> it's partly to do with this control issue. Glyph> This is one of the reasons, but another major reason is that Glyph> 'setTimeout' does not belong as a method of Deferred. If we did Glyph> support cancellation somehow, the way to set a timeout would be to Glyph> do 'reactor.callLater(5.0, myDeferred.cancel)'. Deferred was Glyph> originally in twisted.python and it really should have remained Glyph> there, decoupled from twisted.internet. OK, I agree with that aim. Cancelation (if you want to call it that) using my class is, IMO, decoupled from normal deferreds. Under normal circumstances you don't want to cancel a deferred you've asked for. But if you might, you can get yourself an instance of CancellableDeferred and use it instead. It provides you with a way to cancel the deferred (by which I mean: manually trigger it yourself with a value of your choosing, and thereafter ignore anything that might eventually come back from whatever it was you originally called (and which it is presumably now too late to undo)). You can do the canceling via reactor.callLater, just as you describe, but you call a method on the CancellableDeferred instance, not on the deferred. That keeps the canceling functionality out of the deferred class. Is that any clearer? Sorry the code was untested - it was meant to be so simple that it couldn't possibly be wrong :-) Terry [View Less]

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Jan. 3, 2010

Jan. 3, 2010

1 0

Re: [Twisted-Python] Problem fetching page with getPage
by Terry Jones Jan. 2, 2010

Jan. 2, 2010

>>>>> "Glyph" == Glyph Lefkowitz <glyph(a)twistedmatrix.com> writes: Glyph> I'm glad that you tracked this down! Me too. Glyph> According to comments on <http://twistedmatrix.com/trac/ticket/886>, Glyph> this problem was addressed in the new HTTP client implementation. Glyph> Have you considered using the new twisted.web.client.Agent instead Glyph> of getPage? I hadn't looked at it, but now have. The _computeHostValue method looks very promising :-) … [View More]

1 0

Re: [Twisted-Python] Problem fetching page with getPage
by Terry Jones Jan. 2, 2010

Jan. 2, 2010

Hi Glyph Thanks for the reply. I just sent another mail in the thread. >>>>> "Glyph" == Glyph Lefkowitz <glyph(a)twistedmatrix.com> writes: Glyph> Well, I know this isn't terribly helpful, but "a bug in getPage" is Glyph> really the only thing that comes to mind. Or, some Glyph> legal-but-unusual behavior in getPage which triggers a bug on the Glyph> EC2 side of things. The error arose from a combination of things (signing a string that included a host:port … [View More]

1 0

Re: [Twisted-Python] Problem fetching page with getPage
by Terry Jones Jan. 2, 2010

Jan. 2, 2010

>>>>> "Steve" == ssteinerX@gmail com <ssteinerx(a)gmail.com> writes: Steve> On Jan 2, 2010, at 9:34 AM, Terry Jones wrote: >> In any case, it looks like the problem is not in the setup of the request. >> Can anyone offer a reason why httplib might be able to fetch the page >> whereas getPage receives an error? I'm stumped. Steve> Steve> I've had to debug things like this recently and I have two suggestions: Hi Steve Thanks for the helpful … [View More]

2 1

[Twisted-Python] Problem fetching page with getPage
by Terry Jones Jan. 2, 2010

Jan. 2, 2010

I've run into a problem fetching an HTTP page with t.w.client.getPage. It's not simple to make standalone code showing what's going wrong, but the following summarizes where I am and why I find this puzzling. After some setup, I have some a url path, and some headers I want to send. A summary: host = 'ec2.amazon.com' port = 443 path = '/?some=params&are=here&etc=etc' method = 'GET' data = '' headers = { 'some' : 'headers', 'Content-Length' : '0' } url = '… [View More]

4 3

[Twisted-Python] Fwd: Problem fetching page with getPage
by ssteinerX＠gmail.com Jan. 2, 2010

Jan. 2, 2010

Sorry... >> I had a situation where python's HTTPlib stuff was adding an Accept Encoding header >> that didn't put there, that I didn't put there,

1 0

[Twisted-Python] how manual run LoopingCall.f
by ploutosss ploutosss Jan. 1, 2010

Jan. 1, 2010

I use _threadedselect reactor. My waker its one line: eventmodule.post(eventmodule.Event(TWISTEDEVENT, iterateTwisted=func)) its send event, to pygame queue. My pygame loop: class PygameController(object): _tick = Tick() def _eventIterator(self): while True: if self._ticking: yield eventmodule.poll() &… [View More]nbsp; else: yield eventmodule.wait() def run(self): for event in self._eventIterator(): if event.type == NOEVENT: self.manager.post(self._tick) continue elif event.type == TWISTEDEVENT: event.iterateTwisted() if not reactor.running: break continue ... When I need to quickly perform some function, I set the parameter PygameController._ticking to True, and my manager call some function. Its work. When I need to perform some function with a time interval of more than one second, I use LoopingCall. Its work. When I need to perform some function with a time interval of less than one second, LoopingCall too slow. I try use my ticking, but for function is not executed too often, I used the following code: def some_function(self): looping_instance = self.looping call = looping_instance.call if call and call.time - call.seconds() <= 0: looping_instance() but I get this traceback when I stop looping (LoopingCall.stop()): File "/usr/lib64/python2.6/site-packages/twisted/internet/base.py", line 779, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/lib64/python2.6/site-packages/twisted/internet/task.py", line 196, in __call__ d.addErrback(eb) File "/usr/lib64/python2.6/site-packages/twisted/internet/defer.py", line 199, in addErrback errbackKeywords=kw) File "/usr/lib64/python2.6/site-packages/twisted/internet/defer.py", line 181, in addCallbacks self._runCallbacks() --- <exception caught here> --- File "/usr/lib64/python2.6/site-packages/twisted/internet/defer.py", line 323, in _runCallbacks self.result = callback(self.result, *args, **kw) File "/usr/lib64/python2.6/site-packages/twisted/internet/task.py", line 191, in eb d.errback(failure) exceptions.AttributeError: 'NoneType' object has no attribute 'errback' [View Less]

2 1

[Twisted-Python] Lore to Sphinx Conversion Progress Report 3
by Kevin Horn Jan. 1, 2010

Jan. 1, 2010

Hello again, and welcome to another update on the Twisted Lore to Sphinx conversion project. Current Sphinx results are in the usual place at: http://twistedsphinx.funsize.net/ Here's what's new: - The lore2sphinx conversion script is now up on bitbucket. You can find it here: http://bitbucket.org/khorn/lore2sphinx/ . Special thanks to Steve Steiner (ssteinerX) for helping out a bitbucket newbie. - lore2sphinx now reads input and output directories from a config file. The config file can … [View More]have one or more "profiles", each of which contains config info for a set of documentation. At present, the script just uses the default profile (set in the config file), but I'll add a command line switch to specify the profile Real Soon Now (tm), which will make it easy to use a single config file for multiple projects (e.g. twisted, nevow, mantissa, etc.) - lore2sphinx has also been refactored a bit, which solves a lot of whitespace handling issues. The Sphinx build now gives fewer than 100 errors (currently 65, started off in the 500-600 range)! Hooray! Still, not all of the output is fully correct, and some tags (notably tables) are still unhandled. - I still haven't started any work on a theme...I'll get to it eventually, unless someone else does first. Any volunteers? - Divmod docs: new config file support is a step in this direction, look for something soon... - PyOpenSSL docs: kind of a sideline to this so I haven't really moved forward on this yet, but I will eventually If anyone out there would like to help out, I can certainly use the assistance. Lore2sphinx could probably use some extra eyeballs and brains looking at it, thought the basic structure is pretty much laid out. Any ideas/comments/criticisms welcome. If you're intimidated by the idea of working on the actual lore2sphinx conversion script (or just don't want to), here's some other stuff that needs doing: - work on a twisted (or Divmod) Sphinx theme which at least mostly matches the website (or the Divmod website) - identify branches which have lore patches in them - identify tickets which propose to fix/modify Lore functionality - identify wiki pages that need to be changed, and the necessary changes - make suggestions on the overall conversion process, as documented in the proposal. I'm sure I'll manage to get all this done eventually, but with more people, it will get done sooner. And sooner is better. :) If you'd like to help out with one (or more) of these tasks, just give a shout here, or email me off-list. Thanks! Kevin Horn [View Less]

3 4