Mailman 3 November 2008 - Twisted

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 04 Nov '08

04 Nov '08

Hi Glyph Thanks for the further examples. I agree that the classmethod approach is both flexible and powerful. It's probably the most useful thing I've picked up out of thinking about and playing with all this. Terry

1 0

[Twisted-Python] October Sponsored Development
by Jean-Paul Calderone 03 Nov '08

03 Nov '08

Hey all, Just wanted to let everyone know that the summary of the latest round of sponsored Twisted development is up on the labs blog: http://labs.twistedmatrix.com/2008/11/summary-of-october-sponsored-twisted.… Jean-Paul

1 0

[Twisted-Python] RPython and Twisted
by Andrew Francis 03 Nov '08

03 Nov '08

Hi Folks: This is new to me so here goes..... I am taking my first baby steps with PyPy. I wish to start experimenting by re-writing an example (the programme I posted in RPtyhon and converting it to C. However I want to still use Twisted for networking. One approach I was thinking about was to somehow compile the program as a C extension library and somehow have it talk to Twisted - I would like to keep my code and Twisted in the same process. Am I on the right track? Any hints would be appreciated. If there is interest I can summarise my findings. Cheers, Andrew

2 1

[Twisted-Python] Persistent Queue with subscriber worker
by Daniel de la Cuesta 03 Nov '08

03 Nov '08

3 3

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 03 Nov '08

03 Nov '08

>>>>> "Esteve" == Esteve Fernandez <esteve(a)sindominio.net> writes: Esteve> Alternatively a function would suffice: [snip] Esteve> def FooFactory(value): Esteve> return aFuncReturningADeferred(value).addCallback(SimpleFoo) Right. That's JP's solution (though he didn't bother wrapping it in a func): pass the result of the Deferred to the constructor of the class. But I like your __new__ approach better, as it puts the logic for creating deferreds, adding callbacks to them, doing other stuff etc., into the class itself. It's in __new__ instead of __init__, but that seems perfect to me. My approach, like all of them (and necessarily so) was two-phase, but my second phase required the calling of __instantiate__. By using __new__ and __init__ instead of my __init__ and __instantiate__, you moved the phases back and just hitch-hike on Python's existing machinery. That's why I like that approach, plus, as I keep saying, it keeps the Deferred logic inside the class where it belongs. T

2 1

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 03 Nov '08

03 Nov '08

>>>>> "Esteve" == Esteve Fernandez <esteve(a)sindominio.net> writes: Esteve> I'm going to jump in. How about this: That's nice. But you've coupled the args to aFuncReturningADeferred and what it's Deferred ends up returning to the args for __init__ of the class. I think you meant this: from twisted.internet import defer, reactor def aFuncReturningADeferred(value): d = defer.Deferred() reactor.callLater(5, d.callback, value[::-1]) return d class Foo(object): def __new__(cls, *args, **kw): def cb(x): obj = object.__new__(cls) obj.__init__(*args, **kw) return obj return aFuncReturningADeferred("Some value").addCallback(cb) def __init__(self, value): self.value = value def printFoo(obj): print obj.value reactor.stop() d = Foo("Some other value") d.addCallback(printFoo) reactor.run() I.e., __new__ can do whatever it likes with preparing args for aFuncReturningADeferred (including using args and kw), and all of the args to __new__ are passed to __init__, as the caller would expect from a normal class. You're passing the return result of the Deferred as the single arg to __init__. Anyway, I like it more than my solution. Terry

2 2

[Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 03 Nov '08

03 Nov '08

I just posted a blog article with title as above: http://bit.ly/1whZUK Very briefly, I wrote a metaclass to allow you to write classes whose __init__ method uses deferreds. Your __init__ can create deferreds, call functions that return deferreds, and of course return a deferred itself (that's the whole point). Your class instance wont be available until after the deferred your __init__ returns has fired. You use it like this: from txDeferredInitMeta import TxDeferredInitMeta class MyClass(object): __metaclass__ = TxDeferredInitMeta def __init__(self): d = aFuncReturningADeferred() return d def cb((instance, result)): # instance is an instance of MyClass # result is from the callback chain of aFuncReturningADeferred pass d = MyClass() d.__instantiate__() d.addCallback(cb) Metaclass code & test suite at http://foss.fluidinfo.com/txDeferredInitMeta.zip For more details on how it works see http://bit.ly/1whZUK I'll be happy to explain why this is useful if anyone cares. Or maybe there's a better way to do this that I don't know about. I always have the feeling that I know just the barest amount about what's out there in the Twisted codebase, and that every problem I run into must have already been encountered by many and solved in some insanely elegant and general way. In any case, I had fun. Terry

4 3

Re: [Twisted-Python] Passing arguments while running a twistd application
by Terry Jones 03 Nov '08

03 Nov '08

>>>>> "Adriano" == Adriano Marques <py.adriano(a)gmail.com> writes: Adriano> I have a twisted application, which formely was run with command Adriano> line arguments which were used during runtime, eg: I was passing Adriano> the path to the log file as argument. Then, I decided to make my Adriano> app run through twistd as a tac file but then I'm not able to pass Adriano> command line arguments anymore. Is there any way to overcome this Adriano> issue? I also ran into this, and asked for some way to pass args but got told (paraphrasing, from memory) that that's not the way to do it... The way to do it seems to be to convert your application to be a Twisted plugin. It's not too hard. Then you make sure there's a twisted/plugins dir accessible from your path (with no __init__.py files in/under it). Then twistd will find your plugin (just run twistd with no options and you should see your plugin name listed at the bottom among the commands). Then you can add options processing by subclassing the Options class in twisted.python.usage. Sorry to be so terse - no time. The above does work though, and seems to be the Twisted way. Rough (untested) guideline code below. Terry --- from zope.interface import implements from twisted.python import usage from twisted.plugin import IPlugin from twisted.application.service import IServiceMaker from ???? import myservice class Options(usage.Options): optFlags = [ ['verbose', None, 'If True, be verbose.'], ] optParameters = [ ['port', 'p', '0', 'The port number to listen on.'], ] class MyServiceMaker(object): implements(IServiceMaker, IPlugin) # The tapname will appear in the output of twistd when run with no args. # Use twistd -n my-plugin-name # to invoke it. tapname = 'my-plugin-name' description = 'My snazzy service.' options = Options def makeService(self, options): return myservice( options['port'], options['verbose']) serviceMaker = MyServiceMaker()

3 2

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 03 Nov '08

03 Nov '08

Hi Tristan >>>>> "Tristan" == Tristan Seligmann <mithrandi(a)mithrandi.net> writes: Tristan> You can, however, encapsulate it into a classmethod or free Tristan> function that you call to get an instance of MyClass. This is a Tristan> pattern often seen when there are various ways to build an Tristan> instance of some class; you'll have a very basic __init__, and Tristan> then a bunch of factory functions or classmethods that do more Tristan> complicated work before passing the results in to the class's Tristan> constructor. OK, thanks. It feels to me (please correct me if I'm wrong) that the main reason you'd want to follow this pattern is because you can't deal with Deferreds in an __init__ method. If you weren't dealing with Deferreds, you could just pass args to __init__ and have it figure out what to do. While it might be simpler to do as you describe, and have various external factories, it feels (to me anyway) like you're taking behavior that's class-dependent and putting it outside the class. __init__ can't return a result, presumably because if it has something it wants to remember, or give you access to, it can put it on self, provide you with a method, etc. But in the Twisted world, I think it makes sense that there is some provision for what's essentially a two-phase __init__. I quite like my approach, because the programmer wanting to write "normal" Twisted code will likely find it natural that __init__ could work with Deferreds, including returning one. To me that feels very natural. Having said that, I don't particularly like the fact that there's a special-case metaclass creating my classes, even though I wrote the code myself :-) >> BTW, I sometimes have the feeling that Deferreds are like how M$ >> characterizes the GPL. Viral. Once you call anything creating a Deferred, >> you're hooked - whatcha gonna do with that Deferred boy? Take one sip of >> the Deferred Kool Aid and the next thing you know you're on a drip. Tristan> Just wait until you start trying to combine iterators and deferreds ;) Oh yeah... I tried that too :-) http://www.twistedmatrix.com/pipermail/twisted-python/2008-June/017984.html Comments on that code very welcome too! Terry

1 0

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 03 Nov '08

03 Nov '08

Hi JP > I usually solve this kind of problem like this: > > d = aFuncReturningADeferred() > d.addCallback(MyClass) > d.addCallback(cb) See? I knew there'd be a 3-line solution somewhere :-) That's very cute, I'd never have thought of that... Python is so nice. My (weak?) response is that your solution isn't quite so general. What if I want __init__ to generate the deferred based on its args, or on the result of other functions called based on its args? Yes, I can do all that work before using your lovely d.addCallback(MyClass), but then the logic of what the class does is starting to leak out into the code that should be calling the class. And you can't encapsulate THAT code into another class, because its __init__ would in turn have to return a deferred. BTW, I sometimes have the feeling that Deferreds are like how M$ characterizes the GPL. Viral. Once you call anything creating a Deferred, you're hooked - whatcha gonna do with that Deferred boy? Take one sip of the Deferred Kool Aid and the next thing you know you're on a drip. > I'll not try to claim anything about the level of elegance, though. :) History will smile on you JP. > For less surprisiness, I'd suggest that you at least restrict the result > of the Deferred returned from __init__ to firing with None or self and > make the argument to the first callback on the Deferred returned by > MyClass() just be the new instance. Hmmm. OK. I think you're right. The caller has no business knowing what the Deferred returned in any case. And if the class itself wants the result it can add a callback and stick it on self. That's good, I had been hoping to return just the instance, but then it occurred to me to send back the result too. BTW, I started out trying to think of a way to just say x = MyClass(...) and have that result in the instance. But the only ways I could think of doing that were disgusting and broken (e.g., make a *function* called MyClass and use inlineCallbacks and some other class the metaclass would build). So the __instantiate__ call seemed like a big win, even though I don't like it much either. Thanks! Terry

2 1