Mailman 3 November 2008 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] regarding xml elements
by Raj kumar 02 Oct '13

02 Oct '13

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml file as a child of an existing element? Thanks in advance.. Raj.. Forgot the famous last words? Access your message archive online at http://in.messenger.yahoo.com/webmessengerpromo.php

9 14

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] An XMPP client with SSL support
by Pedro Sanchez 18 Sep '09

18 Sep '09

Hello, I have a sample XMPP client with the following structure: from twisted.words.protocols.jabber import client, jid from twisted.internet import reactor ... factory = client.basicClientFactory(myJid, myPasswd) reactor.connectTCP(host,5222,factory) reactor.run() It works well, I can connect to the XMPP server. But I would appreciate if some one could give me some hints on what's required to get the client to connect using a secure SSL link. Thanks, -- Pedro

3 3

[Twisted-Python] SQLAlchemy and Twisted
by alex clemesha 13 Feb '09

13 Feb '09

Hi, I didn't want to hijack the 'Twisted Presentation Materials..' thread, so I started this one ... First off, in the other thread, Glenn, you mentioned Storm. I have heard of Storm, but never got any deeper that noticing that it is still a work in progress (maybe things have settled down ?) If anyone has anything to say on how it might be a good choice with Twisted, please feel free to enlighten us! But, I hoping to solve some immediate problems with Twisted and SQLAlchemy, so ... >> >> * A main "App Server", that controls high level access with "twisted.cred" >> * Web frontend: "twisted.web2" >> * Data: some "twisted.enterprise.adbapi", moving to SQLAlchemy. > > Interesting; how are you handling asynchrony in SA? This is the issue that we're stilling unclear on, and I really would like to know of the best way of integrating Twisted with SQLAlchemy. Twisted devs / experts: Please, could you comment on the right way of using Twisted and SQLAlchemy together. Anyone: If anyone know of existing code that integrates Twisted and SQLAlchemy nicely, please share! RIght away I just went about the problem be wrapping each call to SQLAlchemy in a "deferToThread" inside my "DatabaseManager" class. At startup of my app, I make an instance of this class, and pass it to each "avatar" (in "requestAvatar", in my portal) Here is my Database Manager: (you will actually see that I 'turned off' the deferToThread wrapping for now): http://trac.knoboo.com/browser/branches/knoboo-sqlalchemy-take2/knoboo/data… but I really haven't convinced myself that this is the best way of doing things. Hell, it could be totally wrong, so if anyone has any advice, I would be very appreciative! > I'm aware of sAsync, > but haven't looked much into it. (I'm a step or two behind you, actively > using adbapi, and thinking about moving to SA when I get some breathing > room.) I've looked into SAsync as well, but it is not totally clear to me what the extra benefit of it is (again, anyone who has a good description, please tell) The main developer of sAsync use to post to this mailing list, but I haven't seen him post in a fairly long while. So, if others are thinking about using SQLAlchemy with Twisted, let's discuss it more. Thanks very much, Alex

5 5

Re: [Twisted-Python] What to do when a service fails to start, also, deferred and startService
by Terry Jones 21 Dec '08

21 Dec '08

Hi Glyph Thanks for the detailed reply. >>>>> "glyph" == glyph <glyph(a)divmod.com> writes: glyph> On 27 Nov, 05:16 pm, terry(a)jon.es wrote: glyph> For me, baroque and elaborate start-up dances are a code smell. glyph> Services should be as independent as possible. Of course, sometimes glyph> some kind of initialization conversation is unavoidable, but I do glyph> like to try to keep it as short as possible. I do too. Sometimes it takes (me at least) a few iterations before you see how best to do that. glyph> I think you're misunderstanding what a "service" is. The word is, glyph> perhaps, a bit to lofty for its humble job. A service is just an glyph> event notification mechanism that tells you when it's time to start glyph> up, and when it's time to shut down. glyph> I can understand why it would be attractive to misunderstand in this glyph> way, though: IService doesn't do very much, you have requirements glyph> that it doesn't cover, and if it were the thing you understand it to glyph> be then it would cover those requirements. I'm sure that would be glyph> nicer for you :). glyph> This might seem a bit inconsistent, since stopService uses the glyph> return of a Deferred. However, this is for a very specific reason, glyph> not a generalized error-handling case: you may need to prevent the glyph> *rest* of the system (specifically, the reactor) from completely glyph> shutting down until you've managed to cleanly shut down whatever glyph> you're trying to shut down on potentially remote systems. glyph> startService has no such problem though; the service subsystem has glyph> told you "It's time to start up!" - its job is done, and the reactor glyph> isn't going away as part of service startup, so it's your glyph> responsibility as an application author to make sure your other glyph> dependencies are properly initialized. OK, this is helpful - I have been looking at it from a different point of view, as you've guessed. >> But if something does go wrong, you've got a failure propagating its way >> down a errback chain, eventually (unless an errback switches you back to >> the callback chain) popping out the end and causing the reactor to issue >> an Unhandled Error message. So you can't indicate that the service has >> failed to start by throwing, because the exception is going to pop >> harmlessly out the end of the deferred chain as a generic unhandled >> error and will not cause Twisted to know that the service couldn't >> start. glyph> The key question here is: indicate to whom? If you want to indicate glyph> it to some other object, well, try:except: or addErrback and call a glyph> method on that object. Nothing magic about it. I have code written as a Twisted plugin. So I have a class implementing IServiceMaker and IPlugin, and I create an instance of that class which gets found when I invoke twistd from the command line. So in my case I want to indicate to twistd that the service that my class creates a makeService method to create, but which I do not set in motion, has failed to start and that twistd should exit, or do something other than cheerfully tell me that there's been an Unhandled Error. Does that make more sense? Sorry, I should have said I was using twistd. glyph> In what way would you expect the service mechanism to "deal with" glyph> returning a Deferred? Stop starting other services? Print out some glyph> different log message? I'm not sure what should happen. I'm sitting at the command line, I've asked twistd to start something for me, there's clearly been a problem doing so (and this doesn't have to be baroque, maybe I just couldn't listen on a specific port I wanted, or maybe my code somehow raised an Exception), but I don't seem to have a mechanism for having twistd take any notice at all. I'm just talking about the case where startService calls something that returns a deferred and there's an Exception that comes back down the Deferred chain as a failure. I suppose if startService raises an Exception itself directly, something else happens - maybe twistd exits. glyph> IService is a very, very simple interface. If you want to respond glyph> to failures from startService (deferred failures, exceptions, or glyph> whatever else) in a useful way, then you can write your own glyph> implementation of it which manages startup order, keeps track of glyph> dependencies, and maintains a state machine that handles stopService glyph> appropriately if called in mid- startup. glyph> I don't think that having to implement an interface with 6 methods glyph> on it could be considered "cruel and unusual". If you think so you glyph> may want to investigate options other than Twisted: you will glyph> frequently be expected to implement interfaces with methods on them glyph> ;-). :-) glyph> There's no need to "track down and subclass" lots of things. Your glyph> IService wants the things that it contains to have a richer glyph> interface which allows for error handling, dependencies, and glyph> propagation, so simply write a single wrapper for simpler IService glyph> objects that expands the interface to do the other things that glyph> you're interested in. In the case of a service being started by twistd, it doesn't seem as simple as you describe, but maybe that's my lack of understanding again. I can easily subclass IService, but something else is calling the startService method of that subclass. And that thing, whatever it is, is not expecting me to return a deferred. So if my startService has for some reason got its hands on a deferred, it can't simply hand it back to its caller and have something (twistd in my case) see that an error occurred. It does feel like I have to track down what this something else might be. Either working from my IServiceMaker implementation or working from /usr/bin/twistd to find where startService is not trivial (you guys wrote it, I'm sure the logic is all much clearer to you). After looking through a few files I wind up at twisted/application/app.py, which has a startApplication function that calls service.IService(application).startService(). So I guess that's what is calling my startService. So I could make my own startApplication function, but I then have the same problem, I wind up with a deferred on my hands and my caller is not expecting me to return it. Plus, the startApplication function sits at the top level of twisted/application/app.py, so I have to find whatever is calling that. That seems to be twisted/scripts/_twistd_unix.py, which imports twisted.application.app and has a top-level startApplication that calls app.startApplication. But who is calling that? Looks like twisted/scripts/twistd.py is, and that's called by /usr/bin/twistd. So should I write my own twistd? All this doesn't seem to be a matter of simple subclassing. Plus, I can't just go in and start editing the top-level functions in twisted/application/app.py and twisted/scripts/_twistd_unix.py or code that imports app, etc. Sorry for so many questions - I really don't know if I'm missing something simple here. I do enjoy digging into all this, and I appreciate your apparently limitless patience. I wish I knew it all better. Twisted is complex and it's a pretty good bet that anything you think of or run into as a n00b has been thought of or encountered before, and that whatever way you think of to solve it will probably be non-optimal, or plain wrong, or in ignorance of a solution someone much more experienced has already implemented, or... etc. Hence my many questions. glyph> This all strikes me as totally straightforward and easy, and I don't glyph> think I'm any kind of super-genius for being able to write a few glyph> Python classes that call a few simple start/stop methods in the glyph> order that I want them to run in :). I should have mentioned that I want to use twistd. In fact I have something like a process pool running in one service and I talk to it from another machine. I say "hey, process pool, start me up the following service (a twistd service)" and I would then like to know if that service started, and if not then why not. So having twistd fail or report an error if it can't start a service would be useful. glyph> Doing either of those things would definitely be wrong. There's no glyph> reason to sys.exit or reactor.stop if your application can't start glyph> up, unless your management system specifically calls for such a glyph> thing. Maybe this is a case where it's (semi-)justified. At least if I called sys.exit the twistd process would go away, instead of sitting there acting as though nothing's wrong :-) I can also, of course, try interacting with the service I think I just started on the remote machine, and if I can't then I can tell the original process pool to kill the twistd process. But that seems a pretty roundabout alternative to just having twistd notice that something went awry when calling startService. glyph> In the future, even the Twisted plugin code might be starting some glyph> things in addition to your application. As I mentioned above, a glyph> good reason to do that is to perform diagnostics on failed startups glyph> :). I assume you really mean "startups" and not "services". In which case, I'm 100% sure there's something funny here, but I can't figure it out. I'd love to know, and I'm smiling broadly in any case. Too bad email loses so much humor..... we can always try though. Thanks again, Terry

2 1

[Twisted-Python] Syslog Listener UDP
by Marc Byrd 16 Dec '08

16 Dec '08

Hi, I would like to build a syslog(udp port 514) listener in twisted. The goal is to make it compatible with syslog and syslog-ng, which may be configured on a "client" to send log entries (as they occur) via udp (fire and forget) to a remote server (my twisted server). Is there already something out there that would do this or that would be a good start? Thanks, Marc

4 4

Re: [Twisted-Python] A Python metaclass for Twisted allowing __init__ to return a Deferred
by Terry Jones 06 Dec '08

06 Dec '08

>>>>> "glyph" == glyph <glyph(a)divmod.com> writes: glyph> On 02:33 am, terry(a)jon.es wrote: >> Am I missing something here? glyph> As a matter of fact, yes :). OK, thanks, I get it. That should have been obvious. BTW, I spent an interesting morning getting my head around *exactly* how inlineCallbacks does its thing... No summary of my thoughts could do it justice. I wonder how many people on the planet have been down that rabbit hole. Was the Twisted project always called Twisted? I was thinking it's a really good thing you picked that name - cause you definitely would have had to rename it :-) I hope people don't mind my peanut gallery commentary. We're all in this together, after all. A little gallows humor occasionally seems in order. Thanks again, Terry

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 30 Nov '08

30 Nov '08

1 0

[Twisted-Python] What to do when a service fails to start, also, deferred and startService
by Terry Jones 28 Nov '08

28 Nov '08

I've another of my pesky beginner questions. Note that this subject is somewhat covered in the thread started by Matt Goodall in Jan 2006: http://twistedmatrix.com/pipermail/twisted-python/2006-January/012380.html I imagine that it must be common that people write services that don't just simply launch things listening on sockets, but instead need to do a couple of things, one after another, in order to get going and to be ready to provide their service (or multiservice). If you do need to write something like that, it seems the chances are pretty high you're going to be calling code somewhere along the way that returns a deferred. And because the twisted/application/service.py code that calls startService doesn't handle deferreds being returned, this creates a real problem. At least as far as I understand things - which, as usual, may not be very far. If nothing goes wrong with the deferreds that startService is creating (via whatever its calling), then you'll probably get away with things even though your service will not really be up until after the deferreds fire, which can be some time after the code calling startService gets its deferred back (and ignores it). But if something does go wrong, you've got a failure propagating its way down a errback chain, eventually (unless an errback switches you back to the callback chain) popping out the end and causing the reactor to issue an Unhandled Error message. So you can't indicate that the service has failed to start by throwing, because the exception is going to pop harmlessly out the end of the deferred chain as a generic unhandled error and will not cause Twisted to know that the service couldn't start. This all feels quite ironic :-) Twisted leads you coyly into the dark and powerful world of working with and heavily depending on Deferreds. But then, right when you expect it to be there for you, covering your back, it throws up its hands as if to say "What!!? You expect me to deal with you returning a Deferred? You gotta be kidding, sucker." I could follow Moof's approach (last poster in the above thread), but that seems to just pass the problem on to a higher level, where something else is calling startService (or something earlier) and so on up until we reach the topmost point at which something is not allowing/expecting a deferred to come back. Should I track down and subclass all these things? That would seem cruel and unusual punishment to the faithful Deferred user, having to go in and subclass core classes because they don't deal with Deferreds. I could do something dramatic, like call reactor.stop or sys.exit in my errback chain, but those seem completely wrong. Apart from the (remote?) possibility that something other than Twisted plugin code is trying to start my service, it's also anachronistic because it will happen at some unpredictable time after startService has gotten back (and ignored) the deferred and Twisted has moved on (perhaps even to start other services). Terry

2 1