Mailman 3 May 2006 - Twisted

[Twisted-Python] Twisted + SQLAlchemy
by Ed Suominen May 5, 2006

May 5, 2006

I've received an inquiry about how I'm using SQLAlchemy under Twisted. I thought I'd provide the answer so that others can benefit from it as well: http://foss.eepatents.com/pNetworkX/browser/trunk/pnetworkx/database.py Note particularly the "transact" function, which is used as a decorator. Thanks to Valentino Volonghi (aka Dialtone) for the fine idea of using a decorator to set up a nonblocking transaction (using deferToThread) for whatever functions are so designated. Here's a usage example, with various database-accessing methods that are decorated with "@transact": http://foss.eepatents.com/pNetworkX/browser/trunk/pnetworkx/edges.py Best regards, Ed Suominen

1 0

[Twisted-Python] protocol factory
by aerodew May 5, 2006

May 5, 2006

i think Factory is not to the point,just concerned with its concept,factory should only take care of producing proper protocol instance.i can't see any clue of protocol context in the defintion of factory.so the boudary of factory is loom,i don't like it.or on other hand,we can it can be categoried into abstract factroy,then i proprose following refraction. class ProtoAbsFactory: implements(interfaces.IProtocolFactory) protocol=None context=None numProtos=0 def buildProtocol(self): pass def buildContext(self): pass class ProtoContext: def doStart(self): pass def doStop(self): pass class ClientProtoContext(ProtoContext): def startedCnnecting(self): pass def clientConnectionFailed(self): pass def clientConnectionLost(self): pass class ReconnectingClientProtoContext(ClientProtoContext): def clientConnectoionFailed(self,connector,reason): self.connector=connector self.retry() def clientConnectionLost(self,connector,reason): self.connector=connector self.retry() def retry(self): self.connector.connect() by introducing context,we make solution for some situation very simple,just like http,since it is a stateless ,we have to bring in session to store state information during the communication,here by refining the context ,we can easily achieve it

1 0

[Twisted-Python] No-op TimerService on Windows?
by Eric Faurot May 5, 2006

May 5, 2006

Hi, In the process of exploring my options for deploying twisted applications on Windows, I noticed that when running the application, _twistw (line 43) also starts a TimerService around a no-op: app.startApplication(internet.TimerService(0.1, lambda:None), 0) I suppose there is very good reason to do so, but I do not see it. Can somebody enlighten me? Eric.

3 3

[Twisted-Python] Multiple main loops
by David Pratt May 4, 2006

May 4, 2006

Hi. I am hoping you can clarify for me whether it is possible to run multiple independent main loops in Twisted. It is my understanding that this would mean separate independent reactors. Googling, I found this patch but after looking into the repository, it was never committed. I would appreciate any comments on the patch or attempts people have made on the integration of a second twisted loop. At the present time I am using twisted with wxpython with wxpython as main loop and also running a selectreactor instance. Here there are two separate loops, interoperating and operating independently, however they are not both twisted loops. Zope currently runs using a reactor instance for each zope instance but I would like to consider a separate independent loop for each client to synchronize to a server without paying a penalty. Zope's ZEO (basically a synchronization server) currently uses a simple zrpc protocol and runs independently but interoperates with a separate ansyncore main loop. The current system works well for the exception that it would be great to simplify this as well as improve security using twisted's built in capabilities. Many thanks. Regards David Index: twisted/internet/default.py =================================================================== RCS file: /cvs/Twisted/twisted/internet/default.py,v retrieving revision 1.79 diff -u -r1.79 default.py --- twisted/internet/default.py 10 May 2003 12:52:37 -0000 1.79 +++ twisted/internet/default.py 11 May 2003 23:11:26 -0000 @@ -367,8 +367,6 @@ # global state for selector -reads = {} -writes = {} def win32select(r, w, e, timeout=None): @@ -403,13 +401,18 @@ __implements__ = (PosixReactorBase.__implements__, IReactorFDSet) + def __init__(self): + PosixReactorBase.__init__(self) + self.reads = {} + self.writes = {} + def _preenDescriptors(self): log.msg("Malformed file descriptor found. Preening lists.") - readers = reads.keys() - writers = writes.keys() - reads.clear() - writes.clear() - for selDict, selList in ((reads, readers), (writes, writers)): + readers = self.reads.keys() + writers = self.writes.keys() + self.reads.clear() + self.writes.clear() + for selDict, selList in ((self.reads, readers), (self.writes, writers)): for selectable in selList: try: select.select([selectable], [selectable], [selectable], 0) @@ -419,19 +422,19 @@ selDict[selectable] = 1 - def doSelect(self, timeout, - # Since this loop should really be as fast as possible, - # I'm caching these global attributes so the interpreter - # will hit them in the local namespace. - reads=reads, - writes=writes, - rhk=reads.has_key, - whk=writes.has_key): + def doSelect(self, timeout): """Run one iteration of the I/O monitor loop. This will run all selectables who had input or output readiness waiting for them. """ + # Since this loop should really be as fast as possible, + # I'm caching these global attributes so the interpreter + # will hit them in the local namespace. + reads=self.reads + writes=self.writes + rhk=reads.has_key + whk=writes.has_key while 1: try: r, w, ignored = _select(reads.keys(), @@ -502,33 +505,33 @@ def addReader(self, reader): """Add a FileDescriptor for notification of data available to read. """ - reads[reader] = 1 + self.reads[reader] = 1 def addWriter(self, writer): """Add a FileDescriptor for notification of data available to write. """ - writes[writer] = 1 + self.writes[writer] = 1 def removeReader(self, reader): """Remove a Selectable for notification of data available to read. """ - if reads.has_key(reader): - del reads[reader] + if self.reads.has_key(reader): + del self.reads[reader] def removeWriter(self, writer): """Remove a Selectable for notification of data available to write. """ - if writes.has_key(writer): - del writes[writer] + if self.writes.has_key(writer): + del self.writes[writer] def removeAll(self): """Remove all readers and writers, and return list of Selectables.""" - readers = reads.keys() + readers = self.reads.keys() for reader in readers: - if reads.has_key(reader): - del reads[reader] - if writes.has_key(reader): - del writes[reader] + if self.reads.has_key(reader): + del self.reads[reader] + if self.writes.has_key(reader): + del self.writes[reader] return readers

6 14

[Twisted-Python] Re: [Xapian-discuss] Twisted + Xapian WriteableDatabase leaves db_lock file on exit.
by jarrod roberson May 3, 2006

May 3, 2006

On 5/3/06, Olly Betts <olly(a)survex.com> wrote: > > It seems this has cropped up before - see Michael Salib's point "2." > here: > > http://article.gmane.org/gmane.comp.search.xapian.general/1387 > > I've not yet implemented WritableDatabase::close() though. > > Incidentally, the stale lockfile on exit issue goes away if you use > flint instead of quartz, though the database lock will still be held > while the process keeps running which could still be a problem. > > Cheers, > Olly > thanks for the hint about using Flint instead. I thought I was but I guess I didn't add it to my .profile using Flint it starts and stops and doesn't cause any problems now. a .close() method on WritableDatabase might be a good addition. Just so you can force the db closed when you think it might not be being de-referenced reliably. It seems to be the same type of issue listed in that link you posted. Twisted is probably holding on to the Service object and that is holding on to the db object. I KNOW I am not sharing db objects nor document objects since it is all single threaded and all self contained in that one index method.

1 0

[Twisted-Python] Re: [Xapian-discuss] Twisted + Xapian WriteableDatabase leaves db_lock file on exit.
by jarrod roberson May 3, 2006

May 3, 2006

On 5/3/06, Olly Betts <olly(a)survex.com> wrote: > > On Tue, May 02, 2006 at 11:25:25PM -0400, jarrod roberson wrote: > > xapian and xapian bindings 0.9.2 svn 6532 > > That's rather old - is this reproducible with 0.9.5? > > > It leaves the db_lock file everytime and I don't know what else it might > not > > be cleaning up correctly either. > > The db_lock file gets deleted when the C++ WritableDatabase destructor > is called. So if it's not deleted, then the destructor isn't being > called. > > If you can reproduce with the latest release in a simple example > (ideally without twisted being involved), I can try to investigate. > > Cheers, > Olly > yes, just upgraded and it does the same thing with 0.9.5 the problem is I can't reproduce it outside of Twisted. all my command line tools that I have written ( all in Python ) that use the exact same code exit and close properly.

1 1

[Twisted-Python] Re: [Xapian-discuss] Twisted + Xapian WriteableDatabase leaves db_lock file on exit.
by jarrod roberson May 3, 2006

May 3, 2006

On 5/3/06, Terry Jones <tcj25(a)cam.ac.uk> wrote: > > I'd like to know the answer, when you get it (assuming it's not just an > old > version of Xapian). > > nope, I installed the latest stable 0.9.5 and it does the same thing, doesn't close xapian correctly. :-)

1 0

[Twisted-Python] Run a function repeatedly at different intervals
by Khoa Nguyen May 3, 2006

May 3, 2006

Hi, I am new to Twisted Python. I would like to fire a function several times at different intervals. For example, at 1 second fire f(), at 2 second fire f() again, at 4 second fire f(), and so on... How do I do that? I looked at the doc and it has task.LoopingCall(second) but the time interval is fixed... Thanks, Khoa

2 1

Re: [Twisted-Python] Question on pamauth.py
by Terry Jones May 3, 2006

May 3, 2006

Hi Duncan, thanks for the reply. >>>>> "Duncan" == Duncan McGreggor <duncan.mcgreggor(a)gmail.com> writes: Duncan> The really great thing about cred is that it works the same Duncan> regardless of the underlying authentication mechanism. I suggest Duncan> writing a couple toy examples and plugging those into your twisted Duncan> app, just to get a feel for how things work. In the process you Duncan> will gain genuine knowledge (as opposed to "a feel"). OK, I'll try that. Duncan> You probably wouldn't ever need to use twisted.cred.pamauth. You would Duncan> use the twisted.cred.checkers.PluggableAuthenticationModulesChecker, Duncan> and that works just like, well, a cred checker :-) Duncan> You biggest problem is actually going to be getting PyPAM Duncan> working. As far as I know, and as far as tummy.com knows (the Duncan> original sponsors of PyPAM), there's been no release since 1999. I Duncan> toyed with the idea of using it at one point, but the amount of Duncan> work necessary in updating the python was too onerous. Perhaps you Duncan> have a stronger stomach than I :-) >From your reply and that of Phil Mayers (thanks Phil), I see that my mail wasn't really clear. I didn't really want to know about pamauth specifically (despite my subject), but about back end cred checkers (not twisted.cred itself) that I could use with twisted.cred. When I went looking for options on this (e.g., based on LDAP, /etc/passwd, PAM, other db approaches, etc), I was surprised to find myself reading about pamauth, that it was in the Twisted tarball I'd installed, but that I'd read no mention of it. So I wondered about it, and also whether there might be other back ends to drop in. Thanks anyway for the details on pamauth, I'm pretty sure I wont use it. Duncan> and it's up to you to decide what to do about persisting Duncan> authentication information. That was what I was meaning to ask: what do other people do about this (persisting authentication information)? Are there recognized best practices for what I described? Maybe the question is too basic! I expected someone might say "oh, just tell cred to use the XYZ python module". Duncan> I'm totally not being snotty when I say this (again): cred is Duncan> cred. One of the really great things about it is how easy it is to Duncan> do whatever you want. You can create interfaces that represent Duncan> groups, have a single realm and set/check interfaces, have Duncan> different realms for different levels of access, etc. You can have Duncan> your resources implement your group/role interfaces. There's all Duncan> kinds of stuff you can do, it really depends on your needs. Once Duncan> you are comfortable with it and have written a few working Duncan> toys/examples, you will see the potential. I think I see the potential, that's why I'm here bugging you all. I'm still trying to understand some things, and at least right now, to know if there are more standard components that I'm not aware of (that I can drop into the architecture provided by Twisted, and, in this particular case, cred). Duncan> Writing the code necessary to plug your twisted app into somebody Duncan> else's authentication system can be easier than doing it any other Duncan> way (given that you are working with open, sane standards). I've Duncan> actually written working cred code faster than other project Duncan> members have *configured* (GUI) the user management systems I was Duncan> writing against. I read that in your blog the other night :-) Duncan> Here's a great resource: Duncan> http://twistedmatrix.com/projects/core/documentation/howto/cred.html Yes, thanks, I'd read that a couple of times too. Duncan> And if you don't have the O'Reilly twisted book, do yourself a Duncan> favor and pick up a copy. Abe did a wonderful job explaining cred Duncan> very clearly in it. And the book is on its way. I ordered it after reading the review here http://tv.debian.net/articles/review-snakeball/ Anyway, thanks again for the reply. I'm trying to figure out how to make about half a dozen things hang together nicely, and I've (obviously) never used Twisted. I'm getting there. I'll take your advice about small examples. Regards, Terry

1 0

[Twisted-Python] status of IM
by Manlio Perillo May 3, 2006

May 3, 2006

Hi. IM unfortunately uses GTK 1.x. I would like to use twisted.words for chat but what is its actual status? I can try to write a version of IM that uses GTK 2.x, but if all is going to change... Thanks Manlio Perillo

1 0