Mailman 3 May 2005 - Twisted

[Twisted-Python] Reactors and New Asynchronous APIs
by Patrick Mézard 05 May '05

05 May '05

Hello, I have several questions about twisted reactors and extending the twisted framework with new asynchronous APIs. 1- What is the status of the IOCP reactor? Is it stable? If it is, is there any reason for not selecting it as the default reactor for win32 NT platforms? 2- I am considering the integration of a custom database API (like Berkeley DB) in a twisted application. I could easily wrap the API in a C++ python module and access it synchronously from twisted, at least in a prototyping phase. Obviously, if the blocking calls are too long it would make the whole framework completely useless. Then, the calls could be made from twisted managed threads. Assuming the module code was written to minimize GIL contention it would probably greatly improve the situation. The most efficient solution would be to plug the asynchronous API directly into the reactor: Is it possible to extend reactors with new asynchronous API implemented in separate client modules? With my little knowledge of the twisted framework and quick glances at the selectreactor and iocpreactor implementation, I think the answer is no. The practical solution would be to duplicate the reactor code and extend it with the new API calls. I do not know if it would be easy or not to provide a pluggable asynchronous framework *at C module level*. Basically, the reactor is nothing else than a thread-safe event queue with a (post/get)-event API. If this abstraction can be implemented at C level, then plugging new asynchronous handlers would be clearly doable, _iocp.c is a good example for that. Now, I am not really knowledgeable (yet) about python C module coding, and I guess it is not exactly easy to bind C callbacks from one python C module into another using normal python code as glue. In the end, this is just another plugin/component system like COM to write so... Any thoughts? Patrick Mézard

3 2

[Twisted-Python] Closing Jabber connection
by Carolina Ardohain 04 May '05

04 May '05

I try to launch several jabber connections (BTW I use twisted 1.3). When the login is successful there's no problem, but whe the password is wrong I wanna to disconnect, I call connector.disconnect() but the event client.BasicAuthenticator.AUTH_FAILED_EVENT keep calling again and again. How I should close the connection so that event don't be called anymore? from twisted.protocols import jabber, xmlstream from twisted.protocols.jabber import client, jid from twisted.internet import reactor, defer class Connection: def __init__(self, who, myjid): self.jid = jid.JID(myjid) self.who = who def connect(self, passwd, port=5222): self.factory = client.basicClientFactory(self.jid, passwd) self.factory.addBootstrap(xmlstream.STREAM_AUTHD_EVENT, self.onAuthSuccess) self.factory.addBootstrap(client.BasicAuthenticator.AUTH_FAILED_EVENT, self.onAuthFailed) self.connector = reactor.connectTCP(self.jid.host, port, self.factory) def onAuthSuccess(self, stream): print "%s: Successful connection" % (self.who) self.stream = stream def onAuthFailed(self, stream): print "%s: Failed connection" % (self.who) self.connector.disconnect() c1 = Connection('1', 'client1@localhost/res1') c2 = Connection('2', 'client2@localhost/res1') c1.connect('client1') c2.connect('client22') reactor.run() The output is ('2' has wrong password) 1: Successful connection 2: Failed connection 2: Failed connection 2: Failed connection 2: Failed connection .... 2: Failed connection

11 24

[Twisted-Python] twisted.news
by Manlio Perillo 04 May '05

04 May '05

Regards and thanks for your work on Twisted. I have noted that twisted.news isn't installed with the last version of the installer for Windows. Why? Manlio Perillo

3 2

[Twisted-Python] Issue with stopListening
by Örjan Reinholdsen 04 May '05

04 May '05

All, I have written a Twisted server that will be used as a test server to test the client software ability to gracefully handle server shutdown and thereafter reconnect. For that purpose I need my twisted server to periodically stop listening on ports for a while, and then commence normal operation again. For some reason I don't seem to succeede since the twisted server never seems to stop listening on a port even if I call the stopListening method on the IListeningPort object. Anyone got a clue to what I'm doing wrong? Regards /Örjan ---- [...] ports = [8007,8008,8009] listenPorts = [] for port in ports: print "Listening on port %d" % port listenPorts.append(reactor.listenTCP(port, myFactory)) length = len(listenPorts) ptime = 10 dtime = 5 def Down(ticks): print "Stop listening on port %d" % ports[ticks%length] listenPorts[ticks%length].stopListening() reactor.callLater(dtime, Up, ticks) def Up(ticks): print "Start listening on port %d" % ports[ticks%length] listenPorts[ticks%length].startListening() reactor.callLater(ptime, Down, ticks+1) reactor.callLater(ptime, Down, 0) reactor.run()

2 1

Re: [Twisted-Python] blocking and threads
by Joachim Boomberschloss 03 May '05

03 May '05

Thanks, how embarassing... I suggest adding it to the "using threads" howto (I'd do it myself if I knew how). Another idea: make an asynchronous file-handling module that wraps Python's file IO with something that has a deferred-oriented interface. Is there already something like that too? Joe --- Jp Calderone <exarkun(a)divmod.com> wrote: > On Wed, 27 Apr 2005 05:49:01 -0700 (PDT), Joachim > Boomberschloss <boomberschloss(a)yahoo.com> wrote: > >Hi all, > > > >I am just wondering: from the Twisted how-tos, it > >appears that doing anything that could possibly > take > >arbitrarily long to execute should not be done in > the > >reactor's main thread; i.e. it should be done using > an > >asynchronous library (such as Twisted's network > >communication facilities), or in a different > thread, > >using the reactor's thread pool. > > > >So it would seem that many things that may be > >considered "primitive" in Python, such as file IO, > >require some kind of patching if they are to be > >immediately usable by a Twisted application. I came > up > >with the following solution, which enables calling > any > >function in a different thread with a deferred > >interface; I just wanted to make sure that I'm not > >completely missing some point: > > > >def deferToThread(func, *args, **kargs): > > """executes the given function in a thread, and > >passes the return value to the deferred we > return""" > > d = defer.Deferred() > > reactor.callInThread(_calledInThread, d, func, > *args, > >**kargs) > > return d > > > >def _calledInThread(d, func, *args, **kargs): > > try: > > retval = func(*args, **kargs) > > except Exception, x: > > reactor.callFromThread(d.errback, x) > > else: > > reactor.callFromThread(d.callback, retval) > > > > Looks like you've got a pretty firm understanding. > The one thing you did miss is the > twisted.internet.threads module, which provides the > deferToThread function. > > Jp > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

5 5

[Twisted-Python] Weekly Bug Summary
by James Y Knight 03 May '05

03 May '05

Bug summary ______________________ Since 2005-04-25 14:26:00.540000: Bugs opened: 11 Bugs closed: 7 Total open bugs: 350 (+4) Urgent: 7 (+1) Bug: 137 (+2) Feature: 136 (+1) Wish: 70 (+0) New / Reopened Bugs ______________________ ==== Urgent ==== [RELEASE MANAGEMENT] release 2.0.1 (4 days ago) http://twistedmatrix.com/bugs/issue1007 opened by jknight ProtocolWrapper's behavior is unspecified (6 days ago) http://twistedmatrix.com/bugs/issue1004 opened by exarkun ==== Bug ==== twisted.news.news (3 hours ago) http://twistedmatrix.com/bugs/issue1012 opened by manlio [WEB] Missing return in HTTPPageGetter.handleStatus_301()? (3 hours ago) http://twistedmatrix.com/bugs/issue1011 opened by jojo [DEFERRED, DOCUMENTATION] Document defgen (3 days ago) http://twistedmatrix.com/bugs/issue1009 opened by hypatia [PB] CopiedFailure printTraceback method signature doesn't match Failure's (4 days ago) http://twistedmatrix.com/bugs/issue1006 opened by micktwomey [TRIAL] Trial sucks and is broken (5 days ago) http://twistedmatrix.com/bugs/issue1005 opened by exarkun [FTP] [PATCH] twisted.protocols.ftp enchanements (7 days ago) http://twistedmatrix.com/bugs/issue1003 opened by mkerrin ==== Feature ==== [WEB] Be more failure tolerant in HTTPClient.lineReceived? (3 hours ago) http://twistedmatrix.com/bugs/issue1010 opened by jojo [ENTERPRISE] Add runWithConnection to adbapi.ConnectionPool (3 weeks ago) CLOSED http://twistedmatrix.com/bugs/issue975 opened by mg ==== Wish ==== [WIN32] Implement spawnProcess for IOCP reactor (3 days ago) http://twistedmatrix.com/bugs/issue1008 opened by justinj Bugs Closed ______________________ ==== Urgent ==== [XISH] Erroneous .encode('utf-8') in twisted.xish.domish (1 week ago) http://twistedmatrix.com/bugs/issue1002 opened by ralphm, closed by ralphm ==== Bug ==== [CORE, WIN32] twistenv.bat doesn't correctly set PATHEXT (2 weeks ago) http://twistedmatrix.com/bugs/issue991 opened by exarkun, closed by moonfallen [RELEASE MANAGEMENT] Better error message when missing twisted package (1 month ago) http://twistedmatrix.com/bugs/issue953 opened by ianb, closed by exarkun [TRIAL] [TEST] setUpClass run even if class set to skip (4 months ago) http://twistedmatrix.com/bugs/issue831 opened by exarkun, closed by exarkun [TRIAL] when two test modules have the same classname, class name isn't reported second time (4 months ago) http://twistedmatrix.com/bugs/issue832 opened by radix, closed by exarkun ==== Feature ==== [ENTERPRISE] Add runWithConnection to adbapi.ConnectionPool (3 weeks ago) http://twistedmatrix.com/bugs/issue975 opened by mg, closed by davep ==== Wish ==== [PB] [PATCH] Return PB Exceptions that have registered unjellyable (4 months ago) http://twistedmatrix.com/bugs/issue844 opened by justinj, closed by exarkun

1 0

[Twisted-Python] Another win32 installer release
by Cory Dodt 02 May '05

02 May '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 As usual, they're here. As usual, this is based on OFFICIAL, RELEASED Twisted 2.0.0 but the *installers themselves* are considered RC's. This fixes issue 991. It also fixes issue 996 for Windows. (Issue 996 pertains to other platforms as well.) http://twisted.sf.net/Twisted_NoDocs-2.0.0.win32(installer-rc4)-py2.4.exe http://twisted.sf.net/Twisted_NoDocs-2.0.0.win32(installer-rc4)-py2.3.exe C -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCdnVS3A5SrXAiHQcRAlVoAKCCbL5jqhNkceTkTRDrOZOS7ca0/QCgpgjo oqUbR6/dFtdiTpxymCjOw4o= =ebHv -----END PGP SIGNATURE-----

2 1

[Twisted-Python] Newbie reactor.crash()/reactor.stop() question
by T Barket 02 May '05

02 May '05

Hello, I have set up a server and client with perspective broker and can successfully move data back and forth between them. however, i want to use the client interactively at the python prompt. ie, i want to create a client object, make a request of the server and get the data returned to me. then i want to do some work with the data interactively at the python command prompt, until i need new data. then i want to use the client again to make another request of the server, etc. i have read i cannot stop and restart a reactor (which is what i would like to do on the client side). then i found out about reactor.crash(). that seems to work fine, which i think allow me to put the reactor on hold so i can get back to the command prompt and do some work until i need to make another request to the server. At least I can get it to work as expected. however, i now dont know how to call "reactor.stop()" before I exit python. once the client finishes, it sets a "reactor.crash()". now when I try to call reactor.stop(), i get the following error that the reactor isnt running: raise RuntimeError, "can't stop reactor that isn't running" RuntimeError: can't stop reactor that isn't running so now i cant stop the reactor using reactor.stop(). and furthermore, when i try to exit the python command prompt (having run reactor.crash() but not reactor.stop()), the python command prompt hangs instead of exiting normally and I have to kill its process. is there any way to exit python normally when the reactor has been put into a reactor.crash()? im using twisted 2.0 and python 2.4.1 on windows xp sp2. TIA, Tom

3 2

[Twisted-Python] dbus + twisted
by Anthony Baxter 02 May '05

02 May '05

So it turns out that the python dbus bindings just use glib - this means that if you're using gtk2reactor, you can use dbus and it all Just Works. This is very cool. It would be nice if there was a way to say (in dbus) "Give me any pending event" in a non-blocking way -- we could then write something similar to the current wxsupport module and use dbus from any reactor. I'm not sure that this would be all that easy, as dbus supplies bindings to other event loops rather than to the raw network layer - but it's something that's probably worth looking into further... Expect to see shtoom getting all dbussed up in the near future. ;) Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 2