Mailman 3 September 2009 - Twisted

Re: [Twisted-Python] Twisted OpenSSL Server and multiple clients
by gary clark 27 Sep '09

27 Sep '09

Ok I'm answering my own questions: After browsing around I kinda pieced together what is going on. Basically the factory object manages the persistent information of protcols. You can use self.clients.append( newclient ) to store when a connection is made and self.clients.remove( client ) when a client disconnects within the factory object to cycle through all the protocol (clients) in the factory: In the factory: for proto in self.clients proto.transport.write( message ) I found the below code very useful: import string from twisted.application import service, internet from twisted.internet import reactor, protocol from twisted.internet.protocol import Protocol, Factory port = 80 class AirChat(Protocol): username = "" session = False def connectionMade( self ): self.factory.numProtocols = self.factory.numProtocols+1 self.factory.addClient( self ) def connectionLost( self , reason ): self.factory.numProtocols = self.factory.numProtocols-1 self.factory.delClient( self ) def dataReceived( self , data ): # parse out HTTP a = data.split( '\x00' )[0].split(' ') # look for HTTP header if a[0] == 'GET' or a[0] == 'POST': self.transport.write( """HTTP/1.1 200 OK\r\n\r\n""" ) self.transport.write( """<html><strong>AIR Chat 1.0</strong></html>""" ) self.transport.loseConnection() return a = data.split( '\x00' )[0].split( '||' ) if a[0] == '1': self.factory.sendAll( '1||' + self.factory.stripChars( a[1] ) + '||' + self.factory.stripChars( a[2] ) + '\x00' ) elif a[0] == '2': self.transport.write( '2||' + str( self.factory.numProtocols ) + '\x00') class AirChatFactory(Factory): protocol = AirChat numProtocols = 0 clients = [] def stripChars(self, textIn): textIn = textIn.replace('<', '') textIn = textIn.replace('</', '') textIn = textIn.replace('>', '') textIn = textIn.replace('/>', '') return textIn def addClient(self, newclient): self.clients.append( newclient ) def delClient(self, client): self.clients.remove( client ) def sendAll(self, message): for proto in self.clients: proto.transport.write( message ) class AirChatService( internet.TCPServer ): def __init__(self): internet.TCPServer.__init__(self, 80, AirChatFactory()) I think thats good for now. I dont have to create my own repository just manipulate the factory object, marvelous. Time for a pint. Thanks, Garyc --- On Sun, 9/27/09, gary clark <burslem2001(a)yahoo.com> wrote: > From: gary clark <burslem2001(a)yahoo.com> > Subject: Twisted OpenSSL Server and multiple clients > To: twisted-python(a)twistedmatrix.com > Date: Sunday, September 27, 2009, 3:08 PM > Hello, > > I am new to Twisted, hence the red flag and patience card > pulled out. > > I have written a client that connects perfectly well to a > twisted server thats using open-ssl to communicate to each > other. Basically I took the open-ssl echo server example and > tweaked it. > > Thats all fine and dandy, but I want to have multiple > clients that can connect to the server. > > So I have a few questions: > > 1) When a client connects to my server? What connection > information > can be collected and stored persistently? > > > Upon a client connecting I want to store > connection information and > I want to remove connection information > when its disconnected from > the persistent store. > > 2) What persistent store is adopted for twisted if any? > > 3) I want to know when a table has changed so I can then > pull information > from a table and use the connection > information if present to send data > to my client. How is this accomplished in > twisted? > > Again I apologise in advance, if someone can shed some > light into how twisted accomplishes the basic tasks of > multiple clients thats great. > > Oh boy. > > Thanks, > Garyc > > > >

1 0

[Twisted-Python] Twisted OpenSSL Server and multiple clients
by gary clark 27 Sep '09

27 Sep '09

Hello, I am new to Twisted, hence the red flag and patience card pulled out. I have written a client that connects perfectly well to a twisted server thats using open-ssl to communicate to each other. Basically I took the open-ssl echo server example and tweaked it. Thats all fine and dandy, but I want to have multiple clients that can connect to the server. So I have a few questions: 1) When a client connects to my server? What connection information can be collected and stored persistently? Upon a client connecting I want to store connection information and I want to remove connection information when its disconnected from the persistent store. 2) What persistent store is adopted for twisted if any? 3) I want to know when a table has changed so I can then pull information from a table and use the connection information if present to send data to my client. How is this accomplished in twisted? Again I apologise in advance, if someone can shed some light into how twisted accomplishes the basic tasks of multiple clients thats great. Oh boy. Thanks, Garyc

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 26 Sep '09

26 Sep '09

1 0

Re: [Twisted-Python] Trial: Blocking until all deferreds have fired in test
by Terry Jones 25 Sep '09

25 Sep '09

Hi Mark. I'd write your FooListener class like this (untested): class FooListener(object): def fooBar(): d = doSomeSetup() d.addCallback(lambda ign: doSomethingThatReturnsADeferred()) d.addCallbacks(lambda ign: 'success!', lambda ign: 'failure!') return d And leave your unit test as is. Then the unit test (with the yield on fooBar() will do the right thing. And your other (non-test) code will continue to get what it was getting before. BTW, it's a slight red flag (to me) that you're returning the 'failure!' string as the value of the deferred (i.e., if an error happens, that errback function returns a string, which is going to get you back onto the callback chain in the deferred - since you're not raising or returning a failure). I say it's a red flag because I tried stuff like that once and later realized that things are much cleaner if you just let Twisted handle errors in it's normal way. To do so you could just drop the addErrback above and let any error flow back to the caller of fooBar in the normal way (on the errback chain) rather than trying to catch problems and return a result on the callback chain that indicates an error (i.e., a string like 'failure!'). Hope that helps. Terry

1 0

[Twisted-Python] Trial: Blocking until all deferreds have fired in test
by Mark Visser 25 Sep '09

25 Sep '09

I have a class that looks something like this: |class FooListener(object): def __init__(self): pass @inlineCallbacks def fooBar(): try: yield doSomeSetup() doSomethingThatReturnsADeferred() returnValue('success!') except: returnValue('failure!') |Note that doSomethingThatReturnsADeferred() has no callbacks. I also have a unit test that looks something like this: |def TestFoo(unittest.TestCase): def setUp(self): timeout = 1000000 self.listener = FooListener() @inlineCallbacks def testFooBar(self): yield self.listener.fooBar() | When I run this, I get this error: [ERROR]: foo.TestFoo.testFooBar Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. Selectables: <twisted.internet.process.ProcessWriter object at 0x3b7b650> ------------------------------------------------------------------------------- Ran 1 tests in 9.028s FAILED (errors=1, successes=1) I'm clever enough to understand that Trial isn't waiting for the "backgrounded" Deferreds to fire, but I'm not clever enough to figure how to fix it. :-) I could hack around this by passing a test_mode parameter to TestFoo.__init__ and "yield"-ing doSomethingThatReturnsADeferred(), but I'd rather not introduce special cases for testing. Is there a way to instruct Trial to "wait" for Deferreds to fire? thanks -- Mark Visser, Software Director Lumière VFX Email: markv(a)lumierevfx.com Phone: +1-514-316-1080 x3030

1 0

Re: [Twisted-Python] Pyjamas + web2py + twisted vs Django + Pinax
by Luke Kenneth Casson Leighton 24 Sep '09

24 Sep '09

In-Reply-To=65BAE9EEDADD409CA83C32F2E54A0CB9%40Podcaster sorry, folks: i didn't see this message, and it's taken me ten months of obscure searches to find it. > > I'm about to start programming a game in Python. The game will > > be a web-based board game with a social aspect, something like > > http://www.chesspark.com. I want the game to run in both browsers > > and as a desktop client. and presumably you don't want to duplicate effort. ergo, pyjamas is a good choice. actually... it's your _only_ python choice if you don't want to burden the users. your other choices are: adobe AIR, silverlight, and, more recently, appcelerator. appcelerator is interesting: you get to run IronPython because appcelerator's "titanium" is actually a silverlight / browser plugin that gives IronPython apps access to the DOM model of the browser. > > Therefore I'm wondering what python-based > > technologies I need. I know that Javascript/AJAX is necessary for > > the browser part so I have decided on the following: > > > > 1. Pyjamas - (for the AJAX browser stuff) > > > > 2. Pyjamas-desktop - (for the desktop client) pyjamas-desktop now has XULrunner and MSHTML as back-end engines. the MSHTML one is particularly strategically significant, as the only real requirements are, instead of a whopping 30mb bundle including gtk, cairo, xulrunner, pygtk, pycairo, pygobject _and_ not to mention python itself, you can just install... python-comtypes (which is a 200k download). for the POSIX-compliant systems, such as debian, there's "apt-get install hulahop", and you get python-xpcom, xulrunner and all other dependencies pulled in "for free". > > 3. Web2py (or djanjo, or django+pinax) - for the website on which the game will run and for database access and social part - login, chat, members, profiles, game history, etc.) > > > > 4. Twisted - for the networking and chat functionality. > > > > 5. Pygame - for most of the game stuff, sprites, inages, etc. (Actually I don't know if Pygame is necessary for a board-type game.) no, it's not. i wrote a board-type game "meta-engine" which involved moving of pyjamas.ui.Image "tokens" around a pyjamas.ui.AbsolutePanel and it was incredibly simple and it actually worked. i went to a lot of trouble to use the django server and AJAX (actually JSONRPC) to communicate "moving" of tokens. as one user moved a token, the fact that it had "moved" would be noted in a database table (id, game_session_id, user_id, token_id) of course obviously excluding the user who had moved it. a continuous poll by a user of the database would empty the table of "moved" notifications for that user, and return a list of new position info for the tokens that needed to be moved. update speed turned out to be about 1-2 frames per second: perfectly adequate for a board game. oh: for "Go", due to the number of tokens, however, i found that over 700 tokens resulted in _considerably_ slow Firefox 2 initialisation times, which turns out to be a known bug in Firefox 2. adding over 256 absolute-positioned items to a div element using DOM manipulation is *SEVERELY* cpu-intensive: beyond some sort of fixed limit, adding elements turns from O(N) to O(N^2). for full information on the issue, courtesy of john gunther, and a solution, see: http://pyjamas.svn.sourceforge.net/viewvc/pyjamas/trunk/library/pyjamas/cha… i also added "spectators" which were simply "users with no rights to move tokens". they got their own "chat" channel so as not to interfere with the players. twisted users are probably screaming loudly at the "polling" technique especially one that hammers a database so heavily, but hey, you can't have everything :) if anyone knows of any better event-driven methods including example code that can be queried with JSONRPC i'm _really_ interested to hear about it. think in terms of "don't respond to the JSONRPC request unless either 15 seconds has passed, or there are some events which have been triggered by other users". yes i fully understand the complexities involved and that one possible implementation is a single-process asynchronous JSONRPC HTTP service, which is _nasty_ as hell: i'm interested to hear if there's anything better. > > My understanding of the above technologies is still limited and I'm not sure if I need all of these because some functionality may be overlapped. e.g. I don't know where Pyjamas leaves off and where web2py/danjo/pinax begins. > > > > Can someone please give me some advice? > > That's a fine laundry list of frameworks, but i believe getting all of > them to cooperate would be practically impossible. (Note: a framework > is a library that's grown to the point where it thinks it owns your > whole program. Getting multiple frameworks to play nicely together is > usually hard. There are rare exceptions, like the beautiful integration > between Twisted and PyGTK, but they're rare.) the integration point of choice between pyjamas and frameworks is JSONRPC, simply because it was the first one that was implemented, and it's turned out to be incredibly practical. examination of the wiki http://code.google.com/p/pyjamas/wiki shows over 90% of the wiki pages are "how to use pyjamas with <insert framework>". including two or three methods for twisted, and two or more with django. > I know that Django and pygame and Twisted are useful, mature, documented > frameworks that actually get used in production by people other than > their authors. I don't know about Pyjamas. http://pyjs.org/FAQ.html "i'm a bit underwhelmed by the apparent lack of documentation": says "go back to main site, http://pyjs.org, see Documentation section" which has these: http://pyjs.org/api http://pyjs.org/book/output/Bookreader.html still relevant, because the U.I API hasn't really changed: http://code.google.com/webtoolkit/documentation for best results, read: http://google-web-toolkit.googlecode.com/svn/javadoc/1.6/index.html specifically the U.I "package": http://google-web-toolkit.googlecode.com/svn/javadoc/1.6/com/google/gwt/use… > If putting 100% Python in one end and getting Ajax out the other is > really important to you, then you need Pyjamas to work. (Or wait for > PyPy and its JavaScript backend to mature from experimental research > code to production-quality stuff.) that's not going to happen. the pypy javascript back-end has been abandoned. it's still available in a branch, but you are not going to get anywhere with it. plus, due to the fact that the sole and exclusive focus of the pypy javascript back-end is to be totally 100% python interoperable, to the absolute exclusion of all else (e.g. speed), even if you got it going, you'd find that apps ran like treacle. then, if you tried to debug the app, you'd find that it was near-impossible, because the code is near-pure-assembler-like, having gone through rpython, first. being utterly divorced from its original python, you are entirely at the mercy of the pypy developers for bug-fixes aaand... they've abandoned it. bottom line: forget pypy. > So my advice is to write a simple > tic-tac-toe game in Pyjamas, and find out. or a lightout game. http://pyjs.org/examples/lightout/output/lightout.html or a sudoku game. http://lkcl.net/sudoku/output/Sudoku.html (in each case, full source code is 1 level below) of course, these are single-user games. what you should be slightly surprised by is the fact that the source code of each app is so damn small. the sudoku one, it hardly looks like there's any actual "pyjamas" code in it at all. you have to _really_ peer very hard at the code, looking for clues such as "ListBox", "Grid", "HTML" and "HorizontalPanel" in order to discern the bits that just get on with serving up a sudoku game and the bits which "glorify pyjamas" so to speak. "oh look, there's a button. i wonder if it calls this funny method named 'onClick' if the button gets clicked. why yes, what a surprise, it does!" anyway this is a bit long, i'll stop now :) l.

1 0

Re: [Twisted-Python] Unit tests, inline callbacks, and exception handling
by Terry Jones 24 Sep '09

24 Sep '09

Hi Patrick You have try: client2 = Client(username = "Test") yield client2.connect(port = server_port) assert False except UsernameAlreadyExists, ex: pass You could instead try: client2 = Client(username = "Test") d = client2.connect(port = server_port) self.failUnlessFailure(d, UsernameAlreadyExists) yield d Terry

1 0

[Twisted-Python] Unit tests, inline callbacks, and exception handling
by Patrick Hartling 24 Sep '09

24 Sep '09

We are using nose and the twisted.trial.unittest.TestCase class from Twisted 8.2.0 to write tests that use inline callbacks. In one of the tests, we are trying to validate that the code will behave correctly by raising an exception if two clients with the same user name connect to a server. The basic test code is similar to this: @inlineCallbacks def testDuplicateConnect(self): server_port = random.randint(10000, 40000) server = Server() server.listen(port = server_port) client1 = Client(username = "Test") yield client1.connect(port = server_port) try: client2 = Client(username = "Test") yield client2.connect(port = server_port) assert False except UsernameAlreadyExists, ex: pass yield server.close() yield client.close() The test method executes and returns successfully. The exception is raised and caught correctly, but because a twisted.python.failure.Failure object was seen during deferred processing, twisted.trial.unittest.TestCase._cleanUp() concludes that an error occurred. That is perhaps not the most accurate way of describing what happens, but the end result is that a caught exception ends up being reported as an error. My question is this: are there any tips for performing exception handling in test methods that are decorated with @inlineCallbacks? I have been tracing through a lot of code trying to find any extension point, hook, etc. where I could indicate that the exception was caught correctly, but I have not found anything yet. Adding the obvious @raises decoration does not work because the method catches the exception. The closest I have come is to call self._observer._ignoreErrors() from the test method with the exception type in question, but that does not strike me as being on the right track. -Patrick -- Patrick L. Hartling Senior Software Engineer, Priority 5 http://www.priority5.com/ The information transmitted in this communication is intended only for the person or entity to which it is addressed and contains proprietary material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please destroy any copies, contact the sender and delete the material from any computer.

2 2

Re: [Twisted-Python] VC++ client and a Twisted OpenSSL Server
by gary clark 24 Sep '09

24 Sep '09

Ok I downloaded wireshark onto the linux box and commenced sniffing the packets. I saw the problem with sending the packets back (Host unreachable). Dont know why the pings were not being stopped (proabably allowing ICMP). I then enable the port on the linux box. I am now seeing the following: error:0200274D:system library:connect:reason(1869) ECONNREFUSED = 0x274D = 10061 Any ideas would be appreciated. Thanks, Garyc --- On Wed, 9/23/09, gary clark <burslem2001(a)yahoo.com> wrote: > From: gary clark <burslem2001(a)yahoo.com> > Subject: [Twisted-Python] VC++ client and a Twisted OpenSSL Server > To: twisted-python(a)twistedmatrix.com > Date: Wednesday, September 23, 2009, 8:03 PM > Hello, > > error:02002751:system library:connect:reason(1873) > > Is dumped out from the client. This apparently is a : > > 0x2751 == 10065 == (WSA)EHOSTUNREACH > > "A socket operation was attempted to an unreachable > host." > > Strange because I can ping the server machine from the > client. Not very informative. > > Oh boy any ideas? I'm running the client on windows and the > twisted server on linux. > > Thanks, > Garyc > >

1 2

[Twisted-Python] VC++ client and a Twisted OpenSSL Server
by gary clark 23 Sep '09

23 Sep '09

Hello, error:02002751:system library:connect:reason(1873) Is dumped out from the client. This apparently is a : 0x2751 == 10065 == (WSA)EHOSTUNREACH > "A socket operation was attempted to an unreachable host." Strange because I can ping the server machine from the client. Not very informative. Oh boy any ideas? I'm running the client on windows and the twisted server on linux. Thanks, Garyc

1 0