Twisted October 2007

twisted@python.org

58 participants
47 discussions

Start a nNew thread

RE: [Twisted-Python] Need some help with my first Twisted program
by McCann, Brian 26 Oct '07

26 Oct '07

1 0

RE: [Twisted-Python] Need some help with my first Twisted program
by McCann, Brian 26 Oct '07

26 Oct '07

2 1

[Twisted-Python] Using reactor to connect to multiple servers
by Vijay Mathew 26 Oct '07

26 Oct '07

Hi all, I am in the process of writing a http client using twisted. I want it to connect to multiple http servers and download more than one page asynchronously, without using threads. How can I use the reactor object to connect to multiple servers and download data like this? thanks, --Vijay

2 1

[Twisted-Python] Twistd problem with redirection
by Roland Hedberg 24 Oct '07

24 Oct '07

Hi! I've encountered a strange problem which I'd like to have some help with. I'm building some software using Twisted and have encountered the following problem. I use twistd 2.5.0 and starts the server using: twistd pyom --logfile=x.log ... Now, the tap file that is run by twistd loads and initiates modules dependent on the configuration. And these modules might load other modules. So, what happens is that, depending (looks like it) on how complex the setup is, the redirect of stdout output for print to the logfile is not universally done. Consistent but depending on the configuration, the output from print for one of modules continues to go to stdout instead of to the logfile. Anyone got a clue as to what is happening and what I can do about it ? -- Roland

2 2

[Twisted-Python] CFReactor redux
by Phil Christensen 24 Oct '07

24 Oct '07

It's been awhile since this was brought up on the mailing list, but I'd like to see if anyone has any real interest in using Twisted with Cocoa on Mac OS X. It was suggested to me that one thing that was needed was a reactor based on ThreadedSelectReactor that would integrate with the Cocoa runtime. I've posted my results here: http://twistedmatrix.com/trac/ticket/1833 This class is extremely trivial, but it makes working with PyObjC/ Twisted apps a bit more straightforward. However, there are significant issues with the entire TSR approach. The primary one is that it's impossible to do any Trial testing, because as far as I can tell, it is impossible to make the CFReactor's run() method return without terminating the entire process. I have a couple questions for the group, namely: 1.) Does anyone with a better grasp of Cocoa/PyObjC development have any ideas on how a suitable reactor might be implemented? 2.) Is there any way to run the unit tests without restarting the reactor between each test? I'm sure this would cause many tests to fail, but at least we'd be able to see if there was any other major issues. I'm not suggesting any changes be made to trial. I'm just looking for any possible way to run the tests manually. 3.) Will anyone use this? I know the core devs are generally uninterested in supporting a Cocoa reactor, but I'm glad to maintain this if it would be used. -phil

3 3

[Twisted-Python] reading serial port asynchronously
by Tom Brown 22 Oct '07

22 Oct '07

Hi, I have an application that reads characters off the serial port in a thread. The problem is, the entire application will hang sometimes. I believe it is a threading problem, but I can't pinpoint why the application hangs. So, I'd like to get away from threads and thought Twisted may help me. I was told Twisted has the ability to interface with serial ports. I don't know where to look for the information on how to do this. Any ideas on where to start looking or pointers on how to accomplish this? Thanks, Tom

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 21 Oct '07

21 Oct '07

Bug summary ______________________ Summary for 2007-10-14 through 2007-10-21 Bugs opened: 4 Bugs closed: 1 Total open bugs: 1043 (+3) |== Type Changes |== Priority Changes |== Component Changes |Defect: +2 |Highest: -1 |Core: +3 |Enhancement: +1 |Normal: +3 |Names: +1 |Lowest: +1 |Web2: -1 New / Reopened Bugs ______________________ ===== Normal ===== [#2850] getHostByName Error when DNS Server return Name Server (opened by wangminghua) defect names http://twistedmatrix.com/trac/ticket/2850 [#2851] No hooks in NetstringReceiver to catch parse errors. (opened by jml) enhancement core http://twistedmatrix.com/trac/ticket/2851 [#2853] findFailure result is used exclusively, ignoring new traceback (opened by Peaker) defect core http://twistedmatrix.com/trac/ticket/2853 ===== Lowest ===== [#2852] spawnProcess doc glitch (opened by alanfranzoni) defect core http://twistedmatrix.com/trac/ticket/2852 Closed Bugs ______________________ ===== Highest ===== [#2304] t.web2.client.http.HTTPClientChannelRequest.connectionLost does nothing (opened by ghazel, closed by therve, fixed) defect web2 http://twistedmatrix.com/trac/ticket/2304 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2853] findFailure result is used exclusively, ignoring new traceback (since 2007-10-17 22:04:16). Mean open ticket age: 544 days, 17:31:29.430966. Median: 446 days, 3:59:09.191273. Standard deviation: 404 days, 0:42:51.794059. Interquartile range: 505 days, 3:59:29. Mean time between ticket creation and ticket resolution: 122 days, 16:37:43.001128. Median: 14 days, 16:22:15. Standard deviation is 226 days, 21:33:13.531588. The interquartile range is 130 days, 2:23:34. Mean time spent in review: 68 days, 20:15:20.554302. Median: 4 days, 9:08:37. Standard deviation: 151 days, 9:40:09.422509. Interquartile range: 31 days, 18:40:54. Mean number of times a ticket is reviewed: 1.96567505721. Median: 1 Standard deviation: 1.61407221677. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 12 unique ticket reporters 7 unique ticket reviewers 2 unique ticket resolvers In the last 24 weeks, 69 unique ticket reporters 16 unique ticket reviewers 13 unique ticket resolvers In the last 48 weeks, 152 unique ticket reporters 18 unique ticket reviewers 25 unique ticket resolvers

1 0

RE: [Twisted-Python] Need some help with my first Twisted program
by McCann, Brian 19 Oct '07

19 Oct '07

> -----Original Message----- > From: twisted-python-bounces(a)twistedmatrix.com > [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf > Of Jean-Paul Calderone > Sent: Friday, October 19, 2007 4:29 PM > To: Twisted general discussion > Subject: Re: [Twisted-Python] Need some help with my first > Twisted program > > On Fri, 19 Oct 2007 16:13:20 -0400, "McCann, Brian" > <bmccann(a)andmore.com> wrote: > >I'm looking for what is probably an simple solution I can't > figure out > >on my own. I'm writing an SSH server based on the example on the web > >(using conch). I'm trying to figure out how to detect when > the client > >exists (for example, when I just close out PuTTY), but I > can't get this > >to work right. Looking through the API docs I found > "connectionLost()", > >which I put in my protocol class (EchoProtocol in the > example), but it's > >never getting called. > > > >Can someone please tell me what I'm doing wrong? > > You seem to be on the right track. However, you haven't > supplied enough > details for anyone to be sure why it isn't behaving as you > expect. If you > can post a minimal example of a program which you think > should be working > (ie, giving you connection lost notification) but isn't, then > someone can > probably point you to a solution. > > Jean-Paul > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > Sorry about that. Here is most of what is involved. (Code below) I would think that when I close my PuTTY window, the connectionLost function in the EchoProtocol would be called...but clearly I'm missing some crucial step. --Brian from twisted.cred import portal, checkers from twisted.conch import error, avatar from twisted.conch.checkers import SSHPublicKeyDatabase from twisted.conch.ssh import factory, userauth, connection, keys, session from twisted.internet import reactor, protocol, defer from twisted.python import log from zope.interface import implements import sys import socket import threading log.startLogging(sys.stderr) """Example of running another protocol over an SSH channel. log in with username "user" and password "password". """ class defaultdict(dict): def __init__(self, default=None): dict.__init__(self) self.default = default def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: return self.default class ExampleAvatar(avatar.ConchUser): def __init__(self, username): avatar.ConchUser.__init__(self) self.username = username self.channelLookup.update({'session':session.SSHSession}) class ExampleRealm: implements(portal.IRealm) def requestAvatar(self, avatarId, mind, *interfaces): return interfaces[0], ExampleAvatar(avatarId), lambda: None class EchoProtocol(protocol.Protocol): """this is our example protocol that we will run over SSH """ def connectionMade(self): self.transport.write("Connected!\r\n") self.commandBuffer = "" def connectionLost(self,reason): print "lost:%s" %reason self.serialServerSocket.close() self.alive = False self.serialSocketReadThread.join() publicKey = 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEArzJx8OYOnJmzf4tfBEvLi8DVPrJ3/c9k2I/Az64fxjHf 9imyRJbixtQhlH9lfNjUIx+4LmrJH5QNRsFporcHDKOTwTTYLh5KmRpslkYHRivcJSkbh/C+ BR3utDS555mV' privateKey = """-----BEGIN RSA PRIVATE KEY----- MIIByAIBAAJhAK8ycfDmDpyZs3+LXwRLy4vA1T6yd/3PZNiPwM+uH8Yx3/YpskSW 4sbUIZR/ZXzY1CMfuC5qyR+UDUbBaaK3Bwyjk8E02C4eSpkabJZGB0Yr3CUpG4fw vgUd7rQ0ueeZlQIBIwJgbh+1VZfr7WftK5lu7MHtqE1S1vPWZQYE3+VUn8yJADyb Z4fsZaCrzW9lkIqXkE3GIY+ojdhZhkO1gbG0118sIgphwSWKRxK0mvh6ERxKqIt1 xJEJO74EykXZV4oNJ8sjAjEA3J9r2ZghVhGN6V8DnQrTk24Td0E8hU8AcP0FVP+8 PQm/g/aXf2QQkQT+omdHVEJrAjEAy0pL0EBH6EVS98evDCBtQw22OZT52qXlAwZ2 gyTriKFVoqjeEjt3SZKKqXHSApP/AjBLpF99zcJJZRq2abgYlf9lv1chkrWqDHUu DZttmYJeEfiFBBavVYIF1dOlZT0G8jMCMBc7sOSZodFnAiryP+Qg9otSBjJ3bQML pSTqy7c3a2AScC/YyOwkDaICHnnD3XyjMwIxALRzl0tQEKMXs6hH8ToUdlLROCrP EhQ0wahUTCk1gKA4uPD6TMTChavbh4K63OvbKg== -----END RSA PRIVATE KEY-----""" class InMemoryPublicKeyChecker(SSHPublicKeyDatabase): def checkKey(self, credentials): return credentials.username == 'user' and \ keys.getPublicKeyString(data=publicKey) == credentials.blob class ExampleSession: def __init__(self, avatar): """ We don't use it, but the adapter is passed the avatar as its first argument. """ def getPty(self, term, windowSize, attrs): pass def execCommand(self, proto, cmd): raise Exception("no executing commands") def openShell(self, trans): ep = EchoProtocol() ep.makeConnection(trans) trans.makeConnection(session.wrapProtocol(ep)) def eofReceived(self): pass def closed(self): pass from twisted.python import components components.registerAdapter(ExampleSession, ExampleAvatar, session.ISession) class ExampleFactory(factory.SSHFactory): publicKeys = { 'ssh-rsa': keys.getPublicKeyString(data=publicKey) } privateKeys = { 'ssh-rsa': keys.getPrivateKeyObject(data=privateKey) } services = { 'ssh-userauth': userauth.SSHUserAuthServer, 'ssh-connection': connection.SSHConnection } portal = portal.Portal(ExampleRealm()) passwdDB = checkers.InMemoryUsernamePasswordDatabaseDontUse() passwdDB.addUser('user', 'password') portal.registerChecker(passwdDB) portal.registerChecker(InMemoryPublicKeyChecker()) ExampleFactory.portal = portal if __name__ == '__main__': reactor.listenTCP(5022, ExampleFactory()) reactor.run()

2 1

[Twisted-Python] Need some help with my first Twisted program
by McCann, Brian 19 Oct '07

19 Oct '07

I'm looking for what is probably an simple solution I can't figure out on my own. I'm writing an SSH server based on the example on the web (using conch). I'm trying to figure out how to detect when the client exists (for example, when I just close out PuTTY), but I can't get this to work right. Looking through the API docs I found "connectionLost()", which I put in my protocol class (EchoProtocol in the example), but it's never getting called. Can someone please tell me what I'm doing wrong? Thanks, --Brian

2 1

Re: [Twisted-Python] Re: Content Encoding : gzip ?
by anurag uniyal 19 Oct '07

19 Oct '07

Thanks. So it seems I have to write my own code, there in nothing like I can set some flag. So if that is the case shouldn't I just be gzipping and unzipping data write and read to request object. rgds anurag ----- Original Message ---- From: David Bolen <db3l.net(a)gmail.com> To: twisted-python(a)twistedmatrix.com Sent: Friday, October 19, 2007 1:02:21 AM Subject: [Twisted-Python] Re: Content Encoding : gzip ? anurag uniyal <anuraguniyal(a)yahoo.com> writes: > Is it possible to send and recieve compressed content using twisted.web? > > I have set Accept-encoding to gzip but it doesn't make any difference. > Will I have to cater for this myself? Yes, in twisted.web. In twisted.web2, there's a filter that supports it. Here's a small wrapper class I wrote recently that I've used to handle this in twisted.web, with the goal of minimal changes to existing resource code. I use it by noticing, in the original request handler, the ability to handle gzip encodings, and then wrap the original request in this class (since there's no built-in filtering of the output stream in twisted.web). It's worked fine in my tests to date, and in limited production use, but to be honest it hasn't seen large scale, long term use because my original use case only yielded ~15% compression (multimedia files) and using this negates the ability to know the full length in advance, thus clients can't give projected download times. (My files are large enough that compressing twice is high overhead, and it wouldn't play well with the wrapper approach anyway). In any event, hopefully it'll be a useful starting point for you. Here's the wrapper: - - - - - - - - - - - - - - - - - - - - - - - - - import struct import zlib class GzipRequest(object): """Wrapper for a request that applies a gzip content encoding""" def __init__(self, request, compressLevel=6): self.request = request self.request.setHeader('Content-Encoding', 'gzip') # Borrowed from twisted.web2 gzip filter self.compress = zlib.compressobj(compressLevel, zlib.DEFLATED, -zlib.MAX_WBITS, zlib.DEF_MEM_LEVEL,0) def __getattr__(self, attr): if 'request' in self.__dict__: return getattr(self.request, attr) else: raise AttributeError, attr def __setattr__(self, attr, value): if 'request' in self.__dict__: return setattr(self.request, attr, value) else: self.__dict__[attr] = value def write(self, data): if not self.request.startedWriting: self.crc = zlib.crc32('') self.size = self.csize = 0 # XXX: Zap any length for now since we don't know final size if 'content-length' in self.request.headers: del self.request.headers['content-length'] # Borrow header information from twisted.web2 gzip filter self.request.write('\037\213\010\000' '\0\0\0\0' '\002\377') self.crc = zlib.crc32(data, self.crc) self.size += len(data) cdata = self.compress.compress(data) self.csize += len(cdata) if cdata: self.request.write(cdata) elif self.request.producer: # Simulate another pull even though it hasn't really made it # out to the consumer yet. self.request.producer.resumeProducing() def finish(self): remain = self.compress.flush() self.csize += len(remain) if remain: self.request.write(remain) self.request.write(struct.pack('<LL', self.crc & 0xFFFFFFFFL, self.size & 0xFFFFFFFFL)) self.request.finish() - - - - - - - - - - - - - - - - - - - - - - - - - and here's a sample of using it. This code is from one of my Resource objects - if I were to use it more generally in my site I'd extract that into a mix-in class of some sort - currently there was only one specific resource (the multimedia files) I wanted to support it for: def render_GET(self, request): # (... argument validation ...) accept_encoding = request.getHeader('accept-encoding') if accept_encoding: encodings = accept_encoding.split(',') for encoding in encodings: name = encoding.split(';')[0].strip() if name == 'gzip': request = GzipRequest(request) break # At this point, use 'request' as normal -- David _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

2 1

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

Twisted October 2007