Mailman 3 July 2007 - Twisted

[Twisted-Python] SSHv2 client script for Cisco routers
by Chris Hallman 26 Jul '07

26 Jul '07

Does anyone have an example of a SSHv2 client script? I need to write a script that uses SSHv2 to execute multiple commands on hundreds of routers. I already have a script that does this with the Telnetlib. I need to transition to SSHv2.

3 6

[Twisted-Python] How to achieve a reliable "connection" over UDP?
by Peter Valdemar Mørch 25 Jul '07

25 Jul '07

Hi, I'm looking to implement an application that needs reliable connections, like one gets with TCP, but only using UDP "on the wire". The application can only use UDP, as I'd like to rely on UDP hole punching (p2p). (P2P over TCP is notoriously difficult). I want to be certain that my packets get delivered, that they are in order, to transfer data larter than 64K, etc - just like in TCP. Of course, I could implement the TCP protocol myself for use over UDP, just as the IP protocol stack does for TCP over IP. Or use TCP over PPP over UDP. But is there "a better way"? I'm thinking basically all p2p applications will need this sort of "reliability" over UDP for e.g. file transfer, so I'm hoping there is something already written. How would one go about this? Peter -- Peter Valdemar Mørch http://www.morch.com

5 7

[Twisted-Python] Testing with trial, adbapi questions
by Brendon Colby 24 Jul '07

24 Jul '07

Greetings, I've begun moving an application I wrote from asynchat to twisted. I've got basic functionality, I've converted my DB stuff to adbapi and have begun writing unit tests. I have a few questions that I haven't been able to get answered through the API docs, twisted.words protocol code, mailing list posts etc. I'd be very grateful for some assistance! 1. Right now I'm defining my dbPool in my factory init, and running a method to load some data from the DB. I've copied the IRC BasicServerFunctionalityTestCase and modified it for my use: class BasicServerFunctionalityTestCase(unittest.TestCase): def setUp(self): self.f = StringIOWithoutClosing() self.t = FileWrapperThatWorks(self.f) # I need this to return an IPv4Address! self.p = sserverd.SServerProtocol() self.p.factory = sserverd.SServerFactory(self.p) self.p.makeConnection(self.t) self.p.factory.dbPool.start() def tearDown(self): self.p.factory.dbPool.close() def check(self, s): self.assertEquals(self.f.getvalue(), s) def testSendDeveloperID(self): self.p.lineReceived('{%s:"PG"}\0' % sserverson.base10toN(104)) self.check("test\0") I've gotten past the database threads hanging open by running start() and close() manually. Now, my database callbacks that load necessary data appear to be getting called AFTER my test runs. I know they're getting called, but when the above test runs, the required data structure is empty. What am I missing here? 2. I'm having a bit of trouble wrapping my head around adbapi. Prior to it, one thing I would do is gather data and return a row, as in, the method "developerExists()" would check the DB and return row[0][0] or something (a bool). Now it seems as though I have to preload this data and check against that. Can I replicate this behavior with adbapi? Is there a better way than having to preload ALL my data up front? How can I get any data from the DB like this: data = getSomethingFromDB() I'm suspicious that I need to adopt a new paradigm here, but am having difficulty seeing past the old way of doing things! 3. I have some methods (as I pointed out in #1) that load data. With adbapi, it appears that I have to do this: def _loadDevelopers(self,results): for result in results: developerID = result[2] ... self.addDeveloper(Developer(authHost,authUrl,developerID,self,rc4key, postHost,postUrl)) def loadDevelopers(self): self.dbPool.runQuery("select * from developers where active=1").addCallback( self._loadDevelopers).addErrback(printError) # Before, I would just load my data here! Is this correct? i.e. a DB method that gets called, each having a corresponding callback method that does the real work? (This seems sort of a pain to me...) I feel as though I'm close to "getting it" (twisted in general, adbapi) but feel that I have yet to connect some key pieces of information. 4. I was digging through the adbapi code, and it appears that the only type of result set I can get is a tuple (because only cursors are used...). Is there a way I can get a dictionary returned, keys are column names? i.e. a MySqlDB.fetch_row(how=1). I'm just not seeing how I can do this with adbapi or, rather, how I can pass through to the MySqlDB module to get this type of result set. 5. I've always run my application using DJB's daemontools (http://cr.yp.to/daemontools.html). Is there an incredibly compelling reason to switch to using twistd? It seems to have some pretty neat features, but I'm just not 100% sure I want to give up daemontools yet. Thanks! Brendon Colby

7 16

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 22 Jul '07

22 Jul '07

Bug summary ______________________ Summary for 2007-07-15 through 2007-07-22 Bugs opened: 12 Bugs closed: 2 Total open bugs: 995 (+10) |== Type Changes |== Priority Changes |== Component Changes |Defect: +2 |Highest: +7 |Conch: +1 |Enhancement: +1 |Normal: +3 |Core: +5 |Task: +7 |Foolscap: -1 |Mail: +1 |Trial: +1 |Vfs: +1 |Web2: +1 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2737] Two new features for twisted-dev (opened by washort) enhancement core http://twistedmatrix.com/trac/ticket/2737 [#2739] explain, document, and test trial's compatibility requirements for other test tools (opened by glyph) enhancement trial http://twistedmatrix.com/trac/ticket/2739 [#2741] twisted/words/test/test_domish.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun) task words http://twistedmatrix.com/trac/ticket/2741 [#2742] twisted/test/test_policies.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2742 [#2743] twisted/test/test_ssl.py shouldn't use setUpClass and tearDownClass (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2743 [#2744] twisted/test/test_pb.py should use setUp instead of setUpClass and not have any tear down (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2744 [#2745] twisted/test/test_newcred.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/2745 [#2746] twisted/mail/test/test_mail.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun) task mail http://twistedmatrix.com/trac/ticket/2746 ===== Normal ===== [#2735] twisted.vfs.adapters.ftp._FileToConsumerAdapter doesn't fully implement IConsumer (opened by therve) defect vfs http://twistedmatrix.com/trac/ticket/2735 [#2736] StreamProducer defer patch (opened by KyleJ) defect web2 http://twistedmatrix.com/trac/ticket/2736 [#2738] t.i.a.IPv4Address.host containing host name instead of ip address (opened by toidinamai) defect core http://twistedmatrix.com/trac/ticket/2738 [#2740] twisted/conch/test/test_cftp.py should use setUp instead of setUpClass and should not have a tearDown or tearDownClass (opened by exarkun) task conch http://twistedmatrix.com/trac/ticket/2740 Closed Bugs ______________________ ===== Highest ===== [#2387] Deprecate twisted.enterprise.row (opened by itamarst, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2387 ===== Normal ===== [#2731] foolscap: trivial fix (remove path separator) for test failing on windows (opened by strank, closed by warner, fixed) defect foolscap http://twistedmatrix.com/trac/ticket/2731 Ticket Lifetime Stats ---------------------- Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2746] twisted/mail/test/test_mail.py should use setUp and tearDown instead of setUpClass and tearDownClass (since 2007-07-20 10:43:11). Mean open ticket age: 489 days, 12:26:57.685024. Median: 375 days, 14:20:22.503115. Standard deviation: 392 days, 8:34:42.415196. Interquartile range: 472 days, 4:16:50. Mean time between ticket creation and ticket resolution: 119 days, 20:16:45.773628. Median: 14 days, 10:59:04. Standard deviation is 223 days, 8:29:32.610222. The interquartile range is 125 days, 7:42:53. Mean time spent in review: 58 days, 19:39:04.240986. Median: 3 days, 20:59:56. Standard deviation: 127 days, 9:56:33.556086. Interquartile range: 25 days, 4:28:51. Mean number of times a ticket is reviewed: 1.8. Median: 1 Standard deviation: 1.37784670354. Interquartile range: 1.

1 0

[Twisted-Python] How to receive a big stream data?
by steven wang 20 Jul '07

20 Jul '07

Dear all, I'm using the Twisted to write a socket server. I has a problem in the socket server. I could not receive a big size data when the data size is bigger then the connection buffersize. The protocol description: First, Client send data head to Server. Head Format: ABC,123456,XXX,XXX,XXX^ Second, Client send big data to Server, the big data is stream data. After Server received the big data and then save it to database. My problem source code : ================================================================================================ class MyProtocol(protocol.Protocol): is_big_data = False big_data = '' big_data_size = 0 def connectionMade(self): self.big_data = '' self.big_data_size = 0 self.is_big_data = False pass def dataReceived(self, data): if check_head_ok(data): # To receive the big size data if self.is_big_data: return # The big data will overflow the Twisted receive buffer size, so loop to receive data if is_big_size_data: big_data += data if big_data_size == len(big_data): # To save data to database .... pass def connectionLost(self, reason): self.big_data = '' self.big_data_size = 0 self.is_big_data = False pass def check_head_ok(self, head): if data.startswith('ABC'): values = data.split(',') self.big_data_size = int(values[1]) self.is_big_data = self.big_data_size > 0 return True else: return False ================================================================================================ If you know it, Please tell me. Thank you very much! :) Beat Regards, Steven. Wang

5 8

[Twisted-Python] schemas (in Zope and Foolscap)
by Stephen Waterbury 20 Jul '07

20 Jul '07

Hi Brian, I am in the process of converting some of my code to use zope.schema instead of a similar thing that I independently invented. Since I am also planning on using Foolscap, and since Foolscap has a schema module, I compared zope.schema and foolscap.schema, and to me they seem to have much overlap in design intent. In particular, in the zope.schema use cases ... http://wiki.zope.org/zope3/Zope3SchemasUseCases ... one of the cases cited is: Method uses schema for its arguments A more general case than above, any method can provide a schema to which its arguments should conform. This schema can be used to create a form or to validate arguments upon any call. ... which seems almost exactly the foolscap.schema use case. Moreover, zope.schema has "constraints" that are similar in concept to the ones in foolscap.schema. So I was wondering -- would it make sense to use zope.schema in Foolscap? I plan to use them in my code, so from my own selfish point of view that would be nice because it would unify everything relating to the schema concept, and I wouldn't have to have separate usages and imports of zope.schema and foolscap.schema. Although "There should be one -- and preferably only one -- obvious way to do it" doesn't necessarily -- or even that often -- apply to libraries, this might be a good case for applying it, for reasons somewhat similar those underlying twisted's adoption of zope.interface. A nice practical factor is that zope.schema has been packaged as a separate namespace module, similarly to zope.interface. I'd be interested to know your thoughts. Cheers, Steve

2 2

[Twisted-Python] Making a persistent SSH connection for a GUI
by Paul_S_Johnson＠mnb.uscourts.gov 20 Jul '07

20 Jul '07

I am not new to Python but am to Twisted. I am also no guru when it comes to complex OO in Python. I have a PythonCard app that's a dashboard-like app and have it opening child windows that open SSH connections to various servers. I see plenty of conch examples of "open, get and close" but I would like to keep the connection to the server persistent so avoid the necessary connection overhead each time I want to fetch info from a server from the parent window. Does anyone have an example of a persistent conch connection inside wxPython or similar GUI? I have scoured the web and come up empty handed though I found a few shreds that have part of the answer. I've tried to tweak the nonpersistent examples, but making my way through four layers of subclasses is a bit daunting. Any help much appreciated. Paul.

2 2

[Twisted-Python] twisted.plugin
by Vincent Bernat 19 Jul '07

19 Jul '07

Hi ! I am trying to use twisted.plugin with no luck. Here is how I do: I have a class IServicePlugin that inherits from Interface. In some package x.y.z (with empty __init__.py), I have a module w containing a class c inheriting of object and implementing IPlugin and IServicePlugin. To load plugins, here is what I do: import x.y.z as package for plugin in getPlugins(IServicePlugin, package): print repr(plugin) But I get nothing. How can I debug this ? twisted.plugin is filled with black magic for me. ;-) dropin.cache contains the string x.y.z.w but nothing about class c Thanks.

2 2

[Twisted-Python] VFS Status
by Andy Gayton 18 Jul '07

18 Jul '07

Hey guys, VFS has languished in a state of neglect for the last 2 years. What is currently in the repo is the output of 2 and a bit sprints - and some patches from canonical so it works for their immediate needs for launch pad. In retrospect - we got a bit excited that first sprint in Hobart, and we probably shouldn't have pushed what we had into the main twisted repo until I could show the commitment to get it into a more solid form. VFS trunk, currently defines a synchronous API. The API isn't complete. The filesystem backend uses its own utils, instead of the established twisted.python.filepath. Variable naming is all over the place, and the code quality is pretty poor - coming from a sprint were the focus was on how much could we get working before the weekend is out. The branch to rectify these issues is: branches/makeVFSAsync-1223-2 Things done so far in the branch: * The API is better defined and is async. There's still some debate over the API, particularly over meta-data. But I think its a good step forward and if merged the interface could be upgraded from obscenely unstable to unstable. * The memory and filesystem backends have been ported to meet the new API. * The filesystem implementation uses twisted.python.filepath for most operations. Things still left to do for the branch to be merged: * clean up the adapter tests: http://twistedmatrix.com/trac/ticket/1264 * port the existing adapters to use the new async backends * clean up variable/module naming to be more consistent * remove all of the now unused code * ensure all patches since branching from canonical are incorporated cheers, Andy.

2 1

[Twisted-Python] Twisted server / MS .NET client
by Arye 18 Jul '07

18 Jul '07

Hello and thanks for your input, In fact it seems that this was a WSDL definition issue. Newer .NET software prefers to use document/literal style and it looks like Twisted uses the older rpc/encoded style at least with soapPy. I will need to swich to ZSI for that. All the best, Arye.

1 0