Mailman 3 October 2007 - Twisted

[Twisted-Python] twisted.names.client.getHostByName Error when DNS Server return Auth NS Server
by wangmm 14 Oct '07

14 Oct '07

There is a bug in the extractRecord function of common.py. And I open a ticket in http://twistedmatrix.com/trac/ticket/2850 ---------------------------------------------------------------------------------------------------- Python Version:2.5 Twisted Version:2.5 Names Version:0.4 When we got a resolver and call getHostByname, perhaps DNS server can return CNAME, Auth NS Server Name and Auth NS Server IP Address. So extractRecord function will extract the Auth NS Server and send the DNS request to the Auth NS Server. But there is a bug in the extractRecord function. Code: # No answers, but maybe there's a hint at who we should be asking about this for r in answers: if r.type == dns.NS: from twisted.names import client r = client.Resolver(servers=[(str( r.payload.name), dns.PORT)]) return r.lookupAddress(str(name) Generally, the Auth NS Server is the Domain name, and it's IP contained in the addition fields. If we get the NS Server by r.payload.name, we will get the DNS Server Domain, so the Domain as server will pass to client.queryUDPas DNS Server Address. UDP.write will complaint that warnings.warn("Please only pass IPs to write(), not hostnames "+addr[0] <http://twistedmatrix.com/trac/changeset/0>, DeprecationWarning? <http://twistedmatrix.com/trac/wiki/DeprecationWarning>, stacklevel=2). In addition, self.socket.sendto(datagram, addr) will call socket.gethostbyname(addr), but this call is block. I think we should use the Auth NS Server IP to replace the Domain. The patch code as fellow: Any comment is welcome! # No answers, but maybe there's a hint at who we should be asking about this for r in answers: if r.type == dns.NS: from twisted.names import client server = str(r.payload.name) for s in answers: if s.type==dns.A and str(s.name)==server : server = socket.inet_ntop(socket.AF_INET, s.payload.address) break r = client.Resolver(servers=[(server, dns.PORT)])

1 0

[Twisted-Python] Regarding "add" method in microdom.lmx
by Raj kumar 14 Oct '07

14 Oct '07

Hi every one, This is raj.. I am new to twisted python. I have created a xml and want to add some content to it but i am unable to edit the xml file.. I tried in this way..... .............................................................. Method(self, X,Y,Z,xmlfile=None) xmlfile = xmlfile or self.xmlfile . . jump = microdom.lmx('jumping') jump.add("jumpat", xjump=X, yjump=Y) save_xml(jump.node,xmlfile) then the xml file is showing <jumping> </jumping> but not the jumpat tag and content ............................................................... if i try in this below manner ...................................... Method(self, X,Y,Z,xmlfile=None) xmlfile = xmlfile or self.xmlfile . . jump = microdom.lmx('jumping') jump.add("jumpat") save_xml(jump.node,xmlfile) then xml file is showing <jumping> <jumpat> </jumpat> </jumping> .................................. Can any one help me how to add those X,Y,Z values in that? I wanted the out put to be stored as below <jumping> <jumpat x=value,y=value,z=value> </jumpat> <jumpat x=value,y=value,z=value> </jumpat> </jumping> Thank you in advance. --------------------------------- Unlimited freedom, unlimited storage. Get it now

1 0

[Twisted-Python] UDP with multiple connections
by Simon Pickles 14 Oct '07

14 Oct '07

Ok, UDP is new to me, so please be gentle.... I've got a server authentication app which needs to accept many client connections, but also dispatch messages to a master server. Am I best using unconnected UDP? do i then have to deal with each received datagram by checking which host is has come from and acting accordingly? thanks Simon

7 13

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 13 Oct '07

13 Oct '07

Bug summary ______________________ Summary for 2007-10-07 through 2007-10-14 Bugs opened: 5 Bugs closed: 3 Total open bugs: 1040 (+2) |== Type Changes |== Priority Changes |== Component Changes |Defect: +0 |Highest: +0 |Core: +1 |Enhancement: +1 |High: -1 |Trial: +1 |Task: +1 |Normal: +2 |Low: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2849] A reentrant addCallbacks on a Deferred adds to mid-chain (opened by Peaker) defect core http://twistedmatrix.com/trac/ticket/2849 ===== Normal ===== [#2845] twisted.internet.thread._putResultInDeferred should be public (opened by therve) enhancement core http://twistedmatrix.com/trac/ticket/2845 [#2847] Get rid of all "API Stability: unstable" markers (opened by radix) task core http://twistedmatrix.com/trac/ticket/2847 [#2848] typos in doc/core/man/tap2{deb,rpm}.1 (opened by exarkun) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/2848 ===== Low ===== [#2846] trial coverage option should include tracing in threads (opened by therve) enhancement trial http://twistedmatrix.com/trac/ticket/2846 Closed Bugs ______________________ ===== Highest ===== [#2457] failed reactor selection is reported poorly by trial and twistd (opened by exarkun, closed by therve, fixed) enhancement core http://twistedmatrix.com/trac/ticket/2457 ===== High ===== [#411 ] Deferreds are good at blowing the stack (opened by etrepum, closed by Peaker, invalid) defect core http://twistedmatrix.com/trac/ticket/411 ===== Normal ===== [#2848] typos in doc/core/man/tap2{deb,rpm}.1 (opened by exarkun, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/2848 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2849] A reentrant addCallbacks on a Deferred adds to mid-chain (since 2007-10-10 11:25:51). Mean open ticket age: 539 days, 13:55:11.952859. Median: 440 days, 3:14:16.977859. Standard deviation: 403 days, 6:49:39.251191. Interquartile range: 504 days, 16:10:28. Mean time between ticket creation and ticket resolution: 122 days, 14:09:56.608352. Median: 14 days, 16:22:15. Standard deviation is 226 days, 22:06:10.127155. The interquartile range is 130 days, 2:23:34. Mean time spent in review: 67 days, 20:17:30.264549. Median: 4 days, 9:08:37. Standard deviation: 149 days, 7:51:46.848091. Interquartile range: 31 days, 6:31:05. Mean number of times a ticket is reviewed: 1.95150115473. Median: 1 Standard deviation: 1.58950187867. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 10 unique ticket reporters 7 unique ticket reviewers 2 unique ticket resolvers In the last 24 weeks, 70 unique ticket reporters 16 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 152 unique ticket reporters 18 unique ticket reviewers 26 unique ticket resolvers

1 0

[Twisted-Python] Running commands (ssh) from a GUI client
by Raúl Gómez C. 11 Oct '07

11 Oct '07

Hi list, this is my first post :p I'm new to Twisted/Conch and I was wondering how can I modify the sshsimpleclient.py in order to run several commands, on a user request mode, without the need to authenticate just before every command, just like a normal interactive ssh session. This is for a GUI front end I'm writing (PythonCard) that will execute some benchmarks on a remote server. Thanks in advance... Raul

6 13

[Twisted-Python] stdio.StandardIO, ServerProtocol and Services.
by kgi 09 Oct '07

09 Oct '07

Hi all. I'm trying to write a "terminal" component, wrapped in a service.Service, to drop into an application framework so that I can use it to build interactive command-line tools. I've almost, but not quite, got it working. I know about manhole, but I want to write a simple command-based syntax, rather than exposing a python namespace and interpreter. I could also just go with a telnet or ssh interface, but that would require an additional step (the actual telnet or ssh). There's also stdiodemo.py, but that doesn't offer readline-like functionality, which would be a Very Good Thing. (Besides, I'm really curious as to why this approach is not working!) The code below is a minimal stripped-down example of what I've got so far through frantic grepping and glimpseing through the Twisted codebase. (although there are some fragmentary holdovers from various experiments; ignore these). At the moment, the command handler (the lineReceived() method) just echoes the argument with some '+' signs prepended. The termios stuff comes from twisted/conch/stdio.py, via a mailing list post saying something to the effect of, "this is the minimal amount you have to do in order to hook up your terminal to your process's stdio"; I don't have the ML link handy right now, sorry. Note that whereas stdio.py passes a Protocol argument to ServerProtocol, I've created an intermediate class, CLIServerProtocol, which has a protocolFactory attribute. An examination of twisted/conch/insults/insults.py shows that this should be equivalent. For the curious bystanders, HistoricRecvLine is derived as follows: HistoricRecvLine -> RecvLine -> TerminalProtocol ... and TerminalProtocol implements things like connectionMade, etc. Here's the behaviour I see: With the code as is, I can type in lines and get a reply from lineReceived. However, none of the cursor keys or other things for which there is special code in twisted/conch/recvline.py work; I just get control codes all over the terminal. If I uncomment the "reactor.run()", however, the behaviour changes completely, and becomes much closer to what I expect: keypresses like Home, cursor keys, etc, are honoured, as well as the history. However, Ctrl-C merely loses the connection, and the terminal hangs and I have to kill the twistd process externally. The fact that adding reactor.run() (which shouldn't be needed in a Service, right?) implies to me that the reactor isn't starting up properly without it, and I can't see why. Any advice would be greatly appreciated. Ricky -- import os, tty, sys, termios from twisted.application import service from twisted.internet import reactor, stdio, protocol, defer from twisted.python import failure, reflect, log from twisted.protocols import basic from twisted.application import internet from twisted.conch.insults import insults from twisted.conch.manhole import ColoredManhole # from twisted.conch.stdio import ConsoleManhole from twisted.application import service from twisted.conch import recvline class CLIProtocol ( recvline.HistoricRecvLine ): service = None def connectionMade ( self ): recvline.HistoricRecvLine.connectionMade ( self ) self.keyHandlers [ '\x01' ] = self.handle_HOME self.keyHandlers [ '\x03' ] = self.handle_QUIT self.keyHandlers [ '\x1a' ] = self.handle_QUIT def connectionLost ( self, reason ): log.msg ( "Connection Lost" ) def handle_QUIT ( self ): self.terminal.loseConnection() def lineReceived ( self, line ): self.terminal.write ( '+++' + line ) class CLIServerProtocol ( insults.ServerProtocol ): protocolFactory = CLIProtocol class CLIService ( service.Service ): def startService ( self ): fd = sys.__stdin__.fileno() oldSettings = termios.tcgetattr ( fd ) tty.setraw ( fd ) try: p = CLIServerProtocol() stdio.StandardIO ( p ) # reactor.run() finally: termios.tcsetattr ( fd, termios.TCSANOW, oldSettings ) os.write ( fd, "\r\x1bc\r" ) return service.Service.startService ( self ) ###################################################################### # Create the application service hierarchy. ###################################################################### application = service.Application ( 'cliapp' ) cs = CLIService() cs.setServiceParent ( application )

5 10

Re: [Twisted-Python] How to switch users in SSH session.
by Ross.McKerchar＠sophos.com 09 Oct '07

09 Oct '07

Paul_S_Johnson at mnb.uscourts.gov Paul_S_Johnson at mnb.uscourts.gov Thu Sep 27 16:00:14 EDT 200: > After much wrangling and a small miracle I have managed to write an object > that fetches the output of three commands in the order given through an > SSH connection. How come I cannot switch users? Some of the information I > need can only be fetched through a root account and security is such that > I cannot log in remotely from a root account but must switch once logged > in from an account with less than root privileges. I've been trying something similar. As you've probably found out you cant just write in the password to su - you'll get an error "Standard in must be a tty". Basically I think su needs a shell to work. Consequently, before sending your password, you'll need to request a shell. I would've been utterly clueless about this if it wasn't for the post @ http://twistedmatrix.com/pipermail/twisted-python/2007-July/015793.html, from which I managed to extract the crucial code: term = 'ansi' winsz = struct.pack('4H', 80, 100, 80, 100) winSize = struct.unpack('4H', winsz) ptyReqData = session.packRequest_pty_req(term, winSize, '') self.conn.sendRequest(self, 'pty-req', ptyReqData) self.conn.sendRequest(self, 'shell', '') Sticking this in your SSHChannel.channelOpen method will allow you to get a shell. Unfortunately you then have to interact with the session in an expect-like manner which turns the whole procedure into one big easily broken hack. Basically I just write "su -" and wait for my method dataReceived to return "password:" at which point I write in the password. Another point to mention is that the ssh session is still not root, the root functionality is being provided by the shell instance. Consequently any new channels opened up will not be root so you cant easily use this method to scp file's that are only readable for root. You can of course cat from the shell session but then you have issues with line wrapping and reliably figuring out when the file ends and your shell prompt begins. I looked into trying to do some setreuid/setuid magic in the hope that I could issue a command from my root shell prompt that would elevate the whole ssh process to root allowing all my channels root priviliges. However I soon got well out of my depth and decided it wasn't possible, at least by me :) So, as far as I can see, it's not possible, if your server doesn't allow root access to open up a channel that has root priviliges. The only way to do it is in the aformentioned messy way via a shell & su (which I gave up on when my regular expression took up two lines on my screen and still didn't reliable match half the shell prompts it needed to). I would love somebody to prove me wrong and let me know of a nice way round this. Of course - these problems are nothing to do with twisted and all to do with the way ssh works. -ross -- Ross McKerchar Systems Analyst, Sophos Tel: 01235 559933 Web: http://www.sophos.com Sophos - security and control Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.

1 0

Re: [Twisted-Python] Returning a Deferred as a result from another Deferred
by Jean-Paul Calderone 09 Oct '07

09 Oct '07

On Fri, 5 Oct 2007 19:00:51 -0700, Ryan Fugger <rfugger(a)gmail.com> wrote: >On 10/5/07, Jonathan Lange <jml(a)mumak.net> wrote: >> On 10/6/07, Ryan Fugger <rfugger(a)gmail.com> wrote: >> > Returning a Deferred as a result from another Deferred seems to be >> > disallowed by an assertion at the start of the callback chain. Why is >> > this? Right now I have worked around this limitation by wrapping my >> > Deferred in a list and then unwrapping it in the callback. >> > >> >> No it's not. I can guarantee this. >> >> What's the content of the AssertionError that you are getting? > >No content in the error. Line 238 of twisted/internet/defer.py (first >line in Deferred.callback) is: > >assert not isinstance(result, Deferred) > >That's pretty explicit in disallowing Deferreds... I'm working with >the version 2.5 release. You've misread the code. Jonathan is correct. The rest of this thread seems to go off on some tangent about threads which may or may not be relevant to whatever problem you're trying to solve. What led you to be reading this code in the first place? Glyph did make a good point, though. You shouldn't ever have a Deferred in a thread other than the reactor thread. You shouldn't make them, you shouldn't add callbacks to them, you shouldn't call them back. It might be legitimate to get one from the reactor thread, hold it for a while, then send it back to the reactor thread, but I can't think of a case where this would really be very useful. Jean-Paul > >Ryan > >_______________________________________________ >Twisted-Python mailing list >Twisted-Python(a)twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >

2 2

[Twisted-Python] Marking as review
by Jonathan Lange 08 Oct '07

08 Oct '07

Hello guys, Not a big deal, but is it still a good idea to mark tickets as "HIGHEST" priority when we put them up for review? 'kenaan' seems to be doing a good enough job of reminding people about review tickets as is, and I could use the extra mouse-clicks (I'm saving!) jml

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 06 Oct '07

06 Oct '07

Bug summary ______________________ Summary for 2007-09-30 through 2007-10-07 Bugs opened: 5 Bugs closed: 2 Total open bugs: 1038 (+3) |== Type Changes |== Priority Changes |== Component Changes |Defect: +3 |Highest: -2 |Core: +0 |High: +1 |Release Management: +1 |Normal: +3 |Trial: -1 |Low: +1 |Web: +2 |Web2: +1 New / Reopened Bugs ______________________ ===== High ===== [#2843] twisted.web2.log.FileAccessLoggingObserver should take a file like object instead of a logPath. (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/2843 ===== Normal ===== [#2841] [PATCH] Add a redirect limit to HTTPPageGetter in twisted/web/client.py (opened by alexis) defect web http://twistedmatrix.com/trac/ticket/2841 [#2842] [PATCH] Handle bad formatted headers in lineRecieved() in twisted/web/http.py (opened by alexis) defect web http://twistedmatrix.com/trac/ticket/2842 [#2844] Missing parens in multiexception (opened by fumanchu) defect core http://twistedmatrix.com/trac/ticket/2844 ===== Low ===== [#2840] fails to build on Mac OS X with QuickTime 6 headers (opened by zooko) defect release management http://twistedmatrix.com/trac/ticket/2840 Closed Bugs ______________________ ===== Highest ===== [#2837] TreeReporter setup/teardown error handling broke (opened by pjd, closed by therve, fixed) defect trial http://twistedmatrix.com/trac/ticket/2837 [#1192] twisted.test.test_policies calls reactor.iterate() (opened by exarkun, closed by therve, fixed) defect core http://twistedmatrix.com/trac/ticket/1192 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2844] Missing parens in multiexception (since 2007-10-05 12:22:50). Mean open ticket age: 535 days, 4:48:39.014318. Median: 436 days, 3:18:46.639559. Standard deviation: 403 days, 8:09:45.372209. Interquartile range: 504 days, 4:36:19. Mean time between ticket creation and ticket resolution: 121 days, 20:41:54.745053. Median: 14 days, 13:40:23. Standard deviation is 224 days, 23:12:37.447753. The interquartile range is 128 days, 12:08:19. Mean time spent in review: 66 days, 19:49:44.881401. Median: 4 days, 9:08:37. Standard deviation: 147 days, 6:13:52.498708. Interquartile range: 29 days, 10:44:36. Mean number of times a ticket is reviewed: 1.94431554524. Median: 1 Standard deviation: 1.58638608769. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 12 unique ticket reporters 7 unique ticket reviewers 2 unique ticket resolvers In the last 24 weeks, 73 unique ticket reporters 16 unique ticket reviewers 14 unique ticket resolvers In the last 48 weeks, 155 unique ticket reporters 18 unique ticket reviewers 26 unique ticket resolvers

1 0