Mailman 3 February 2004 - Twisted

[Twisted-Python] p2p, gnutella
by stephan 16 Feb '04

16 Feb '04

I am looking forward to add p2p functionality to my app. Does anybody know what the current status of twisted's gnutella implementation is? There seems to have been a semi-finished implemention in 1.0.2alpha4 which I can't find in 1.1.1 anymore. I might also be willing to implement missing parts but I would need to get a short briefing on how things are. thanks, _stephan

1 0

[Twisted-Python] twisted.names; tinydns file format patch
by Andy Gayton 15 Feb '04

15 Feb '04

For anyone else who, like me, can't read bind formatted files :) It seems to work well enough even if twisted.names and tinydns behave slightly differently. Supports all common data lines from: http://cr.yp.to/djbdns/tinydns-data.html, but doesn't support wildcards, timestamps or client location filtering plus one data file can only contain one SOA. Generated against Twisted 1.1.2alpha2 using diff -urN cheers, Andy. diff -urN ./names/authority.py /usr/local/python/lib/python2.3/site-packages/twisted/names/authority.py --- ./names/authority.py Fri Sep 19 14:37:13 2003 +++ /usr/local/python/lib/python2.3/site-packages/twisted/names/authority.py Sun Feb 15 21:40:34 2004 @@ -226,13 +226,12 @@ L.append(line.split()) return filter(None, L) - def parseLines(self, lines): TTL = 60 * 60 * 3 ORIGIN = self.origin - + self.records = {} - + for (line, index) in zip(lines, range(len(lines))): if line[0] == '$TTL': TTL = dns.str2time(line[1]) @@ -321,3 +320,97 @@ # print 'rdata is ', rdata self.addRecord(owner, ttl, type, domain, cls, rdata) + + +class TinyDNSAuthority(FileAuthority): + """An Authority that loads Tiny DNS configuration files""" + + def loadFile(self, filename): + self.origin = os.path.basename(filename) + '.' # XXX - this might suck + lines = open(filename).readlines() + lines = self.stripComments(lines) + lines = self.collapseContinuations(lines) + self.parseLines(lines) + + + def stripComments(self, lines): + return [ + x for x in [ + a.find('#') == -1 and a or a[:a.find('#')] for a in [ + b.strip() for b in lines + ] + ] if x != '' + ] + + + def collapseContinuations(self, lines): + return lines + + + def parseLines(self, lines): + # http://cr.yp.to/djbdns/tinydns-data.html + + self.records = {} + + for line in lines: + # .fqdn:ip:x:ttl:timestamp:lo + if line[0] == "." : + (fqdn, ip, x, ttl) = line[1:].split(":") + if x.find('.') == -1 : + x = "%s.ns.%s" % ( x, fqdn ) + + soa_record = dns.Record_SOA( mname = x, + rname = "hostmaster.%s" % ( fqdn ), + serial = time.time(), + refresh = 16384, + retry = 2048, + expire = 1048576, + minimum = 2560, + ttl = ttl ) + + self.addRecord( fqdn, soa_record ) + self.soa = (fqdn, soa_record) + + self.addRecord( x, dns.Record_NS( fqdn, ttl ) ) + + if ip != '': + self.addRecord( x, dns.Record_A( ip, ttl ) ) + + # &fqdn:ip:x:ttl:timestamp:lo + elif line[0] == "&" : + (fqdn, ip, x, ttl) = line[1:].split(":") + if x.find('.') == -1 : + x = "%s.ns.%s" % ( x, fqdn ) + + self.addRecord( x, dns.Record_NS( fqdn, ttl ) ) + self.addRecord( x, dns.Record_A( ip, ttl ) ) + + # =fqdn:ip:ttl:timestamp:lo + elif line[0] == "=" : + (fqdn, ip, ttl) = line[1:].split(":") + self.addRecord( fqdn, dns.Record_A( ip, ttl ) ) + (a,b,c,d) = ip.split('.') + self.addRecord( '%s.%s.%s.%s.in-addr.arpa' % (d,c,b,a), dns.Record_PTR( fqdn, ttl ) ) + + # +fqdn:ip:ttl:timestamp:lo + elif line[0] == "+" : + (fqdn, ip, ttl) = line[1:].split(":") + self.addRecord( fqdn, dns.Record_A( ip, ttl ) ) + + # @fqdn:ip:x:dist:ttl:timestamp:lo + elif line[0] == "@" : + (fqdn, ip, x, dist, ttl) = line[1:].split(":") + if x.find('.') == -1 : + x = "%s.mx.%s" % ( x, fqdn ) + if dist == '' : + dist = '0' + + self.addRecord( x, dns.Record_A( ip, ttl ) ) + self.addRecord( fqdn, dns.Record_MX( dist, x ) ) + + else : + raise NotImplementedError, "Data line '%s' not supported yet" % line[0] + + + def addRecord( self, domain, record ) : + self.records.setdefault(domain.lower(), []).append(record) diff -urN ./names/tap.py /usr/local/python/lib/python2.3/site-packages/twisted/names/tap.py --- ./names/tap.py Thu Dec 4 20:54:03 2003 +++ /usr/local/python/lib/python2.3/site-packages/twisted/names/tap.py Sun Feb 15 07:00:15 2004 @@ -50,6 +50,7 @@ usage.Options.__init__(self) self['verbose'] = 0 self.bindfiles = [] + self.tinydnsfiles = [] self.zonefiles = [] self.secondaries = [] @@ -60,6 +61,14 @@ raise usage.UsageError(filename + ": No such file") self.zonefiles.append(filename) + + def opt_tinydnszone(self, filename): + """Specify the filename of a Tiny DNS syntax zone definition""" + if not os.path.exists(filename): + raise usage.UsageError(filename + ": No such file") + self.tinydnsfiles.append(filename) + + def opt_bindzone(self, filename): """Specify the filename of a BIND9 syntax zone definition""" if not os.path.exists(filename): @@ -95,12 +104,21 @@ except Exception, e: traceback.print_exc() raise usage.UsageError("Invalid syntax in " + f) + for f in self.bindfiles: try: self.zones.append(authority.BindAuthority(f)) except Exception, e: traceback.print_exc() raise usage.UsageError("Invalid syntax in " + f) + + for f in self.tinydnsfiles: + try: + self.zones.append(authority.TinyDNSAuthority(f)) + except Exception, e: + traceback.print_exc() + raise usage.UsageError("Invalid syntax in " + f) + for f in self.secondaries: self.svcs.append(secondary.SecondaryAuthorityService(*f)) self.zones.append(self.svcs[-1].getAuthority())

1 0

[Twisted-Python] Where are stored the filenames of the files uploaded using www Form?
by Frantisek Fuka 15 Feb '04

15 Feb '04

I want to use Twisted on my server so that I can upload generic files using WWW interface. (I use only the basic Twisted Web bones, no Woven etc.) When I upload the files, the session.args dictionary contains the names of form elements and the contents of the uploaded files. But how can I find out the filenames of the uploaded files, so that "picture7.jpg" could be automatically stored as "picture7.jpg"? Thanks -- Frantisek Fuka (yes, that IS my real name) (and it's pronounced "Fran-tjee-shek Foo-kah") ---------------------------------------------------- My E-mail: fuka(a)fuxoft.cz My Homepage: http://www.fuxoft.cz My ICQ: 2745855

1 0

[Twisted-Python] twisted.internet.error.CannotListenError
by Peter Stradinger 15 Feb '04

15 Feb '04

Hi, I'm fairly new to Twisted and trying to get a twisted reactor to work with wxPython. I'm trying the perspective broker examples, and when I try to do: def OnFileitems0Menu(self, event): self.Close(true) reactor.stop() It shuts down the program fine, but then when I try to run it again I get: twisted.internet.error.CannotListenError: Couldn't listen on any:8800: (10048, 'Address already in use'). As an error. I can manually stop the process and then it will work again, but I don't know how to release the port when I shut down the program. Am I missing something obvious? A search of the archives hasn't shown me any way out. Peter __________________________________ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html

1 0

[Twisted-Python] twisted.names; authority query
by Andy Gayton 15 Feb '04

15 Feb '04

Hi there, I've extended twisted.names to support tinydns data files. names isn't responding with authoritive answers the way I was expecting it to though. names responds that it is authoritive for my name server, but not for anything else. This is the tinydns file I'm using: ----------------------------------- .cablelounge.com::ns1.cablelounge.com:259200 =cablelounge.com:69.10.144.170:86400 +www.cablelounge.com:69.10.144.170:86400 +miles.cablelounge.com:69.10.144.170:86400 +twisted.cablelounge.com:203.113.212.32:900 +ns1.cablelounge.com:69.10.144.170:86400 @cablelounge.com:69.10.144.170:mail.cablelounge.com:0:86400 ------------------------------------ names says that it is authoritive for ns1.cablelounge.com, but nothing else. tinydns responds as authoritive for all of the above names: #> dig @cablelounge.com cablelounge.com I ran a standard pyzone file through names and got the same result .. so I'm not sure where I'm going wrong .. . Here's what I did with the pyzone file: zones: ----------------------- zone = [ SOA( "fred.com", mname = "ns1.fred.com", rname = "hostmaster.fred.com", serial = 2004021501, refresh = "1H", retry = "1H", expire = "1H", minimum = "1H" ), A("fred.com", "192.168.10.10"), NS("ns1.fred.com", "fred.com"), A("ns1.fred.com", "192.168.10.10"), ] ------------------------ #> mktap dns --pyzone=zones --port 5053 #> twistd -f dns #> dig -p 5053 @cablelounge.com fred.com ; <<>> DiG 9.2.1 <<>> -p 5053 @cablelounge.com fred.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48786 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;fred.com. IN A ;; ANSWER SECTION: fred.com. 3600 IN A 192.168.10.10 ;; Query time: 2 msec ;; SERVER: 69.10.144.170#5053(cablelounge.com) ;; WHEN: Sun Feb 15 10:01:29 2004 ;; MSG SIZE rcvd: 42 #> dig -p 5053 @cablelounge.com ns1.fred.com ; <<>> DiG 9.2.1 <<>> -p 5053 @cablelounge.com ns1.fred.com ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63056 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;ns1.fred.com. IN A ;; ANSWER SECTION: ns1.fred.com. 3600 IN A 192.168.10.10 ;; AUTHORITY SECTION: ns1.fred.com. 3600 IN NS fred.com. ;; ADDITIONAL SECTION: fred.com. 3600 IN A 192.168.10.10 ;; Query time: 2 msec ;; SERVER: 69.10.144.170#5053(cablelounge.com) ;; WHEN: Sun Feb 15 10:02:24 2004 ;; MSG SIZE rcvd: 76 twisted.names is showing as authoritive for ns1.fred.com, but not fred.com. Has anyone had a similiar experience with twisted.names? What am I overlooking? regards, Andy.

1 0

Re: [Twisted-Python] pb server how to get client IP address
by Brian Warner 14 Feb '04

14 Feb '04

Martin Stenhård <stenis(a)stenhard.net> writes: > > Yes this works find in my server for my admin interface (simple telenet > bases) that use protocol.ServerFactory() and then use LineReceiver. > > But how do I use getPeer() in a Perspective Broker server for example > http://www.twistedmatrix.com/documents/examples/pbbenchserver.py Ah, yes, that is trickier. You need to get access to the Broker object (which is a Protocol subclass), and called methods are not generally given access to that. The Perspective/ViewPoint -style stuff gives invoked methods access to some initial Perspective object, but even that doesn't know who the Broker is. The method which invokes your perspective_* method knows, but it isn't telling. I poked around a bit and haven't found any way to get at it. If the client has given you access to one if its own objects (perhaps as the 'mind' reference in the realm.requestAvatar() call), then you can use it to get the Broker: mind.broker.transport.getPeer() Other than that, I don't see how to do it. The details of the connection are a little *too* abstracted out. Maybe someone else has an idea? sorry, -Brian PS: and no, inspect.stack()[1][0].f_locals['broker'] doesn't count..

3 2

[Twisted-Python] web messenger with ICQ and MSN support
by Nicol�s Ech�niz 14 Feb '04

14 Feb '04

Hi everybody, This is my first message to the list so I'll briefly introduce myself. I'm Nicolás Echániz, from Buenos Aires, Argentina. I've been programing python for 4 or 5 years now and one of my projects is this instant messenger (www.EcMess.com), which is a web based messenger. It's based on Sam Rushing's Medusa framework, runs on a Linux server and uses MySQL for data storage. It works really nice. The fact is that I'd like to add ICQ and MSN support (it's a much requested feature) and I've been thinking that the best way might be to rewrite my server using Twisted as a replacement framework for Medusa. Now, I've been looking at the notes on 1.1.1 release and found that MSNP 8 is the supported protocol at this time, but as far as I know, clients using MSNP 8 can't communicate with the MSN network since protocol 9 was released. So, my question is: Are there any plans to support MSNP 9 in the near future? I know that Gaim has updated it's protocol support to v.9. Do you mantain any relation with their development team? Or do you at twisted-matrix hack the protocol on your own without reusing other project's experience? If I may be of any help in this matter, please let me know. I'm not a C programmer, but then I don't know if your MSNP implementation is written in C and wrapped or written in pure Python. If the latter is true, then I might be able to help. I thank you all in advance, Nicolás Echániz www.EcMess.com

3 2

[Twisted-Python] basic auth example with db backend
by Stephen Waterbury 13 Feb '04

13 Feb '04

This is a somewhat less trivial (and more robust) example than the "cred" example I posted earlier. BasicAuthResource is lifted from moshez's example in bug 393 with some typos corrected and a notification to the user if something gets majorly screwed up, so the auth form doesn't continue to pop up in that case. UserDbCredChecker borrows radix's approach from a db auth he wrote for a spike we did a while back. Again, I'd appreciate any suggestions/comments from twisted gurus -- especially if you see any security problems, etc.! Cheers, Steve # Twisted, the Framework of Your Internet # Copyright (C) 2001 Matthew W. Lefkowitz # # This library is free software; you can redistribute it and/or # modify it under the terms of version 2.1 of the GNU Lesser General Public # License as published by the Free Software Foundation. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA """ This is a basic auth example with a db-based checker. Note: this example assumes that PostgreSQL is installed and a database named 'userdb' has been created. If it has, this script will set up the users table, populate it with a test user, and drop the table when it exits, so it can be modified and run again. """ import sys import md5 from pyPgSQL import PgSQL from twisted.cred import error from twisted.cred import portal from twisted.cred import checkers from twisted.cred import credentials from twisted.enterprise import adbapi from twisted.internet import reactor from twisted.python import components from twisted.python import failure from twisted.python import log from twisted.web import resource from twisted.web import server schema = """ CREATE TABLE users ( username varchar(64) PRIMARY KEY, password varchar(32) ) """ testdata = """ INSERT INTO users VALUES ( 'Zaphod', '%s' ) """ % md5.md5('s3krit').hexdigest() class AvatarResource: __implements__ = resource.IResource def __init__(self, id): self.id = id def render(self, request): return """<html> Hello, <b>%s</b> ...<br> Welcome to the Restaurant at the End of the Universe! </html> """ % self.id def logout(self): pass class Realm: __implements__ = portal.IRealm def requestAvatar(self, avatarId, mind, *interfaces): av = AvatarResource(avatarId) return resource.IResource, av, av.logout class UserDb: def __init__(self): # db setup and teardown are synchronous; no big deal. conn = PgSQL.connect(database='userdb') curs = conn.cursor() curs.execute(schema) curs.execute(testdata) conn.commit() conn.close() self.pool = adbapi.ConnectionPool('pyPgSQL.PgSQL', database='userdb') def teardown(self): conn = PgSQL.connect(database='userdb') curs = conn.cursor() curs.execute('DELETE FROM users') curs.execute('DROP TABLE users') conn.commit() conn.close() class UserDbCredChecker: __implements__ = (checkers.ICredentialsChecker,) credentialInterfaces = (credentials.IUsernamePassword,) def __init__(self, userdb): self.db = userdb def _verifyCreds(self, creds): """ Verify credentials, returning the username if successful. """ res = self.db.pool.runQuery( """SELECT * from users where username = %s and password = %s""", (creds.username, md5.md5(creds.password).hexdigest())) def succeeded(r): try: if len(r) == 1: return creds.username elif len(r) == 0: return failure.Failure(error.UnauthorizedLogin()) except: # Houston, we have a problem ... # should probably have user notify admin e = 'User database appears to be borken' raise ValueError, e def failed(e): raise LookupError, e return res.addCallbacks(succeeded, failed) def requestAvatarId(self, c): if not c.username or not c.password: return failure.Failure(error.UnauthorizedLogin()) return self._verifyCreds(c) class BasicAuthResource: __implements__ = resource.IResource isLeaf = True httpRealm="Restaurant at the End of the Universe" def __init__(self, portal): self.portal = portal def render(self, request): username, password = request.getUser(), request.getPassword() creds = credentials.UsernamePassword(username, password) d = self.portal.login(creds, None, resource.IResource) def cb((_, r, logout)): request.notifyFinish().addBoth(lambda _: logout()) result = resource.getChildForRequest(r, request).render(request) if result != server.NOT_DONE_YET: request.write(result) request.finish() def eb(f): try: f.trap(error.LoginFailed, error.UnauthorizedLogin) request.setResponseCode(401) request.setHeader("www-authenticate", 'Basic realm="%s"' % self.httpRealm) request.finish() except: log.msg('Error during auth: %s' % f) MALFUNCTION = """<html><h2>Something untoward happened!</h2> <hr>Please notify the sysadmin: <a href="mailto:sysadmin@host.domain"> sysadmin(a)host.domain</a><hr></html>""" request.write(MALFUNCTION) request.finish() d.addCallbacks(callback=cb,errback=eb) d.addErrback(log.err) return server.NOT_DONE_YET r = Realm() p = portal.Portal(r) d = UserDb() c = UserDbCredChecker(d) p.registerChecker(c) root.putChild('login', BasicAuthResource(p)) log.startLogging(sys.stdout) site = server.Site(root) reactor.listenTCP(1999, site) reactor.run() d.teardown()

1 0

[Twisted-Python] Windows Programming Info
by Michael Schneider 13 Feb '04

13 Feb '04

Thanks to all for the help with win32eventreactor and spawnprocess. I came across a great windows package called wmi (for microsoft's Windows Managment Interface). It gives you very low level access to your windows system. It uses win32. Here is the URL: http://tgolden.sc.sabren.com/python/wmi.html Here is some sample code to run Thanks for the help on this, I will post my final code for the compute node class ---------------------------------------------------------- import wmi c = wmi.WMI () diskList = [x for x in c.Win32_LogicalDisk (DriveType=3) ] procList = [proc for proc in c.Win32_Process () ] for x in range(10): for disk in diskList: print disk for process in procList: print process ---------------------------------------------------------- Note: to kill a job, you can call terminate on a process object result = process.Terminate () ------------------------------------------ disk output: instance of Win32_LogicalDisk { Caption = "G:"; Compressed = FALSE; CreationClassName = "Win32_LogicalDisk"; Description = "Local Fixed Disk"; DeviceID = "G:"; DriveType = 3; FileSystem = "FAT32"; FreeSpace = "9506979840"; MaximumComponentLength = 255; MediaType = 12; Name = "G:"; Size = "12572344320"; SupportsDiskQuotas = FALSE; SupportsFileBasedCompression = FALSE; SystemCreationClassName = "Win32_ComputerSystem"; SystemName = "BDSWXP1"; VolumeDirty = FALSE; VolumeName = "LOCAL DISK"; VolumeSerialNumber = "ED293D74"; }; ------------------------------------------------- process output instance of Win32_Process { Caption = "System Idle Process"; CreationClassName = "Win32_Process"; CSCreationClassName = "Win32_ComputerSystem"; CSName = "BDSWXP1"; Description = "System Idle Process"; Handle = "0"; HandleCount = 0; KernelModeTime = "7163013906250"; Name = "System Idle Process"; OSCreationClassName = "Win32_OperatingSystem"; OSName = "Microsoft Windows XP Professional|C:\\WINDOWS|\\Device\\Harddi sk0\\Partition1"; OtherOperationCount = "0"; OtherTransferCount = "0"; PageFaults = 1; PageFileUsage = 0; ParentProcessId = 0; PeakPageFileUsage = 0; PeakVirtualSize = "0"; PeakWorkingSetSize = 20480; Priority = 0; PrivatePageCount = "0"; ProcessId = 0; QuotaNonPagedPoolUsage = 0; QuotaPagedPoolUsage = 0; QuotaPeakNonPagedPoolUsage = 0; QuotaPeakPagedPoolUsage = 0; ReadOperationCount = "0"; ReadTransferCount = "0"; SessionId = 0; ThreadCount = 1; UserModeTime = "0"; VirtualSize = "0"; WindowsVersion = "5.1.2600"; WorkingSetSize = "20480"; WriteOperationCount = "0"; WriteTransferCount = "0"; };

1 0

Re: [Twisted-Python] pb server how to get client IP address
by Brian Warner 13 Feb '04

13 Feb '04

Martin Stenhård <stenis(a)stenhard.net> writes: > > I have a question on how to get ip address of the clients that connects to > the server. From the Protocol object, the .transport offers a pair of methods called getHost() and getPeer(). So something like this should work: ==== BEGIN from twisted.internet import reactor, protocol class MyProtocol(protocol.Protocol): def connectionMade(self): print "connection from", self.transport.getPeer() f = protocol.ServerFactory() f.protocol = MyProtocol reactor.listenTCP(9111, f) reactor.run() ==== END When I run this on my system, and connect with 'nc localhost 9111', I get: % /tmp/p.py connection from ('INET', '127.0.0.1', 38002) The value it returns depends upon the kind of transport, but TCP connections will give you a tuple of ('INET', hostname, port), where 'hostname' is a dotted-quad IP address. I'll add this to the FAQ. cheers, -Brian

2 1