[Twisted-Python] mktap dns is broken if you switch subnets
Hi. Lately, I've been playing with using "mktap dns" as a resolver for my laptop. Except, it's making me angry. It doesn't reload resolv.conf. My resolv.conf nameserver list is changed by DHCP about twice per day, atleast. In fact, it even loads the resolv.conf only in __init__, so the list gets stored in the tap file. I need to mktap, tap2deb and dpkg -i whenever I switch subnets! Here's a quick-and-stupid patch that makes it reload resolv.conf regularly, currently made to reread it every minute. This is "close enough". Actually, I want to make it get a notification from somewhere.. I definitely want this implemented. Please tell what direction should I go with the patch, and what do you think is missing from "production quality"? Index: twisted/names/client.py =================================================================== RCS file: /cvs/Twisted/twisted/names/client.py,v retrieving revision 1.31 diff -u -r1.31 client.py --- twisted/names/client.py 29 Mar 2003 18:10:14 -0000 1.31 +++ twisted/names/client.py 30 Apr 2003 15:56:00 -0000 @@ -30,6 +30,8 @@ import socket import os +import errno +import time # Twisted imports from twisted.python.runtime import platform @@ -52,6 +54,10 @@ protocol = None connections = None + resolv = None + resolv_last_read = 0 + resolv_read_interval = 60 + def __init__(self, resolv = None, servers = None, timeout = (1, 3, 11, 45)): """ @type servers: C{list} of C{(str, int)} or C{None} @@ -78,10 +84,9 @@ else: self.servers = servers - if resolv and os.path.exists(resolv): - self.parseConfig(resolv) + self.resolv = resolv - if not len(self.servers): + if not len(self.servers) and not resolv: raise ValueError, "No nameservers specified" self.factory = DNSClientFactory(self, timeout) @@ -99,20 +104,36 @@ d['connections'] = [] return d + def maybeParseConfig(self): + if self.resolv_last_read + self.resolv_read_interval < time.time(): + self.parseConfig() - def parseConfig(self, conf): - lines = open(conf).readlines() + def parseConfig(self): + try: + file = open(self.resolv) + except IOError, e: + if e.errno == errno.ENOENT: + return + else: + raise + + lines = file.readlines() + self.resolv_last_read = os.fstat(file.fileno()).st_mtime + file.close() + servers = [] for l in lines: l = l.strip() if l.startswith('nameserver'): - self.servers.append((l.split()[1], dns.PORT)) - log.msg("Resolver added %r to server list" % (self.servers[-1],)) + resolver = (l.split()[1], dns.PORT) + servers.append(resolver) + log.msg("Resolver added %r to server list" % (resolver,)) elif l.startswith('domain'): self.domain = l.split()[1] self.search = None elif l.startswith('search'): self.search = l.split()[1:] self.domain = None + self.dynamicServers = servers def pickServer(self): @@ -122,8 +143,15 @@ TODO: Weight servers for response time so faster ones can be preferred. """ - self.index = (self.index + 1) % len(self.servers) - return self.servers[self.index] + self.parseConfig() + if not self.servers and not self.dynamicServers: + return None + self.index = ((self.index + 1) + % (len(self.servers) + len(self.dynamicServers))) + if self.index < len(self.servers): + return self.servers[self.index] + else: + return self.dynamicServers[self.index - len(self.servers)] def connectionMade(self, protocol): @@ -155,7 +183,10 @@ if timeout is None: timeout = self.timeout address = self.pickServer() - d = self.protocol.query(address, queries, timeout[0]) + if address is not None: + d = self.protocol.query(address, queries, timeout[0]) + else: + d = defer.fail() d.addErrback(self._reissue, address, queries, timeout[1:]) return d @@ -186,7 +217,10 @@ @rtype: C{Deferred} """ if not len(self.connections): - host, port = self.pickServer() + address = self.pickServer() + if address is None: + return defer.fail() + host, port = address from twisted.internet import reactor reactor.connectTCP(host, port, self.factory) self.pending.append((defer.Deferred(), queries, timeout)) -- :(){ :|:&};:
On Wed, Apr 30, 2003 at 07:01:59PM +0300, Tommi Virtanen wrote:
Hi. Lately, I've been playing with using "mktap dns" as a resolver for my laptop. Except, it's making me angry.
It doesn't reload resolv.conf. My resolv.conf nameserver list is changed by DHCP about twice per day, atleast.
The "usability" of t.names could be a bit better, I agree.
In fact, it even loads the resolv.conf only in __init__, so the list gets stored in the tap file. I need to mktap, tap2deb and dpkg -i whenever I switch subnets!
Here's a quick-and-stupid patch that makes it reload resolv.conf regularly, currently made to reread it every minute. This is "close enough". Actually, I want to make it get a notification from somewhere..
It would be great if Twisted could speak FAM. I looked briefly at integrating the python wrapper, but didn't have time to go very in-depth.
I definitely want this implemented. Please tell what direction should I go with the patch, and what do you think is missing from "production quality"?
It basically looks good. One thing I'd like done differently is to have parseConfig() called at a different time. Possibly calling it as a result of connectionMade() further increases an already large latency on queries. If this were done in a callLater() loop, this would keep the parsing overhead out of the path of actual queries the client performs. As a short term solution, I think this is reasonable (but the point is moot if we wrap and use libfam). How's it sound to you?
[snip patch]
Jp -- A disciple of another sect once came to Drescher as he was eating his morning meal. "I would like to give you this personality test," said the outsider, "because I want you to be happy." Drescher took the paper that was offered him and put it into the toaster: "I wish the toaster to be happy, too." -- up 41 days, 15:04, 4 users, load average: 0.00, 0.00, 0.00
On Wed, Apr 30, 2003 at 02:43:57PM -0400, Jp Calderone wrote:
It would be great if Twisted could speak FAM. I looked briefly at integrating the python wrapper, but didn't have time to go very in-depth.
Well, it'll need a secondary mechanism anyway, as FAM is far from portable. Besides, FAM is a daemon. The actual kernel interface it uses is called "dnotify", and it goes like this: fd = open(".", O_RDONLY); fcntl(fd, F_SETSIG, SIGRTMIN); fcntl(fd, F_NOTIFY, DN_MODIFY|DN_CREATE|DN_MULTISHOT); and you'll get an SIGRTMIN every time the dir changes. Allocating an available signal sounds like this is something that should first be "twistedified", and not used raw. This might not even need any C wrapping, or at most exposing those DN_* and F_* constants. If you really want to use that API, and the DN_* constants aren't exported by standard python, it'd be trivial to add them to eunuchs. And fall back to statting the file every once in a while if eunuchs isn't installed or the kernel doesn't understand the fcntl.
I definitely want this implemented. Please tell what direction should I go with the patch, and what do you think is missing from "production quality"? It basically looks good. One thing I'd like done differently is to have parseConfig() called at a different time. Possibly calling it as a result of connectionMade() further increases an already large latency on queries. If this were done in a callLater() loop, this would keep the parsing overhead out of the path of actual queries the client performs.
Yeah, sure. I ended up doing it that way just because I couldn't remember whether callLaters etc are pickled or not, and make it resume from a .tap correctly, etc..
As a short term solution, I think this is reasonable (but the point is moot if we wrap and use libfam). How's it sound to you?
Okay, how about I make it check the file timestamp every n seconds via callLater, and commit that? Maybe tomorrow. -- :(){ :|:&};:
Jp Calderone <exarkun@intarweb.us> writes:
It would be great if Twisted could speak FAM. I looked briefly at integrating the python wrapper, but didn't have time to go very in-depth.
Personally, I find FAM's implementation annoying, because is uses sunrpc, and requires the "security-compromise-waiting-to-happen" portmap daemon be running. I wish they had implemented it with a unix socket instead, but it appears to be aimed at handling remote (NFS) directory changes as well as local ones. Something less server-heavy would seem more appropriate. The Linux mechanism is DNotify, and you can get at most of the functionality from the Python side. I have code in the BuildBot (buildbot.dnotify) which does the necessary Twisted glue, and I'd be happy to contribute that if it seems useable. Code is available from http://buildbot.sf.net . One problem with that module is that DNotify uses SIGIO (or other signal of your choice) to indicate the change, and Python's signal-delivery code doesn't pass the siginfo_t argument to the Python handler, which means multiple DNotify notifications for the same process all look the same. That buildbot code also falls back to polling every 10 seconds if the DNotify fcntl values aren't available (e.g. non-linux). cheers, -Brian
participants (3)
-
Brian Warner -
Jp Calderone -
Tommi Virtanen