[Twisted-Python] async hostname resolution
Diff for twisted/internet/base.py's resolve() function to make it perform lookups asynchronously using the new stuff in twisted/names/client.py. I removed the type argument, since it complicates implementation for other reactors, isn't really useful in 99% of cases, and never worked when set to anything other than A (1) anyway. Maybe it should be deprecated first for a while, but I doubt anyone's using it anyway so it may not matter. Jp -- <Greatnuts> someone gave me bad advice this morning <Greatnuts> told me to use twm <Greatnuts> had to reformat <Greatnuts> just to get it off -- 10:00pm up 21 days, 8:53, 6 users, load average: 0.00, 0.00, 0.00
On Fri, 22 Nov 2002 22:55:04 -0500 Jp Calderone <exarkun@meson.dyndns.org> wrote:
Diff for twisted/internet/base.py's resolve() function to make it perform lookups asynchronously using the new stuff in twisted/names/client.py.
Mmm. I don't like adding a dependency on an external package in twisted.internet. And this doesn't let people replace the name resolver with their own that does e.g. caching. And what if the reactor doesn't support UDP? I suggest instead a reactor.installResolver(r) method, where the default one stays as is using socket module (with the DNS type fix going in though). -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python, Twisted, Zope and Java consulting ***> http://VoteNoWar.org -- vote/donate/volunteer <***
On Fri, Nov 22, 2002 at 11:21:20PM -0500, Itamar Shtull-Trauring wrote:
On Fri, 22 Nov 2002 22:55:04 -0500 Jp Calderone <exarkun@meson.dyndns.org> wrote:
Diff for twisted/internet/base.py's resolve() function to make it perform lookups asynchronously using the new stuff in twisted/names/client.py.
Mmm. I don't like adding a dependency on an external package in twisted.internet. And this doesn't let people replace the name resolver with their own that does e.g. caching.
Oh, *that* was the kind of "pluggable" you and glyph were talking about. Okay, I'll try it that way (probably not 'til tomorrow though).
And what if the reactor doesn't support UDP?
I suggest instead a reactor.installResolver(r) method, where the default one stays as is using socket module (with the DNS type fix going in though).
Sounds good. Jp -- A sad spectacle. If they be inhabited, what a scope for misery and folly. If they be not inhabited, what a waste of space. -- Thomas Carlyle, looking at the stars -- 11:00pm up 21 days, 9:53, 4 users, load average: 0.00, 0.03, 0.01
On Fri, Nov 22, 2002 at 11:21:20PM -0500, Itamar Shtull-Trauring wrote:
On Fri, 22 Nov 2002 22:55:04 -0500 Jp Calderone <exarkun@meson.dyndns.org> wrote:
Diff for twisted/internet/base.py's resolve() function to make it perform lookups asynchronously using the new stuff in twisted/names/client.py.
Mmm. I don't like adding a dependency on an external package in twisted.internet. And this doesn't let people replace the name resolver with their own that does e.g. caching. And what if the reactor doesn't support UDP?
I suggest instead a reactor.installResolver(r) method, where the default one stays as is using socket module (with the DNS type fix going in though).
Why wouldn't we use the twisted.names resolver by default, if available? -- Christopher Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/
On Sat, Nov 23, 2002 at 01:06:43PM -0000, Moshe Zadka wrote:
On Sat, 23 Nov 2002, Christopher Armstrong <radix@twistedmatrix.com> wrote:
Why wouldn't we use the twisted.names resolver by default, if available?
/etc/hosts, for one.
And if it supported /etc/hosts? :) -- Christopher Armstrong << radix@twistedmatrix.com >> http://twistedmatrix.com/users/radix.twistd/
On Sat, 23 Nov 2002 14:54:16 +0100, Christopher Armstrong <radix@twistedmatrix.com> wrote:
And if it supported /etc/hosts? :)
/etc/hosts. Windows registry keys pointing at DNS servers. ZoneAlarm. Potentially there are wacky plug-ins for the system host resolution technique which use LDAP rather than DNS. (I don't know if any exist, but there's nothing to say they couldn't ^_^). Effectively replacing the system doman name lookup is tricky, especially on win32. The twisted.names resolver should probably be on by default in things like web servers where you've got to do lots of connections and name lookups in a daemon, though, so having installResolver(x) available for application code will be useful. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
On Sat, Nov 23, 2002 at 12:31:21PM -0600, Glyph Lefkowitz wrote:
And if it supported /etc/hosts? :)
/etc/hosts. Windows registry keys pointing at DNS servers. ZoneAlarm. Potentially there are wacky plug-ins for the system host resolution technique which use LDAP rather than DNS. (I don't know if any exist, but there's nothing to say they couldn't ^_^). Effectively replacing the system doman name lookup is tricky, especially on win32.
Note on mordern unixes: libnss (name service switch) allows to admin to use _whatever he wants_ to resolve host names. -- :(){ :|:&};:
On Thu, Nov 28, 2002 at 08:43:17AM +0200, Tommi Virtanen wrote:
[snip] Note on mordern unixes: libnss (name service switch) allows to admin to use _whatever he wants_ to resolve host names.
A quick google didn't turn up the answer: Are there python bindings to libnss? Jp -- <Ape> EludeR: what kind of distro you have? <EludeR> Ape: gentoo <Ape> EludeR: it seems it have bsd-style init? <EludeR> i dont know <EludeR> have no clue -- 12:00am up 26 days, 10:54, 4 users, load average: 0.02, 0.11, 0.09
Hi, Jp Calderone:
A quick google didn't turn up the answer: Are there python bindings to libnss?
"include socket". The standard gethost* library calls end up using NSS. -- Matthias Urlichs | noris network AG | http://smurf.noris.de/
Tommi Virtanen wrote:
On Sat, Nov 23, 2002 at 12:31:21PM -0600, Glyph Lefkowitz wrote:
And if it supported /etc/hosts? :)
/etc/hosts. Windows registry keys pointing at DNS servers. ZoneAlarm. Potentially there are wacky plug-ins for the system host resolution technique which use LDAP rather than DNS. (I don't know if any exist, but there's nothing to say they couldn't ^_^). Effectively replacing the system doman name lookup is tricky, especially on win32.
Note on mordern unixes: libnss (name service switch) allows to admin to use _whatever he wants_ to resolve host names.
And I've already pointed this out to exarkun since he's doing the work, but on Linux, glibc 2.2.4 and later expose an asynchronous name lookup interface already. Docs on it can be found here: http://people.redhat.com/drepper/asynchnl.pdf Windows has an asynch interface as well: http://msdn.microsoft.com/library/en-us/winsock/winsock/wsaasyncgethostbynam... - Bruce
On Sat, 23 Nov 2002, Christopher Armstrong <radix@twistedmatrix.com> wrote:
And if it supported /etc/hosts? :)
/etc/hosts is one example for "ways the local resolver can diverge from the DNS". In fact, with the GNU libc, you can dynamically program ways to override/complement name resolution. DJB went down the "I don't care about local resolution, I'll use DNS.". It's ugly and discorteous to the users :)
Hi, Jp Calderone:
I removed the type argument, since it complicates implementation for other reactors, isn't really useful in 99% of cases, and never worked when set to anything other than A (1) anyway. Maybe it should be deprecated first for a while, but I doubt anyone's using it anyway so it may not matter.
What -- you never look up records other than A? I've got a bunch of useful candidates here -- PTR, NS, SOA, CNAME, MX ... and, guess what, I'm going to need all of them (except MX ;-) early next year. -- Matthias Urlichs | noris network AG | http://smurf.noris.de/
On Sat, Nov 23, 2002 at 10:17:05AM +0100, Matthias Urlichs wrote:
Hi,
Hello :)
Jp Calderone:
I removed the type argument, since it complicates implementation for other reactors, isn't really useful in 99% of cases, and never worked when set to anything other than A (1) anyway. Maybe it should be deprecated first for a while, but I doubt anyone's using it anyway so it may not matter.
What -- you never look up records other than A?
I've got a bunch of useful candidates here -- PTR, NS, SOA, CNAME, MX ... and, guess what, I'm going to need all of them (except MX ;-) early next year.
All record types are supported ;) The question is just whether you should go through the reactor interface to make queries about them, or if you should use the "real" name lookup API instead. Essentially.. from twisted.internet import reactor from twisted.protocols import dns reactor.resolve('somehost.com', dns.MX).addCallback(whatever) vs from twisted.names import client client.lookupMailExchange('somehost.com').addCallback(whatever) I can't envision any circumstances where the core would want to do lookups other than of A records, so supporting them /without/ adding a dependancy of twisted.internet on twisted.names (something there are good reasons to want to avoid) seems like an unecessary burden on anyone else who wants to implement the reactor API. Jp -- 5:00pm up 22 days, 3:54, 3 users, load average: 0.00, 0.00, 0.00
participants (8)
-
Bruce Mitchener
-
Christopher Armstrong
-
Glyph Lefkowitz
-
Itamar Shtull-Trauring
-
Jp Calderone
-
Matthias Urlichs
-
Moshe Zadka
-
Tommi Virtanen