[Twisted-Python] Twisted names client.getHostbyName dual-stack operation
Hello, I've tried to use Twisted names.client.getHostByName() method to resolve some addresses for my UDP based app. By default getHostByName() always returns IPv6 address. I don't always have end-to-end IPv6 connectivity, so it often causes communication to fail. I've found a solution to override this behavior with monkey patching [1]. After Twisted 13.2 announcement I've checked the new HostnameEndpoint implementation to find possible solution (it seems to be dedicated to solve similar problems, but for TCP-based protocols). However it seems to use deferToThread(), and built-in Python getaddrinfo(). My questions: 1. Is using deferToThread() a preferred solution? 2. Does someone know a less hacky solution to get IPv4 addresses with getHostbyName()? Best Regards Maciej Wasilak [1] http://stackoverflow.com/questions/13820175/can-twisted-names-client-be-forc...
On Jan 6, 2014, at 12:26 AM, Maciej Wasilak <wasilak@gmail.com> wrote:
After Twisted 13.2 announcement I've checked the new HostnameEndpoint implementation to find possible solution (it seems to be dedicated to solve similar problems, but for TCP-based protocols). However it seems to use deferToThread(), and built-in Python getaddrinfo().
Sadly, the algorithm HostnameEndpoint uses to determine which host to actually send traffic to depends on TCP connection establishment; if you're doing UDP you're on your own with this.
My questions: 1. Is using deferToThread() a preferred solution?
You kinda have to use it. getaddrinfo is actually standardized in an RFC, and that RFC defines no asynchronous version of this API. So if you want proper 6-or-4 behavior, deferToThread(getaddrinfo) is about as well as you can do for the general case. (Now, in specific cases you can use the twisted.names DNS client and avoid the trip out to your operating system and also out to a thread, but this is for somewhat specialized configurations, only if the getaddrinfo approach is not scaling for you.) -glyph
On 06/01/14 08:26, Maciej Wasilak wrote:
Hello,
I've tried to use Twisted names.client.getHostByName() method to resolve some addresses for my UDP based app. By default getHostByName() always returns IPv6 address. I don't always have end-to-end IPv6 connectivity,
I'm not sure if t.n.client implements the same stuff that getaddrinfo does, namely RFC 6724 address selection rules and behaviour, and in particular if it has an equivalent of the AI_ADDRCONFIG flag.
After Twisted 13.2 announcement I've checked the new HostnameEndpoint implementation to find possible solution (it seems to be dedicated to solve similar problems, but for TCP-based protocols). However it seems to use deferToThread(), and built-in Python getaddrinfo().
My questions: 1. Is using deferToThread() a preferred solution?
IMO *all* applications should *always* use getaddrinfo() for resolution of user-supplied names. It really really annoys me when people decide to re-invent it - this is hard to get right, and apps like Chrome having their own implementation precludes things like DHCP-based 6724 tables in the future. So yes, use "deferToThread" and "getaddrinfo". Note that you should not, in general, specify the address family; let the AI_ADDRCONFIG flag sort that out for you.
IMO *all* applications should *always* use getaddrinfo() for resolution of user-supplied names. It really really annoys me when people decide to re-invent it - this is hard to get right, and apps like Chrome having their own implementation precludes things like DHCP-based 6724 tables in the future.
So yes, use "deferToThread" and "getaddrinfo". Note that you should not, in general, specify the address family; let the AI_ADDRCONFIG flag sort
Hello, 6 sty 2014 12:50 "Phil Mayers" <p.mayers@imperial.ac.uk> napisał(a): that out for you.
Thank you very much for your answers. DeferToThread works fine, I just had the (wrong) impression that names is the preferred solution. Best Regards Maciej Wasilak
On Jan 6, 2014, at 3:47 AM, Phil Mayers <p.mayers@imperial.ac.uk> wrote:
On 06/01/14 08:26, Maciej Wasilak wrote:
Hello,
I've tried to use Twisted names.client.getHostByName() method to resolve some addresses for my UDP based app. By default getHostByName() always returns IPv6 address. I don't always have end-to-end IPv6 connectivity,
I'm not sure if t.n.client implements the same stuff that getaddrinfo does, namely RFC 6724 address selection rules and behaviour, and in particular if it has an equivalent of the AI_ADDRCONFIG flag.
After Twisted 13.2 announcement I've checked the new HostnameEndpoint implementation to find possible solution (it seems to be dedicated to solve similar problems, but for TCP-based protocols). However it seems to use deferToThread(), and built-in Python getaddrinfo().
My questions: 1. Is using deferToThread() a preferred solution?
IMO *all* applications should *always* use getaddrinfo() for resolution of user-supplied names. It really really annoys me when people decide to re-invent it - this is hard to get right, and apps like Chrome having their own implementation precludes things like DHCP-based 6724 tables in the future.
So yes, use "deferToThread" and "getaddrinfo". Note that you should not, in general, specify the address family; let the AI_ADDRCONFIG flag sort that out for you.
I have a minor quibble - all applications deployed to other people’s hardware ought to always use getaddrinfo, because the essence of GAI is that it respects local configuration of the platform. No matter how “hard” you try to get it “right”, you can’t ever know what the platform is doing with hostname resolution because it simply isn’t exposed any other way, let alone any asynchronous way.[1] However, if you’re building a big service to run in the cloud, or on a VPN, on specially-tuned machines that you configured, using twisted.names can be more scalable and allow you to react more reasonably to high load situations. Or, at least, it will give you a level of control so that you can tune it, since just doing potentially blocking platform stuff in a thread is not amenable to tuning. -glyph [1]: Actually it is sometimes exposed, for example with platform-specific APIs like <https://developer.apple.com/library/mac/documentation/CoreFoundation/Reference/CFHostRef/Reference/reference.html> and <http://msdn.microsoft.com/en-us/library/windows/desktop/ms741522(v=vs.85).aspx>. But you know, not on, like, Linux.
On 08/01/14 23:05, Glyph Lefkowitz wrote:
However, if you’re building a big service to run in the cloud, or on a VPN, on specially-tuned machines that you configured, using twisted.names can be more scalable and allow you to react more reasonably to high load situations. Or, at least, it will give you a level of control so that you can tune it, since just doing potentially blocking platform stuff in a thread is not amenable to tuning.
Sure, agreed - I folded those concerns into the "user-supplied names" bit ;o) Using a resolver directly is totally appropriate in certain use-cases, and t.names is a good choice if you're using Twisted[1] but I think of it like "should I use UDP instead of TCP for my protocol" - if you have to ask, the answer is no. (In fairness to the Chrome guys, they've probably got the chops to reimplement it properly and are doing enough innovative stuff in Chrome that having an in-browser resolver is reasonable, so my dig at them is probably unfair ;o) [1] Modulo a couple of rather big holes that I know people are already working on, like source-port randomisation for Kaminsky-resistance, and so on.
participants (4)
-
Glyph -
Glyph Lefkowitz
-
Maciej Wasilak -
Phil Mayers