making socket.getaddrinfo use cached dns
I use cached dns lookups with pdnsd on my ubuntu machine to speed up web access as regular lookups can take 15-30 seconds. However, python's mechanize and urllib etc use socket.getaddrinfo, which seems not to be using dns cacheing or taking a long time because of ipv6 lookups. In either case, I subsequent access to the same site to be fast and not require lengthy calls to getaddrinfo. How can I get python to correctly use cached dns lookups and ipv4 only (at least in those cases where it is appropriate). As mentioned here: http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to... It seems that many python libraries are hardcoded to look for both ipv6 and ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps the devs should carry over these optional arguments to higher level libraries like mechanize, urllib, httplib etc.
Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Thu, Jul 07, 2011 at 12:57:59PM -0400, high bandwidth wrote:
I use cached dns lookups with pdnsd on my ubuntu machine to speed up web access as regular lookups can take 15-30 seconds. However, python's mechanize and urllib etc use socket.getaddrinfo, which seems not to be using dns cacheing or taking a long time because of ipv6 lookups. In either case, I subsequent access to the same site to be fast and not require lengthy calls to getaddrinfo. How can I get python to correctly use cached dns lookups and ipv4 only (at least in those cases where it is appropriate).
As mentioned here: http://stackoverflow.com/questions/2014534/force-python-mechanize-urllib2-to... It seems that many python libraries are hardcoded to look for both ipv6 and ipv4. Since ipv6 lookups are extremely slow at least for some users, perhaps the devs should carry over these optional arguments to higher level libraries like mechanize, urllib, httplib etc.
Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd@phdru.name> wrote:
Hello.
We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs);
Well, it seems this post is about adding a new feature isn't it? Cheers, fijal
On Sat, Jul 16, 2011 at 04:12:45PM +0200, Maciej Fijalkowski wrote:
On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd@phdru.name> wrote:
We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs);
Well, it seems this post is about adding a new feature isn't it?
I don't think so. The original post is about problems with pdnsd - could be just a local configuration problem, and has nothing with Python. The original post is about rolling back getaddrinfo and returning to gethostby* - certainly not a new feature. That's how I understand the original post. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sun, Jul 17, 2011 at 12:12 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
On Thu, Jul 7, 2011 at 7:59 PM, Oleg Broytman <phd@phdru.name> wrote:
Hello.
We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs);
Well, it seems this post is about adding a new feature isn't it?
Not really - the key question was "How can I get python to correctly use cached dns lookups and ipv4 only (at least in those cases where it is appropriate)." This isn't the place to ask that question (particularly since it's the wrong question - the real question is why the IPv6 lookups are taking so long. Since we just call into the C standard library for name resolution, whether it's slow or fast is an OS configuration problem). The latter part (very indirectly) made a feature suggestion via the reference off to the SO question. However, hardcoding the *app* to be IPv4 only really isn't a good workaround for IPv6 resolution taking a long time to fail at the OS level. Exposing those flags would encourage people to do exactly that, and that would be a *really* bad idea (it's unfortunate enough that PEP 3144 stalled, or we might have had better support for manipulating IPv6 addresses in the standard library by now. We really shouldn't make things even worse by making it easy for developers with broken IPv6 setups to switch off IPv6 support entirely). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, 17 Jul 2011 00:59:17 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:
Exposing those flags would encourage people to do exactly that, and that would be a *really* bad idea
Making DNS resolution configurable (for example by allowing the user to supply their own resolution function) in the stdlib's network APIs doesn't sound like a really bad idea to me. Of course, a "no_ipv6" flag would scale poorly and would therefore be a bad API.
We really shouldn't make things even worse by making it easy for developers with broken IPv6 setups to switch off IPv6 support entirely).
I don't think moralist arguments should have a weight when deciding which features we add. If developers want to introduce bugs or limitations in their software they will always be able to do it. Regards Antoine.
On Sat, Jul 16, 2011 at 8:26 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
I don't think moralist arguments should have a weight when deciding which features we add. If developers want to introduce bugs or limitations in their software they will always be able to do it.
Actually when designing language features or APIs, what you call "moralist arguments" take place all the time. Personally I don't think there's anything "moral" about wanting to design an API that reduces common mistakes, and API design should always take expected behavior of programmers into account. Experienced developers have a huge store of information about that in their head. Anyway, even before the word "moralist" was used this thread would have been better on python-ideas. -- --Guido van Rossum (python.org/~guido)
participants (6)
-
Antoine Pitrou -
Guido van Rossum -
high bandwidth -
Maciej Fijalkowski -
Nick Coghlan -
Oleg Broytman