[ python-Bugs-583975 ] gethostbyaddr lag

Fri Dec 29 08:38:05 CET 2006

Bugs item #583975, was opened at 2002-07-19 11:41
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=583975&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
>Status: Closed
>Resolution: Works For Me
Priority: 5
Private: No
Submitted By: Jason R. Mastaler (jasonrm)
Assigned to: Nobody/Anonymous (nobody)
Summary: gethostbyaddr lag

Initial Comment:
For more info, also see
http://mail.python.org/pipermail/python-list/2002-July/113706.html

Perl's gethostbyaddr doesn't seem to have this problem
as shown
below.  Should I report this in the bug tracker?

$ time perl -MSocket -lwe 'print +(gethostbyaddr
inet_aton(&quot;datavortex.net&quot;), AF_INET)[0]'
datavortex.net

real    0m0.063s
user    0m0.050s
sys     0m0.010s

$ time python2 -c 'from socket import * ; print
gethostbyaddr(&quot;datavortex.net&quot;)[0]'
datavortex.net

real    0m20.176s
user    0m0.070s
sys     0m0.020s

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2006-12-28 23:38

Message:
Logged In: YES 
user_id=33168
Originator: NO

Not fully diagnosed, no.  I suspect (based on reading the comments here)
that the other programs didn't use the threaded version where python did. 
That's the only reasonable explanation I can come up with.  I'm closing
this since we could never reproduce.  I don't recall ever seeing another
similar bug report.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-22 05:11

Message:
Logged In: YES 
user_id=11375
Originator: NO

Was the cause of this bug ever diagnosed?  Should this report remain open,
or be closed?

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2003-07-24 20:03

Message:
Logged In: YES 
user_id=85984

This problem has cropped up again for another
TMDA user, so we still have this bug in Python.

In short, just as with the previous report, this
user sees a 20 second delay with 
socket.gethostbyaddr() on a hostname which
all other tools (nslookup, host, mail progs, etc)
look up instantaneously.

In other words, Python is the only app on his
system with this problem.  I had him try 
Python 2.3c1 to no avail.

As with the previous user, he is running Linux
(Redhat):

Linux sparge 2.4.20-18.7 #1 Thu May 29 06:51:53 EDT 2003
i686 unknown

glibc-2.2.5-43

Any other information I should provide or any
diagnostics I can have the user run?

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2002-11-02 20:40

Message:
Logged In: YES 
user_id=85984

The problem was under Python 2.2.

Though now the user reports that he can no longer
reproduce the problem, despite not having done any
Python upgrades or system configuration changes.

We might just have to chuck this one up to 
FM (Funny Magic).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-02 13:14

Message:
Logged In: YES 
user_id=33168

Jason, still with us?  What version of python were you
having the problem with? 2.1.x? 2.2.x?  Do you have the
problem with 2.2?  Have you looked at patch #604210
(http://python.org/604210)?  Can you test that?

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2002-09-06 17:20

Message:
Logged In: YES 
user_id=85984

Still having the problem, but I'm unsure how to
proceed.

nslookup and host both return instantly when querying
datavortex.net.

Only Python seems to exhibit this problem, but it 
still could be a system misconfiguration.  This is the
only time I've ever seen/heard of this behavior.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-09-06 15:23

Message:
Logged In: YES 
user_id=33168

Jason, are you still having this problem?  Do you have
anything to report?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-07-22 15:37

Message:
Logged In: YES 
user_id=33168

Looking at the output, the problem is definitely in
gethostbyaddr_r().
This is what python calls from Modules/socketmodule.c
socket_gethostbyaddr().

Notice:  the first attempt to send to the first DNS server
&quot;207.69.188.185&quot; looks like it fails after 5 seconds.  DNS
#2 is attempted &quot;207.69.188.186&quot;, this send also takes 5
seconds, back to #1 ...

The poll is being done in gethostbyaddr_r() (ie, libc).  If
you want to break in a debugger, gethost...() should be
around line 2216 in python 2.2.  You can also put prints in.

As for why perl doesn't have the same problem, it could be
that perl doesn't use the same library call
(gethostbyaddr_r), or attempts to read from /etc/hosts
before contacting the DNS server.  I'm just guessing.  In
order to debug this further, you will probably need the
python sources.  What happens if you do host datavortex.net
(you could also try nslookup)?  Does that return immediately
or wait?

----------------------------------------------------------------------

Comment By: Data Vortex (datavortex)
Date: 2002-07-22 12:15

Message:
Logged In: YES 
user_id=141979

Running strace python2 -c 'from socket import * ; print 
getfqdn()', I can see a pause of several seconds during the 
output of:

socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sin_family=AF_INET, sin_port=htons(53), 
sin_addr=inet_addr(&quot;207.69.188.185&quot;)}}, 28) = 0
send(3, 
&quot;)\351\1\0\0\1\0\0\0\0\0\0\ndatavortex\3net\0\0\34\0\1&quot;, 32, 0)

= 32
gettimeofday({1027364850, 154497}, NULL) = 0
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
recvfrom(3, 
&quot;)\351\201\200\0\1\0\0\0\1\0\0\ndatavortex\3net\0\0\34\0&quot;..., 
1024, 0, {sin_family=AF_INET, sin_p
ort=htons(53), sin_addr=inet_addr(&quot;207.69.188.185&quot;)}}, [16]) 
= 95
close(3)                                = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sin_family=AF_INET, sin_port=htons(53), 
sin_addr=inet_addr(&quot;207.69.188.185&quot;)}}, 28) = 0
send(3, &quot;)\352\1\0\0\1\0\0\0\0\0\0\ndatavortex\3net\3net\0&quot;...,

36, 0) = 36
gettimeofday({1027364850, 169212}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 5000)  = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
connect(4, {sin_family=AF_INET, sin_port=htons(53), 
sin_addr=inet_addr(&quot;207.69.188.186&quot;)}}, 28) = 0
send(4, &quot;)\352\1\0\0\1\0\0\0\0\0\0\ndatavortex\3net\3net\0&quot;...,

36, 0) = 36
gettimeofday({1027364855, 172955}, NULL) = 0
poll([{fd=4, events=POLLIN}], 1, 5000)  = 0
send(3, &quot;)\352\1\0\0\1\0\0\0\0\0\0\ndatavortex\3net\3net\0&quot;...,

36, 0) = 36
gettimeofday({1027364860, 182024}, NULL) = 0
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
recvfrom(3, 
&quot;&lt;\310\201\202\0\1\0\0\0\0\0\0\ndatavortex\3net\3net\0&quot;..., 
1024, 0, {sin_family=AF_INET, sin_por
t=htons(53), sin_addr=inet_addr(&quot;207.69.188.185&quot;)}}, [16]) = 
36
poll([{fd=3, events=POLLIN}], 1, 5000)  = 0
send(4, &quot;)\352\1\0\0\1\0\0\0\0\0\0\ndatavortex\3net\3net\0&quot;...,

36, 0) = 36
gettimeofday({1027364865, 191273}, NULL) = 0

The full output of this command is availible here:  
http://datavortex.net/out.txt

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-07-20 14:09

Message:
Logged In: YES 
user_id=33168

That's a good idea Jack.  On Linux, you can use strace.  On
Solaris, I believe the program is called truss.

$ strace python ...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-07-20 14:03

Message:
Logged In: YES 
user_id=45365

Here's a few ideas on debugging this:
- Easiest would be if you have a system call tracer. Attach it to the
process and see during what system call the 20 second wait occurs. Then
look at it's parameters and see whether there's anything fishy. Or whether
you can recreate the problem in C.
- If you don't have a tracer first split the program in two steps: the
implicit gethostbyname() step and the gethostbyaddr() step. see which one
is the problem. Run this step under the debugger and see where the delay
is. Again, try to recreate the problem.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-07-20 09:02

Message:
Logged In: YES 
user_id=33168

Does this happen consistently (every run) or only the first
time?
Works fine for me (Linux).  What OS are you on?

----------------------------------------------------------------------

Comment By: Jason R. Mastaler (jasonrm)
Date: 2002-07-19 14:46

Message:
Logged In: YES 
user_id=85984

I'm not ruling it out that it could be a local configuration
problem,
it's just that Python is the only application experiencing
this delay.

We've gone through the network configuration and everything
seems
sound, so I'm not sure what more to do.

/etc/resolv.conf is fine -- all entries are operational
nameservers.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-07-19 14:42

Message:
Logged In: YES 
user_id=45365

This smells like a local configuration bug on your system, on my system it
works fine (0.220u 0.110s 0:01.85 17.8%     0+0k 84+10io 0pf+0w). Look in
your /etc/resolv.conf (or similar file for your platform) to see that there
aren't any non-existing hosts listed there.

Although, of course, that doesn't explain why perl has no delay...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=583975&group_id=5470