[Python-bugs-list] [ python-Bugs-539175 ] resolver not thread safe

noreply@sourceforge.net noreply@sourceforge.net
Thu, 04 Apr 2002 13:08:10 -0800


Bugs item #539175, was opened at 2002-04-04 01:54
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=539175&group_id=5470

Category: Threads
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: dustin sallings (dustin)
Assigned to: Nobody/Anonymous (nobody)
Summary: resolver not thread safe

Initial Comment:
I've got an application that does SNMP monitoring and
has a thread listening with SimpleXMLRPCServer for
remote control.  I noticed the XMLRPC listener logging
an incorrect address while snmp jobs were processing:

sw1.west.spy.net - - [04/Apr/2002 01:16:37] "POST /RPC2
HTTP/1.0" 200 -
localhost.west.spy.net - - [04/Apr/2002 01:16:43] "POST
/RPC2 HTTP/1.0" 200 -

sw1 is one of the machines that is being queried, but
the XMLRPC requests are happening over localhost.

gethostbyname() and gethostbyaddr() both return static
data, thus they aren't reentrant.

As a workaround, I copied socket.py to my working
directory and added the following to it:

try:
    import threading
except ImportError, ie:
    sys.stderr.write(str(ie) + "\n")

# mutex for DNS lookups
__dns_mutex=None
try:
    __dns_mutex=threading.Lock()
except NameError:
    pass

def __lock():
    if __dns_mutex!=None:
        __dns_mutex.acquire()
    
def __unlock():
    if __dns_mutex!=None:
        __dns_mutex.release()
    
def gethostbyaddr(addr):
    """Override gethostbyaddr to try to get some thread
safety."""
    rv=None
    try:
        __lock()
        rv=_socket.gethostbyaddr(addr)
    finally:
        __unlock()
    return rv

def gethostbyname(name):
    """Override gethostbyname to try to get some thread
safety."""
    rv=None
    try:
        __lock()
        rv=_socket.gethostbyname(name)
    finally:
        __unlock()
    return rv


----------------------------------------------------------------------

>Comment By: dustin sallings (dustin)
Date: 2002-04-04 13:08

Message:
Logged In: YES 
user_id=43919

The XMLRPC request is clearly being logged as coming from my
cisco switch when it was, in fact, coming from localhost.

I can't find any clear documentation, but it seems that on
at least some systems gethostbyname and gethostbyaddr
reference the same static variable, so having separate locks
for each one (as seen in socketmodule.c) doesn't help
anything.  It's not so much that they're not reentrant, but
you can't call any combination of the two of them at the
same time.  Here's some test code:

#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <assert.h>

int main(int argc, char **argv) {
    struct hostent *byaddr, *byname;
    unsigned int addr;
    struct sockaddr *sa = (struct sockaddr *)&addr;

    addr=1117120483;

    byaddr=gethostbyaddr(sa, sizeof(addr), AF_INET);
    assert(byaddr);
    printf("byaddr:  %s\n", byaddr->h_name);

    byname=gethostbyname("mail.west.spy.net");
    assert(byname);
    printf("byname:  %s\n", byname->h_name);

    printf("\nReprinting:\n\n");

    printf("byaddr:  %s\n", byaddr->h_name);
    printf("byname:  %s\n", byname->h_name);
}


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-04-04 12:06

Message:
Logged In: YES 
user_id=21627

I'm not sure what problem you are reporting. Python does not
attempt to invoke gethostbyname from two threads
simultaneously; this is prevented by the GIL.

On some systems, gethostname is reentrant (in the
gethostname_r incarnation); Python uses that where
available, and releases the GIL before calling it.

So I fail to see the bug.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=539175&group_id=5470