gethostbyname blocking

[marc wyburn]

On Apr 22, 5:19 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
> On Wed, 22 Apr 2009 08:08:51 -0700 (PDT), marc wyburn <marc.wyb... at googlemail.com> wrote:
> >Hi, I am writing an asynchronous ping app to check if 1000s of hosts
> >are alive very quickly.  Everything works extremely quickly unless the
> >host name doesn't have a DNS record.
>
> >when calling socket.gethostbyname if there is no record for the host
> >the result seems to block all other threads.  As an example I have 6
> >threads running and if I pass the class below a Queue with about 30
> >valid addresses with one invalid address in the middle the thread that
> >the exception occurs in seems to block the others.
>
> If your system doesn't have the gethostbyname_r function, then you can
> only make one gethostbyname_ex call at a time - a call to this function
> will block any other calls to this function, since it is not safe to
> call gethostbyname reentrantly.
>
> I'm not sure what the easiest way to determine whether Python has found
> gethostbyname_r or not on your system is.  The configure script used to
> build Python will probably tell, but I doubt you have that lying around.
> You could just assume this is the case, since you're observing behavior
> consistent with it. ;)
>
> An alternate approach would be to use a DNS library which doesn't have
> this restriction.  For example, Twisted includes a DNS library which
> is not implemented in terms of threads.  There are also a couple other
> Python DNS client libraries around.
>
> This isn't /quite/ the same as using gethostbyname, though, since
> gethostbyname is controlled by system configuration in various ways,
> and may do more than just DNS lookups (in fact, it may even be
> configured not to use DNS at all!).  But it's pretty close, and may
> be close enough for your purposes.
>
> Jean-Paul

Thanks for the quick response.  I'm using Win32, I don't think there
is a non-blocking gethostbyname avail.  I managed to find some info on
the net but most of it was old and Nix related and I was hoping it was
a bug that had been fixed and I was making a mistake with my threads.
I wrote a simlar script that uses multiprocess and that seemed to have
less problems but I will look at http://www.dnspython.org/.

Thanks, Marc.