gethostbyname blocking

[Jean-Paul Calderone]

On Thu, 23 Apr 2009 15:16:56 +0200, Piet van Oostrum <piet at cs.uu.nl> wrote:
>>>>>> marc wyburn <marc.wyburn at googlemail.com> (MW) wrote:
>
>>MW> Hi, I am writing an asynchronous ping app to check if 1000s of hosts
>>MW> are alive very quickly.  Everything works extremely quickly unless the
>>MW> host name doesn't have a DNS record.
>
>>MW> when calling socket.gethostbyname if there is no record for the host
>>MW> the result seems to block all other threads.  As an example I have 6
>>MW> threads running and if I pass the class below a Queue with about 30
>>MW> valid addresses with one invalid address in the middle the thread that
>>MW> the exception occurs in seems to block the others.
>
>What you could do is have two Queues, one with host names, and one with
>results from gethostbyname. And an additional thread which gets from the
>first queue, calls gethostbyname and puts the results in the second queue.
>The ping threads then should get from the second queue. The lookup
>thread could even do caching of the results if you often have to repeat
>the pings for thew same host.

You still end up limited to a single host lookup at a time with this
approach.  Using a name lookup API which doesn't have this limitation
will probably produce significantly better results.

Jean-Paul