Threading problem

[Patrick Maupin]

On Apr 25, 2:55 pm, sdistefano <sdistef... at gmail.com> wrote:
> I have the following issue:
>
> My program runs a thread called the MainThread, that loops trough a
> number of URLs and decides when it's the right time for one to be
> fetched.  Once a URL has to be fetched, it's added to a Queue object,
> where the FetchingThread picks up and does the actual work. Often,
> URLs have to be fetched with frequencies of 100ms, so the objects will
> get added to the queue repeatedly. Even though it takes more than
> 100ms to get the URL and process it, ideally what would happen is:
> ms0: Request 1 is sent
> ms100: request 2 is sent
> ms150: request 1 is processed
> ms200: request 3 is sent
> ms250: request 2 is processed
>
> and so on... The problem is that for some reason python runs the main
> thread considerably more than the checking thread. If I ask them both
> to 'print' when run, this becomes obvious ; even if I create more
> check threads than main threads (I even tried 50 check threads and 1
> main thread). Despite my terminology, both the mainthread and
> fetchthread are created in exactly the same way.
>
> what part of threading in python am I not properly understanding?

Unless I'm missing something, your description doesn't make this sound
like either a python-specific problem, or a threading problem. Threads
run when it's their turn and they aren't blocked, and you haven't
described any code that would ever block your main thread, but your
subsidiary threads will often be blocked at a socket waiting for their
HTTP requests to complete.