What is the fastest way to do 400 HTTP requests using requests library?
Steven D'Aprano
steve at pearwood.info
Mon Jan 4 18:38:35 EST 2016
On Tue, 5 Jan 2016 07:50 am, livemsn22 at gmail.com wrote:
> So what is the fastest way to make 400 HTTP requests using "requests"
> library and also using tor proxy?
Since this will be I/O bound, not CPU bound, probably use separate threads.
Push the 400 requests into a queue, then create N threads, where N will need
to be determined by experiment, but will probably be something like 4 or 8,
and let each thread pop a request from the queue as needed.
Are you experienced with threads? Do you need further information about
using threads and queues?
--
Steven
More information about the Python-list
mailing list