Thread is somehow interfering with a while loop called after the thread is started

Dan McInerney dan.h.mcinerney at
Sun Mar 16 14:24:45 CET 2014

Coming back to this a long time later. I figured it out a little bit after
I posted this. I wasn't aware that q.try_run() within the nfqueue module
was a blocking call
I'm not sure I was even aware of what it meant to be blocking vs
nonblocking. Either way, someone from #python looked at my code and told me
nfqueue is a blocking call and I should look into something like twisted.

Twisted is a complex module but I spent a bunch of time looking at it and
eventually had some working code implementing the absolute bare essentials
as I needed them to go asynchronous. This question eventually progressed

Speaking of network concurrency, my recent project has been from a web
scraper point of view and I think gevents+requests or gevents+mechanize are
the way to go for that somewhat-related topic. erequests won't import due
to a bug, grequests has no way to handle exceptions in the fetching of
responses, treq can't find and fill out form logins (maybe with a POST,
haven't explored that route), and scrapy is way too big and complex for my
latest project (, eventlets
I'm sure CAN work with libraries other than urllib2 but it implements and
imports its own urllib2 and it is not a favorable time:effort ratio for me
to try and implement that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the Python-list mailing list