I've got a job-queue process implemented twisted that interfaces with Postgres and occasionally downloads some urls
To handle the URLs, I'm using mechanize. I know that twisted.web1 has the client, but it seems to be not actively maintained, and there's no client in twisted.web2. mechanize is actively maintained and has a bunch of features that I like, so its a win win situation.
except for timeouts
mechanize doesn't have a timeout of its own. its using a global timeout , i think its using the default python socket timeout, and there isn't any override available. a 60s timeout doesn't work for me. i need 10s-- but i can't change the global timeout , as its in a long running process and I don't want to interfere with any of my other tasks.
so I'm thinking that I somehow wrap mechanize and toss it into the event loop.
rather than spend a few hours reinventing the wheel, i figure someone here *might* have done this before and can offer a pointer. anyone?