On Oct 10, 2009, at 7:34 AM, Reza Lotun wrote:
I'm not quite sure what you mean. Are you talking about web.client or web.server?
Web client. I was having trouble with a particular URL which happened to be a download URL and wanted to see what the request and response objects looked like and was trying to find where I could examine them. In the code I posted before, we were creating an HTTPClientFactory, then snagging the deferred. I've simplified that by just using client.getPage(). The advantage of the HTTPClientFactory approach is that you can get at the response_headers.
There is a new twisted.web client implementation in the works which will hopefully be ready for trunk in the near future, and which should provide nearly the same functionality offered by urllib2 (or more).
Where might that be hiding? I'd love to have something that does deferreds like Twisted and also implements all of urllib2. I find the info() from urlopen() particularly helpful when debugging issues like this.
I finally figured out that it was a stupid string formatting bug in the code that was being echoed by Twisted but swallowed by the "Grand Catcher of all Things Exception" in Twisted..
Once I found and fixed that, everything works fine -- until the next time.
Exceptions shouldn't be swallowed by twisted. If you simply attach errBacks then you'll get the exception and full traceback. It really isn't that bad at all - I thought it looked crazy when I first started, but all the information is there to debug, etc.
Yes, the problem was in my mis-reading of the info in my errback. Unfortunately, the way the logging is set up in this particular code, logging.debug() returns 10 zillion times too much repetitive information and it makes it hard to wade through to find the relevant info (like the errback's output I was missing). Thanks again for your help with this and I'd love to know more about this new urllib2 type module for Twisted. Thanks, S