Trying to solve a python/mechanize "error 500" http error

[Larry Bates]

bruce wrote:
> i'm getting the following error:
> 	mechanize._response.httperror_seek_wrapper: HTTP Error 500:
> 
> i'm running python 5.1
> and mechanize 0.1.7b
> 
> I have no idea as to what I have to change/modify/include to handle this
> issue. The link that I'm testing is at the bottom of the page. When I insert
> the link into the browser, I actually get an err page.. so, I suspect that
> there is a handler that I should be able to modify/use to handle this
> situation...
> 
> Thoughts/Comments will be greatly appreciated...
> 
> Thanks
> 
> 
> the output is:
> 
> www =  www.1800ink.com
> url2= http://www.quantcast.com/www.1800ink.com/traffic
> Traceback (most recent call last):
>   File "./reseller_scrape_child.py", line 288, in <module>
>     q1 = shopfuncs.quant(rhref)
>   File "/adkiller/shopfuncs.py", line 56, in quant
>     br.open(url2)
>   File "build/bdist.linux-x86_64/egg/mechanize/_mechanize.py", line 203, in
> open
>   File "build/bdist.linux-x86_64/egg/mechanize/_mechanize.py", line 254, in
> _mech_open
> mechanize._response.httperror_seek_wrapper: HTTP Error 500:
> [root at toshiba adkiller]# ./reseller_scrape_child.py
> 
> 
> 
> my code segment looks like:
> ====================================
> from  mechanize import Browser
> import mechanize
> 	br = Browser()
> 	user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
> 	values1 = {'name' : 'Michael Foord',
> 	          'location' : 'Northampton',
> 	          'language' : 'Python' }
> 	headers = { 'User-Agent' : user_agent }
> 
> 	#br.set_cookiejar(cj)
> 	br.set_handle_redirect(True)
> 	br.set_handle_referer(True)
> 	br.set_handle_robots(False)
> 	br.addheaders = [('User-Agent', 'Firefox')]
> 
> 	url2 ="http://www.quantcast.com/xxxx/traffic"
> 	#gets the page (url) from the quantcast app
> 	url2=url2.replace("xxxx",url)
> 	print "url2=",url2
> 	br.open(url2)
> 
> =======================================
> 
> this works ok for most of the sites.. but something weird is happening with
> the actual page:
> 	http://www.quantcast.com/www.1800ink.com/traffic
> 
> 
> thanks...
> 
> 
> 

Looking up 500 error (here
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
gives me:

10.5.1 500 Internal Server Error

The server encountered an unexpected condition which prevented it from 
fulfilling the request.


On some servers (Amazon), when you get 500 errors you are instructed to try
the request again.  I don't know about this server.

-Larry