[ python-Bugs-1349977 ] urllib2 blocked from news.google.com

[SourceForge.net]

Bugs item #1349977, was opened at 2005-11-06 22:31
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349977&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
>Resolution: Rejected
Priority: 5
Submitted By: Michael Hoisie (asyncster)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 blocked from news.google.com

Initial Comment:
It seems that google is blocking requests from clients
with urllib 2.4 as the user-agent. If you telnet to
news.google.com and type: 

GET / HTTP/1.1
Host: news.google.com
User-agent: Python-urllib/2.4

You get a HTTP/1.1 403 Forbidden


----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2005-11-07 13:20

Message:
Logged In: YES 
user_id=357491

It isn't a Python bug, but then again it got my attention
which means I can contact people within Google to see if
they can find out what happened.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-11-07 06:38

Message:
Logged In: YES 
user_id=6656

In what crazy universe is this a Python bug?  It's up to google what they 
do with http requests, surely.  If you are reasonably sure that your use 
does not violate the terms of use for google news:

http://news.google.com/intl/en_us/terms_google_news.html

Then you can experiment with getting urllib to send a different User-Agent 
header. 

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2005-11-06 23:21

Message:
Logged In: YES 
user_id=357491

I can verify this using urllib.urlretrieve() from the trunk.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1349977&group_id=5470