Python Web Servers and Page Retrievers

[Collin Stocks]

I tried it, and when checking it using a proxy, saw that it didn't really
work, at least in the version that I have (urllib v1.17 and urllib2 v2.5).
It just added that header onto the end, therefore making there two
User-Agent headers, each with different values. I might add that my script
IS able to retrieve search pages from Google, whereas both urllibs are
FORBIDDEN with the headers that they use.

On 4/8/07, Max Erickson <maxerickson at gmail.com> wrote:
>
> Subscriber123 <subscriber123 at gmail.com> wrote:
> > urllib, or urllib2 for advanced users. For example, you can
> > easily set your own headers when retrieving and serving pages,
> > such as the User-Agent header which you cannot set in either
> > urllib or urllib2.
>
> Sure you can. See:
>
> http://www.diveintopython.org/http_web_services/user_agent.html
>
> (though the behavior was changed for python 2.3 to make setting the
> user agent work better)
>
>
> max
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20070411/7671c442/attachment.html>