[Tutor] auto referer handler/opener for urlib2?

[Kent Johnson]

On Tue, Oct 21, 2008 at 12:05 AM, xbmuncher <xboxmuncher at gmail.com> wrote:
> I was reading about urllib2 openers.. Can I make any kind of def or function
> and make the urllib2 "urlopen" function run through this function first
> before opening a url? For example, something like a reporthook function.
> What I want to do is this:
>
> def autoReferer(handle):
> if handle.lastRequest.url != None:
> handle.addheader('referer' : handle.lastRequest.url)
>
>
> How can make this auto referer functionality with urllib2?

You can subclass BaseHandler with a class that has a
http_request(self, request) method. Add an instance of your handler to
the OpenerDirector. http_request() will be called with the request
object as its parameter and returns the request object as its result.
See the docs for BaseHandler.protocol_request (just after this link)
http://docs.python.org/library/urllib2.html#urllib2.BaseHandler.http_error_nnn

and look at the implementation of HTTPCookieProcessor for an example.

Note you must have an implementation of <protocol>_request() for each
protocol you wish to handle, e.g. http and https.

Kent