setting Referer for urllib.urlretrieve

samwyse samwyse at gmail.com
Sun Aug 9 09:13:38 EDT 2009


Here's what I have so far:

import urllib

class AppURLopener(urllib.FancyURLopener):
    version = "App/1.7"
    referrer = None
    def __init__(self, *args):
        urllib.FancyURLopener.__init__(self, *args)
        if self.referrer:
            addheader('Referer', self.referrer)

urllib._urlopener = AppURLopener()

Unfortunately, the 'Referer' header potentially varies for each url
that I retrieve, and the way the module is written, I can't change the
calls to __init__ or open. The best idea I've had is to assign a new
value to my class variable just before calling urllib.urlretrieve(),
but that just seems ugly.  Any ideas?  Thanks.

PS for anyone not familiar with the RFCs:  Yes, I'm spelling
"referrer" correctly everywhere in my code.



More information about the Python-list mailing list