Submitting forms over HTTPS with mechanize
larry.bates at vitalEsafe.com
Thu Sep 4 04:17:56 CEST 2008
> I am working on an academic research project where I need to log in to
> a website (www.lexis.com) over HTTPS and execute a bunch of queries to
> gather a data set. I just discovered the mechanize module, which seems
> great because it's a high-level tool. However, I can't find any decent
> documentation for mechanize apart from the docstrings, which are
> pretty thin. So I just followed some other examples I found online, to
> produce the following:
> baseurl = 'http://www.lexis.com/'
> br = mechanize.Browser()
> br.addheaders = [('User-Agent', 'Firefox')]
> result = br.submit()
> This code hangs at br.submit(), and I can't tell what I'm doing wrong.
> Typically I would inspect the HTTP data with an HTTP debugging proxy
> (Fiddler), but I guess since this is HTTPS I can't do that. Any
> glaring errors in my code?
> By the way, does anyone have suggestions for Python modules that I
> should use instead of mechanize (and that are sufficiently easy)? If
> mechanize fails, I might try modifying some similar Perl code a friend
> sent me that logs into lexis.com.
> Thanks so much,
I've used mechanize quite successfully but others have suggested Twill
http://twill.idyll.org/. It seems to be at least documented.
More information about the Python-list