<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I am trying to obtain data on a https site, but everytime I try to access the data that is behind the logon screen, I get the logon page instead. I was able to successfully do a test problem:<div><br></div><div><code><font color="#7f0055"><b>import </b></font><font color="#000000">sys, urllib2, urllib</font><br>
<font color="#ffffff"></font><br>
<font color="#000000">zipcode = </font><font color="#2a00ff">"48103"</font><br>
<font color="#000000">url = </font><font color="#ff6100">'ht</font><font color="#000000">tp:</font><font color="#3f7f5f">//<a href="http://www.wunderground.com/cgi-bin/findweather/getForecast'">www.wunderground.com/cgi-bin/findweather/getForecast'</a></font><br>
<font color="#000000">data = urllib.urlencode</font><font color="#000000">([(</font><font color="#ff6100">'qu</font><font color="#000000">ery</font><font color="#ff6100">', </font><font color="#000000">zipcode</font><font color="#000000">)])</font><br>
<font color="#000000">req = urllib2.Request</font><font color="#000000">(</font><font color="#000000">url</font><font color="#000000">)</font><br>
<font color="#000000">fd = urllib2.urlopen</font><font color="#000000">(</font><font color="#000000">req, data</font><font color="#000000">)</font><br>
<font color="#7f0055"><b>while </b></font><font color="#990000">1</font><font color="#000000">:</font><br>
<font color="#ffffff"> </font><font color="#000000">data = fd.read</font><font color="#000000">(</font><font color="#990000">1024</font><font color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>if </b></font><font color="#000000">not len</font><font color="#000000">(</font><font color="#000000">data</font><font color="#000000">)</font><font color="#000000">:</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>break</b></font><br>
<font color="#ffffff"> </font><font color="#000000">sys.stdout.write</font><font color="#000000">(</font><font color="#000000">data</font><font color="#000000">)</font></code></div><div><font class="Apple-style-span" face="monospace"><br></font></div><div><font class="Apple-style-span" face="monospace">which performed as I expected. </font></div><div><font class="Apple-style-span" face="monospace"><br></font></div><div><font class="Apple-style-span" face="monospace">However, for the url '<a href="https://secure.umcu.org/cgi-bin/mcw000.cgi?MCWSTART'">https://secure.umcu.org/cgi-bin/mcw000.cgi?MCWSTART'</a></font></div><div><font class="Apple-style-span" face="monospace">with data = ... {'HBUSERNAME':username,'PASSWORD':password}</font></div><div><font class="Apple-style-span" face="monospace"><br></font></div><div><font class="Apple-style-span" face="monospace">I was sent just the logon screen. Am I missing something important that has to do with interaction with javascript, https, or cgi, or something completely different?</font></div><div><font class="Apple-style-span" face="monospace"><br></font></div><div><font class="Apple-style-span" face="monospace"><br></font></div><div><font class="Apple-style-span" face="monospace">Thanks. </font></div></body></html>