[Tutor] Python3: looping through web address

Mon Jul 18 10:10:40 EDT 2016

On 07/18/2016 06:35 AM, Alan Gauld via Tutor wrote:
> Caveat: I'm not a Pandas expert but... This looks odd. You first 
> assign an empty list to ticker and pass that into pandas.DataFrame 
> then assign the result back to ticker. Is that what you intended? Why 
> not just ticker = pd.DataFrame([]) Or is there something clever going on? 
No. nothing clever I am afraid. I wanted to create a variable to store 
data in pandas format. This seemed like a good way of doing it... but 
perhaps not.
>> #open a cvs file with 100+ stock tickers from S&P500. Save in a
>> dataframe 'ticker'.
>> ticker =pd.read_csv('tickers.csv')
> And now you overwrite the dataframe with the content of a csv file?
> I'm not sure what read_csv returns but it seems to me you are
> creating and throwing away a lot of stuff here?
>
the csv is just a column of tickers. I wanted to store them in pandas 
format. this returns a pandas dataframe with contents of the csv file 
stored in two columns (index and ticker values)
>> for t in ticker.itertuples():
>>       response +=
>> urllib.request.urlopen('https:websiteaddress/{ct}/financials'.format(ct=ticker))
> ticker is the iterable, it does not change during the loop.
> Yet in the url it is ticker you assign to the ct marker.
> Don't you want 't'? That's the thing that changes.
     I want the content of the column one in ticker. With each iteration 
of t, which I thought was just a simple numeric progression representing 
a row number, I wanted to get the content of the column one.
> It might help to create and print the address before calling urlopen:
>
> for t in ticker.itertuples():
>      address = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)
>      print address   # remove when done
>      response += urllib.request.urlopen(address)
>
>
>
Thanks for the suggestion. I tried this and received the badstatusline 
error message:

Traceback (most recent call last):
   File "something.py", line 23, in <module>
     response += urllib.request.urlopen(address)
   File "...request.py", line 162, in urlopen
     return opener.open(url, data, timeout)
   File "..request.py", line 465, in open
     response = self._open(req, data)
   File "..request.py", line 483, in _open
     '_open', req)
   File "python3.5/urllib/request.py", line 443, in _call_chain
     result = func(*args)
   File "python3.5/urllib/request.py", line 1283, in https_open
     context=self._context, check_hostname=self._check_hostname)
   File "/python3.5/urllib/request.py", line 1243, in do_open
     r = h.getresponse()
   File "python3.5/http/client.py", line 1174, in getresponse
     response.begin()
   File "python3.5/http/client.py", line 282, in begin
     version, status, reason = self._read_status()
   File "python3.5/http/client.py", line 264, in _read_status
     raise BadStatusLine(line)
http.client.BadStatusLine: <html>