[Tutor] Python3: looping through web address

[Peter Otten]

Umed Saidov wrote:

> Hello Tutor(s),
> 
> I want to create a loop that loops through a web address, changing only
> a part of that address with each loop. I can't seem to be able to get
> the code to do this. Below is the code with some explanation of what I
> am trying to do. The code seems to loop through the ticker symbol file
> ignoring the web address. Is there another way of doing the same thing?
> 
> import urllib.request, json
> import csv
> import pandas as pd
> import itertools
> 
> ticker = []
> ticker = pd.DataFrame(ticker)
> 
> #open a cvs file with 100+ stock tickers from S&P500. Save in a
> dataframe 'ticker'.
> ticker =pd.read_csv('tickers.csv')
> 
> #Loop through ticker, changing the designated part of the web address.
> Download the data into 'response'
> 
> for t in ticker.itertuples():
>      response +=
> 
urllib.request.urlopen('https:websiteaddress/{ct}/financials'.format(ct=ticker))


ticker is the complete DataFrame and t corresponds to one row in the csv. 
Assuming the column in the csv you are interested in is called "foo" you can 
get it with

columnname = "foo" # replace with your own name
for ct in ticker[columnname]:
    url = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)
    print(url)

Once this prints the correct urls you can write a little function

def process_page(url):
    request = urllib.request.urlopen(url)
    ... # do what you want with the request

invoke it with one url and refine it until the function does what you want. 
Then combine it with the loop

columnname = "foo" # replace with your own name
for ct in ticker[columnname]:
    url = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)
    process_page(url)

In short: take small controlled steps to get to the desired result instead 
of writing a complete script that may or may not work.