[Tutor] Python3: looping through web address

Peter Otten __peter__ at web.de
Mon Jul 18 05:11:47 EDT 2016

Umed Saidov wrote:

> Hello Tutor(s),
> I want to create a loop that loops through a web address, changing only
> a part of that address with each loop. I can't seem to be able to get
> the code to do this. Below is the code with some explanation of what I
> am trying to do. The code seems to loop through the ticker symbol file
> ignoring the web address. Is there another way of doing the same thing?
> import urllib.request, json
> import csv
> import pandas as pd
> import itertools
> ticker = []
> ticker = pd.DataFrame(ticker)
> #open a cvs file with 100+ stock tickers from S&P500. Save in a
> dataframe 'ticker'.
> ticker =pd.read_csv('tickers.csv')
> #Loop through ticker, changing the designated part of the web address.
> Download the data into 'response'
> for t in ticker.itertuples():
>      response +=

ticker is the complete DataFrame and t corresponds to one row in the csv. 
Assuming the column in the csv you are interested in is called "foo" you can 
get it with

columnname = "foo" # replace with your own name
for ct in ticker[columnname]:
    url = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)

Once this prints the correct urls you can write a little function

def process_page(url):
    request = urllib.request.urlopen(url)
    ... # do what you want with the request

invoke it with one url and refine it until the function does what you want. 
Then combine it with the loop

columnname = "foo" # replace with your own name
for ct in ticker[columnname]:
    url = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)

In short: take small controlled steps to get to the desired result instead 
of writing a complete script that may or may not work.

More information about the Tutor mailing list