[Tutor] Web Page Scraping
Crusier
crusier at gmail.com
Mon May 23 23:17:54 EDT 2016
Dear All,
I am trying to scrape a web site using Beautiful Soup. However, BS
doesn't show any of the data. I am just wondering if it is Javascript
or some other feature which hides all the data.
I have the following questions:
1) Please advise how to scrape the following data from the website:
'http://www.dbpower.com.hk/en/quote/quote-warrant/code/10348'
Type, Listing Date (Y-M-D), Call / Put, Last Trading Day (Y-M-D),
Strike Price, Maturity Date (Y-M-D), Effective Gearing (X),Time to
Maturity (D),
Delta (%), Daily Theta (%), Board Lot.......
2) I am able to scrape most of the data from the same site
'http://www.dbpower.com.hk/en/quote/quote-cbbc/code/63852'
Please advise what is the difference between these two sites.
Attached is my code
Thank you
Regards,
Hank
from bs4 import BeautifulSoup
import requests
import json
import re
warrants = ['10348']
def web_scraper(warrants):
url = "http://www.dbpower.com.hk/en/quote/quote-warrant/code/"
# Scrape from the Web
for code in warrants:
new_url = url + code
response = requests.get(new_url)
html = response.content
soup = BeautifulSoup(html,"html.parser")
print(soup)
name = soup.findAll('div', attrs={'class': 'article_content'})
#print(name)
for n in name:
name1 = str(n.text)
s_code = name1[:4]
print(name1)
web_scraper(warrants)
More information about the Tutor
mailing list