[Tutor] General question rgrd. usage of libraries
Rafael Knuth
rafael.knuth at gmail.com
Fri May 5 09:45:00 EDT 2017
Hi there,
I just recently learned how to build a basic web scraper with Python
3.5 (I am learning Python for data analytics purposes). Being new to
coding, I have a question:
How do I know which libraries I need to perform a certain task?
For example, in case of this web scraper (which I built with help of a
tutorial on YouTube) I need to have urrlib and Beautiful Soup
import urllib
import urllib.request
from bs4 import BeautifulSoup
theurl = "https://twitter.com/rafaelknuth"
thepage = urllib.request.urlopen(theurl)
soup = BeautifulSoup(thepage, "html.parser")
print(soup.title.text)
i = 1
for tweets in soup.findAll("div",{"class":"content"}):
print(i)
print(tweets.find("p").text)
i = i + 1
Is there a way I can figure out which libraries I need when drafting my code?
Can you share your experiences? Right now, if I wanted for example to
populate a Google Sheet with my scraped web content - how would I know
which libraries I would need to actually make this happen? I am trying
wondering if there is a process to figure out what I exactly need
library-wise.
Thank you,
Rafael
More information about the Tutor
mailing list