[Tutor] Scraping old twitter posts

Mats Wichmann mats at wichmann.us
Thu May 17 10:16:42 EDT 2018


On 05/17/2018 04:47 AM, matej taferner wrote:
> Hi guys !
> I didn't use Python for c. 2 years hence my skills are a little bit rusty.
> I will really appreciate some hints or a guideline.
> I would like to search my Twitter feed and search/scrape for some "specific
> words" and create a list for further analysis.
> Can someone recommend me some quality resources on this topic?

Don't know if anyone has enough experience to comment on "quality" -
we'll wait and see. But it's not really the kind of question this list
is set up for.

Getting data out of a website like Twitter should be done using their
API, not by scraping, if at all possible: the API delivers data in a way
that Twitter is okay with; scraping is against the terms of service
unless you have explicit permission - and don't take that too lightly,
LinkedIn has been busy suing people for scraping their site, for example.

There are several existing projects that get data out of Twitter which
you can search for (look on pypi.org for example, or github).  If your
objective is a Python learning project you might want to look for some
of the tutorials on the subject and avoid ready-made projects.

Python Tutor list will be more useful to you if you come with specific
coding questions - I tried this and it doesn't work; I don't understand
this; etc.

Best of luck!


More information about the Tutor mailing list