Best approach to get data from web page continuously

Joel Goldstick joel.goldstick at
Thu Sep 18 17:49:18 CEST 2014

On Thu, Sep 18, 2014 at 9:30 AM, Juan Christian
<juan0christian at> wrote:
> I'll write a python (Python 3.4.1) script to fetch for new data (topics)
> from this page (
> continuously.
> All the topics follow this structure: <a class="forum_topic_overlay"
> href=""> </a>
> It will work like that: I'll get the last topics, do some background
> checking regarding user level, inventory value, account age, and other
> things, if the user pass in the checking, I'll print some info and links in
> the terminal. The only thing I need to know to start is: What's the better
> way the get this data? Beautiful Soup 4 + requests? urllib? Others?

Requests is a lot simpler than urllib.  I've used BS4.  There is
something called scrapy that is similar I think

> Thanks.
> --

Joel Goldstick

More information about the Python-list mailing list