crawlers in python with graphing?

gene tani gene.tani at
Tue Jun 20 15:39:16 CEST 2006

bryan rasmussen wrote:
> Hi,
> Sorry, was imprecise, I meant not save the downloaded page locally.
> There probably isn't one though, so I should build one myself.
> Probably just need a good crawler that can be set to dump all links
> into dataset that I can analyse with R.
> Cheers,
> Bryan Rasmussen

There's quite a few already: webchecker, Orchid, mechanize, mygale:

More information about the Python-list mailing list