Mimic a web surfer ?

[Ben Hutchings]

In article <3e894b7d$0$29158$626a54ce at news.free.fr>, shagshag13 wrote:
> hello,
> 
> i would like to develop a script which will mimic a web surfer, that is :
> 
> - follow links as a human and thus always come from previous page (i think
> there is something to do with http 'referer' and cookies)
> - be able to log if necessary
> 
> is there anything already existing ? i'm looking for any tips which will
> help me to start...

The HttpUnit package might be useful, but it's written in Java.

> note : this could be used to have "automatic" access to free webmails not
> providing pop protocol.

That has already been done for some popular webmail providers like Hotmail
(look for "gotmail" or "HotWayDaemon") and Yahoo ("FetchYahoo").