download x bytes at a time over network

Matt Nordhoff mnordhoff at mattnordhoff.com
Wed Mar 18 06:24:25 CET 2009


Saurabh wrote:
> Heres the reason behind wanting to get chunks at a time.
> Im actually retrieving data from a list of RSS Feeds and need to
> continuously check for latest posts.
> But I dont want to depend on Last-Modified header or the pubDate tag
> in <channel>. Because a lot of feeds just output date('now')  instead
> of the actual last-updated timestamp.
> But when continuously checking for latest posts, I dont want to
> bombard other people's bandwidth - so I just want to get chunks of
> bytes at a time and internally check for <item>...</item> with my
> database against timestamp values.
> Is there a better way to achieve this ?

For the feeds that *do* set Last-Modified properly, won't you be using
*more* bandwidth by downloading part of the feed instead of just using
If-Modified-Since?
-- 



More information about the Python-list mailing list