How to safely maintain a status file

Plumo richardbp at
Mon Jul 9 07:52:58 CEST 2012

> What are you keeping in this status file that needs to be saved
> several times per second?  Depending on what type of state you're
> storing and how persistent it needs to be, there may be a better way
> to store it.
> Michael

This is for a threaded web crawler. I want to cache what URL's are
currently in the queue so if terminated the crawler can continue next
time from the same point.

More information about the Python-list mailing list