Config & ConfigParser
steve+comp.lang.python at pearwood.info
Wed Mar 6 05:04:29 CET 2013
On Tue, 05 Mar 2013 18:15:20 -0600, Tim Chase wrote:
> On 2013-03-05 15:58, Chuck wrote:
>> Thanks Tim! So much stuff I haven't thought of before. Out of
>> curiosity, what's the benefit of caching the download, instead of
>> downloading to the final destination?
> If your connection gets interrupted, the server goes down, etc, you have
> a partial download. If you've put it directly in the download path,
> your other programs see this partial download. However if your program
> can resume the download where it left off, once it's completed
> successfully, it can atomically move the file to your download location.
> Thus your other programs only ever see all-or-nothing in the download
That's not really a *cache* though.
Personally, I find programs that do that sort of cleverness annoying
rather than helpful. More often than not, they are buggy and fail to
clean up after themselves if the download is interrupted, so the secret
download directory ends up filled with junk:
sort of thing.
Another problem with this tactic is that it makes it unnecessarily
difficult to watch progress of the download, except via the application's
official user interface. (If it gives you any interface for watching
download progress, which is may not.) You have to locate the secret
download directory, work out what file name the app is using for the
temporary file (many apps obfuscate the file name), then watch that file
grow until it suddenly disappears, at which point you then have to change
directories to see if it reappeared where you actually wanted it to be,
or was just deleted by something else.
A third problem: instead of having to worry about having enough disk
space in one location, now you have to worry about having enough disk
space in *two* locations.
I've even see a program download a large file into the temp location,
*unsuccessfully* try to copy it into the final location, then delete the
Yet another problem: websites sometimes lie about the size of some files.
So when you download them, the actual file ends (say) one byte short of
what the webserver claims. There's nothing wrong with the file, it is
actually complete, or at least recoverable (many formats, like JPEG, are
remarkably resistant to damage), but your app will think it never
completes and either never move it to the final destination, or worse,
keep trying to download it over and over and over again.
Finally, moving from the temp directory to the final location is not
necessarily an atomic operation. If it crosses file system boundaries, it
is a two-step process.
I think that the KISS principle applies here. If the user tells your app
to download in some location, your app should download in that location.
More information about the Python-list