Thread Question

[Gerhard Fiedler]

On 2006-08-04 04:22:59, Ritesh Raj Sarraf wrote:

> Gerhard Fiedler wrote:
>> Rather than downloading and zipping in the same thread, you could run
>> multiple threads like you're doing that only download files, and one
>> zip-it-all-up thread. After downloading a file, the download threads
>> place a message in a queue that indicates the file they have
>> downloaded, and the zip-it-all-up thread takes messages out of that
>> queue, one at a time, and zips the files. 
> 
> I was using this approach earlier. The problem with this approach is
> too much temporary disk usage.
> 
> Say I'm downloading 2 GB of data which is a combination of, say 600
> files. Now following this approach, I'll have to make sure that I have
> 4 GB of disk space available on my hard drive.

Not necessarily. You have a minimum speed of the zipping process, and a
maximum speed of the download. Between the two you can figure out what the
maximum required temp storage space is. It's in any case less than the full
amount, and if the minimum zipping speed is faster than the maximum
download speed, it's not more than a few files.

But if you current solution works, then that's good enough :)  It probably
wouldn't be much faster anyway; only would avoid a few waiting periods.

Gerhard