[Tutor] : Threads?

[Hugo González Monteverde]

I have done scripts for decompressing MP3 in the past. I normally follow 
a fork() exec() subprocess scheme, and this helps me gain about 30% 
improvement. I'm not an expert, but I have read that CPU time will be 
better used by several processes than for just one, and while one 
process is waiting for disk IO and the like, another can be number 
crunching.

I have some code, but I will have to look for it and then post it.

I do:

pid = os.fork()
if pid == 0:#child
     os.system("my mp3 decompressor")
     raise SystemExit

Doing this for all files that I have to convert, with up to, say, 5 
subprocesses. Then, on the parent, os.waitpid() will be used to check 
when  subprocesses have terminated and respawned until I run out of files..

I will post some code when I find it.

Hugo


Orri Ganel wrote:
> Kent Johnson wrote:
> 
> 
>>Orri Ganel wrote:
>> 
>>
>>
>>>Hello all,
>>>
>>>I've been working on a program for a week or two now that will convert 
>>>all the wav files in a folder to mp3s, filling the id3 tags with the 
>>>correct information as collected from gracenote.com.  This part works 
>>>fine.  However, the actual conversion to mp3 takes between 30 and 50 
>>>seconds per song, so it's usually about 10 minutes per album.  With this 
>>>in mind, I thought, why not try to use threads so all the conversions 
>>>happen simultaneously?  That way, the whole album will take between 30 
>>>and 50 seconds.  Unfortunately, I can't seem to get a working threaded 
>>>version that significantly reduces the time involved . . . The 
>>>   
>>>
>>
>>The only part you are doing in a thread is the actual conversion. This is likely to be CPU-intensive so running it in multiple threads may not help - you still have only the one CPU to run on. To the extent that you can overlap disk I/O in one conversion with processing in another you may get a win; on the other hand you could just as well have contention for the disk as you try to read and write a bunch of files at the same time.
>>
>>The fetch from gracenote.com seems like a better candidate for threading because there is some latency...but the total time is still probably small compared to the conversion time.
>>
>>Maybe if you have multiple CPUs you can get a speedup by using as many threads as CPUs...I'm not sure how os.system() behaves in this case. You may have to explicitly fork to get a new process.
>>
>>Hmm...come to think of it, os.system() may block other threads, I don't know...you could try subprocess.Popen() instead.
>>
>>Kent
>>
> 
> Thanks for the tip.  Unfortunately, I only have 1 CPU and not the 
> slightest idea how to code for multiple CPUs in any case.  Looks like 
> I'll just hafta deal with a 10-minute time per album.
>