How to schedule system calls with Python

[MRAB]

TerryP wrote:
> On Oct 15, 7:42 pm, Jeremy <jlcon... at gmail.com> wrote:
>> I need to write a Python script that will call some command line
>> programs (using os.system).  I will have many such calls, but I want
>> to control when the calls are made.  I won't know in advance how long
>> each program will run and I don't want to have 10 programs running
>> when I only have one or two processors.  I want to run one at a time
>> (or two if I have two processors), wait until it's finished, and then
>> call the next one.
>>
>> How can I use Python to schedule these commands?
>>
>> Thanks,
>> Jeremy
> 
> External programs are not system calls; external programs are invoked
> through system calls; for example system() is a function call which
> when implemented under UNIX systems invokes some form of fork() and
> exec(), and likely spawn() under Windows NT.
> 
> If you want simple sequenceal execution of external programs, use a
> suitable blocking function to execute them (like system) combined with
> a simple loop over the sequence of commands to run.
> 
> 
> for prog in ['cmd1', 'cmd2', 'cmd3']:
>     os.system(prog)
> 
> blah.
> 
> 
> 
> For anything more detailed (or complex) in response, try being more
> detailed yourself ;).

You could use multithreading: put the commands into a queue; start the
same number of worker threads as there are processors; each worker
thread repeatedly gets a command from the queue and then runs it using
os.system(); if a worker thread finds that the queue is empty when it
tries to get a command, then it terminates.