How to schedule system calls with Python

[Jeremy]

On Oct 15, 6:32 pm, MRAB <pyt... at mrabarnett.plus.com> wrote:
> TerryP wrote:
> > On Oct 15, 7:42 pm, Jeremy <jlcon... at gmail.com> wrote:
> >> I need to write a Python script that will call some command line
> >> programs (using os.system).  I will have many such calls, but I want
> >> to control when the calls are made.  I won't know in advance how long
> >> each program will run and I don't want to have 10 programs running
> >> when I only have one or two processors.  I want to run one at a time
> >> (or two if I have two processors), wait until it's finished, and then
> >> call the next one.
>
> >> How can I use Python to schedule these commands?
>
> >> Thanks,
> >> Jeremy
>
> > External programs are not system calls; external programs are invoked
> > through system calls; for example system() is a function call which
> > when implemented under UNIX systems invokes some form of fork() and
> > exec(), and likely spawn() under Windows NT.
>
> > If you want simple sequenceal execution of external programs, use a
> > suitable blocking function to execute them (like system) combined with
> > a simple loop over the sequence of commands to run.
>
> > for prog in ['cmd1', 'cmd2', 'cmd3']:
> >     os.system(prog)
>
> > blah.
>
> > For anything more detailed (or complex) in response, try being more
> > detailed yourself ;).
>
> You could use multithreading: put the commands into a queue; start the
> same number of worker threads as there are processors; each worker
> thread repeatedly gets a command from the queue and then runs it using
> os.system(); if a worker thread finds that the queue is empty when it
> tries to get a command, then it terminates.

Yes, this is it.  If I have a list of strings which are system
commands, this seems like a more intelligent way of approaching it.
My previous response will work, but won't take advantage of multiple
cpus/cores in a machine without some manual manipulation.  I like this
idea.

Thanks!
Jeremy