[Tutor] Can subprocess run functions ?

[Damon Timm]

On Wed, Jan 7, 2009 at 7:36 PM, wesley chun <wescpy at gmail.com> wrote:
> this has been a highly-desired feature for quite awhile.
>
> starting in 2.6, you can use the new multiprocessing module
> (originally called pyprocessing):
> http://docs.python.org/library/multiprocessing.html
>
> there is a backport to 2.4 and 2.5 here:
> http://pypi.python.org/pypi/multiprocessing/2.6.0.2
>
> there are similar packages called pypar and pprocess:
> http://datamining.anu.edu.au/~ole/pypar/
> http://www.boddie.org.uk/python/pprocess.html
>
> hope this helps!

Thanks Wesley - it does!

As is often the case, the minute I ask a question like this I have
some anxiety that the answer must be right under my nose and I rush
about the tubes of the internet searching for a clue ... I also found:

http://chrisarndt.de/projects/threadpool/threadpool.py.html

Since I have been doing a bit of reading since I started, I was able
to download this "threadpool", import it, and it actually get it to
work.  Here is what I messily put together (using my first referenced
script, as an example) ... I think I may stick with this for a while,
since it seems well-thought out and, as far as I can tell, works!

(No shame in "borrowing" ... though not as cool as making it up myself.)

import subprocess
import threadpool
import os

totProcs = 2 #number of processes to spawn before waiting
flacFiles = ["test.flac","test2.flac","test3.flac","test4.flac","test5.flac","test6.flac"]

def flac_to_mp3(flacfile):
    print "Processing: " + flacfile
    mp3file = flacfile.rsplit('.', 1)[0] + '.mp3'

    p = subprocess.Popen(["flac","--decode","--stdout","--silent",flacfile],
stdout=subprocess.PIPE)
    p1 = subprocess.Popen(["lame","--silent","-",mp3file], stdin=p.stdout)
    p1.communicate()

    #Test file size so we know it is actually waiting until it has been created.
    size = os.path.getsize(mp3file)
    return str("File: " + mp3file + "* Size: " + str(size))

def print_result(request, result):
    print "* Result from request #%s: %r" % (request.requestID, result)

pool = threadpool.ThreadPool(totProcs)
convert = threadpool.makeRequests(flac_to_mp3,flacFiles,print_result)
[pool.putRequest(req) for req in convert]
pool.wait()

print "All Done!"



> -- wesley
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> "Core Python Programming", Prentice Hall, (c)2007,2001
> "Python Fundamentals", Prentice Hall, (c)2009
>    http://corepython.com
>
> wesley.j.chun :: wescpy-at-gmail.com
> python training and technical consulting
> cyberweb.consulting : silicon valley, ca
> http://cyberwebconsulting.com
>