[AstroPy] A question about multiprocessing

[Francesco Pierfederici]

Hi Jean-Baptiste,

A possibly simpler way to do what I *think* you want to do is using
multiprocessing.Pool:

-------------
#!/usr/bin/env python
import multiprocessing as multi


def CatDistort(Image):
    print('Working on Image %s' % (Image))
    return('Processed ' + Image)



NPROC = 4
Images = 'abcdefghijklmnopqrstuvwxyz'
pool = multi.Pool(processes=NPROC)

results = pool.map(CatDistort, Images)

print 'Process completed'

print(results)
exit(0)
---------------------

But then again I might be misunderstanding what you are trying to accomplish.

Cheers,
Francesco





On Fri, Nov 4, 2011 at 10:45 AM, Jean-Baptiste Marquette
<marquett at iap.fr> wrote:
> Dear Python gurus,
> I'm not sure to be on the right mailing list, but I have the following
> problem:
> I have this code ran from Eclipse/PyDev:
> import multiprocessing as multi
> def CatDistort(queue):
>     for Image in iter(queue.get, 'STOP'):
> etc...
> queue = multi.Queue()
> for Image in glob.glob(DirImg + '*r.sdf'):
>     queue.put(Image)
>     print os.path.basename(Image), 'queued'
> queue.put('STOP')
> Jobs = []
> for i in range(NPROC):
>     proc= multi.Process(target=CatDistort, args=[queue])
>     Jobs.append(proc)
>     proc.start()
>
>
>
> for job in Jobs:
>     job.join(360)
> print 'Process completed'
> exit(0)
> The target and the queue work well, but the process never terminates. In
> other words, the final print is never displayed.
> Any hint ?
> Thanks for your help,
> Cheers
> Jean-Baptiste
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>