multiprocessing, what am I doing wrong?
Eric Frederich
eric.frederich at gmail.com
Tue Feb 28 12:16:19 EST 2012
If I do a time.sleep(0.001) right at the beginning of the run() method,
then it completes fine.
I was able to run it through a couple hundred times without problem.
If I sleep for less time than that or not at all, it may or may not
complete.
On Mon, Feb 27, 2012 at 9:38 PM, MRAB <python at mrabarnett.plus.com> wrote:
> On 27/02/2012 16:57, Eric Frederich wrote:
>
>> Still freezing sometimes, like 1 out of 10 times that I run it.
>> Here is updated code and a couple of outputs.
>>
>> [snip]
> I don't know what the problem is. All I can suggest is a slightly
> modified version.
>
> If a worker that says it's terminating without first saying that it's
> got nothing, then I can only assume that the worker had some uncaught
> exception.
>
>
>
> #!/usr/bin/env python
>
> import sys
> import Queue
> import multiprocessing
> import time
>
> def FOO(a, b, c):
> print 'foo', a, b, c
> return (a + b) * c
>
> class MyWorker(multiprocessing.**Process):
> def __init__(self, name, inbox, outbox):
> super(MyWorker, self).__init__()
> self.name = name
> self.inbox = inbox
> self.outbox = outbox
> print >> sys.stderr, 'Created %s' % self.name; sys.stderr.flush()
> def run(self):
> print >> sys.stderr, 'Running %s' % self.name; sys.stderr.flush()
> try:
>
> while True:
> try:
> args = self.inbox.get_nowait()
> print >> sys.stderr, '%s got something to do' %
> self.name; sys.stderr.flush()
> except Queue.Empty:
> print >> sys.stderr, '%s got nothing' % self.name;
> sys.stderr.flush()
> break
> self.outbox.put(FOO(*args))
> finally:
> print >> sys.stderr, '%s is terminating' % self.name;
> sys.stderr.flush()
>
>
> if __name__ == '__main__':
> # This file is being run as the main script. This part won't be
> # run if the file is imported.
>
> print >> sys.stderr, 'Creating todo queue'; sys.stderr.flush()
> todo = multiprocessing.Queue()
>
> for i in xrange(100):
> todo.put((i, i + 1, i + 2))
>
> print >> sys.stderr, 'Creating results queue'; sys.stderr.flush()
> result_queue = multiprocessing.Queue()
>
> print >> sys.stderr, 'Creating Workers'; sys.stderr.flush()
> w1 = MyWorker('Worker 1', todo, result_queue)
> w2 = MyWorker('Worker 2', todo, result_queue)
>
> print >> sys.stderr, 'Starting Worker 1'; sys.stderr.flush()
> w1.start()
> print >> sys.stderr, 'Starting Worker 2'; sys.stderr.flush()
> w2.start()
>
> for i in xrange(100):
> print result_queue.get()
> --
> http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120228/1940bcfd/attachment-0001.html>
More information about the Python-list
mailing list