Multiprocessing queues receiving from wrong process
MRAB
python at mrabarnett.plus.com
Fri Dec 23 20:50:07 EST 2016
On 2016-12-24 01:17, Charles Hixson wrote:
>
>
> On 12/23/2016 01:56 PM, Charles Hixson wrote:
>> I was looking to avoid using a upd connection to transfer messages
>> between processes, so I thought I'd use multiprocessing (which I
>> expect would be faster), but...I sure would appreciate an explanation
>> of this problem.
>>
>> When I run the code (below) instead of messages receiving messages
>> from the correct process I get:
>> (where the first number of a line identifies the index assigned to the
>> process.)
>>
>> waiting for completion
> The receiving process should be the one sent to.
>
> receiving sending sent to
> ndx process name process
> name ndx
>> 1=Process-2 received: at time 0.001243: Process-1 says Hi to 0
>> 0=Process-1 received: at time 0.001211: Process-2 says Hi to 1
>> 0=Process-1 received: at time 0.001144: Process-3 says Hi to 2
>> 4=Process-5 received: at time 0.002324: Process-1 says Hi to 0
>> 0=Process-1 received: at time 0.000953: Process-4 says Hi to 3
>> 0=Process-1 received: at time 0.000674: Process-5 says Hi to 4
>> 3=Process-4 received: at time 0.002114: Process-1 says Hi to 0
>> 3=Process-4 received: at time 0.001864: Process-2 says Hi to 1
>> 4=Process-5 received: at time 0.002094: Process-2 says Hi to 1
>> 2=Process-3 received: at time 0.001711: Process-1 says Hi to 0
>> 4=Process-5 received: at time 0.001885: Process-3 says Hi to 2
>> 4=Process-5 received: at time 0.001586: Process-4 says Hi to 3
>> 1=Process-2 received: at time 0.001456: Process-3 says Hi to 2
>> 3=Process-4 received: at time 0.001734: Process-3 says Hi to 2
>> 2=Process-3 received: at time 0.00158: Process-2 says Hi to 1
>> 2=Process-3 received: at time 0.001444: Process-4 says Hi to 3
>> 2=Process-3 received: at time 0.001088: Process-5 says Hi to 4
>> 3=Process-4 received: at time 0.001221: Process-5 says Hi to 4
>> 1=Process-2 received: at time 0.001212: Process-4 says Hi to 3
>> 1=Process-2 received: at time 0.000885: Process-5 says Hi to 4
>>
>> ## Test multiprocessing queues
>> import multiprocessing as mp
>> import time
>>
>> from multiprocessing import Process
>> from multiprocessing import Queue
>> from queue import Empty
>> from queue import Full
>> from random import random
>>
>>
>> class TestMPQ:
>> """ Class doc """
>>
>> def __init__ (self, ndx):
>> """ Class initialiser """
>> self.name = mp.current_process().name
>> self.ndx = ndx
>>
>> def sendHi (self):
>> for i in range(5):
>> if i != self.ndx:
>> qs[i].put ("{} says Hi to {}".format(self.name,
>> self.ndx))
>>
"self.ndx" is the sender's (my) index, "i" is the receiver's (your
index), so the message you're building is:
<my name> says Hi to <my index>
which you then put in the receiver's (your) queue.
>> def processHi (self):
>> while (True):
>> time.sleep(random() + 0.001)
>> try:
>> msg = qs[self.ndx].get_nowait()
>> print ("{}={} received: {}".format(self.ndx,
>> self.name, msg) )
>> except Empty:
>> break
>> except Exception as ex:
>> print ("Exception: ", repr(ex))
>> break
>>
>> def procHandler (ndx, qs):
>> p = TestMPQ(ndx)
>> p.sendHi()
>> p.processHi()
>>
>> if "__main__" == __name__:
>> qs = []
>> for i in range(5):
>> qs.append(Queue())
>> ps = []
>> for i in range(5):
>> ps.append(Process(target = procHandler, args = (i, qs) ) )
>> ps[i].start()
>> print ("waiting for completion")
>> for i in range(5):
>> ps[i].join()
>>
>>
>
More information about the Python-list
mailing list