[Tutor] When to use multiprocessing Managers?

[James Chapman]

Thanks for the replies so far...

As per the subject line, yes I'm talking about managers and any object
a manager is capable of spawning. This is not version specific or code
specific, it's more a discussion on when to use a manager and when
it's not needed.

>From the OReilly link sent by
Steven...(http://broadcast.oreilly.com/2009/04/pymotw-multiprocessing-part-2.html)
"The Manager is responsible for coordinating shared information state
between all of its users. By creating the list through the manager,
the list is updated in all processes when anyone modifies it. In
addition to lists, dictionaries are also supported."

In my case I have multiple threaded processing sending log messages to
a Queue that was created by the parent process. The parent process
then has a read thread to take messages out of the queue and write
them to file. Whether I create the queue object like this:

log_Q = multiprocessing.Queue()

or whether I create it like this:

multimanager = multiprocessing.Manager()
log_Q = multimanager.Queue()

seems to make no difference. I always get all the messages from all
the threads in all the processes.

Perhaps the manager would be important if I was writing to a Queue and
expecting all threads to see that message? Although if I needed to
command a thread to do something I'd probably have a separate class
and separate thread for that purpose.

James
--
James


On 26 February 2014 14:19, David Palao <dpalao.python at gmail.com> wrote:
> 2014-02-25 11:52 GMT+01:00 James Chapman <james at uplinkzero.com>:
>> Hello tutors
>>
>> I'm curious about managers and when to use them.
>> For example, I see they offer a Queue() for sharing a Q between
>> processes, but if I create a Q in the parent process and pass it down
>> to child processes, then they can put messages into that Q just fine,
>> and I presume the same thing for other objects available under the
>> managers package.
>>
>> So unless the other process is on a different machine, is there a
>> reason to use a manager?
>>
>> Does anyone have any use case examples or snippets I could look at even?
>>
>> Thanks in advance
>> James
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>
> Hello,
> I asked myself the same question when I started using multiprocessing
> time ago. So I was very happy when I saw the question by James.
>
> From my limited knowledge, I would say that a Manager can be useful
> when processes are distributed across different hosts, or if the
> exchange of information between processes is more complex than just a
> couple of synchronization primitives.
>
> Best