Persistent variable in subprocess using multiprocessing?

Piet van Oostrum piet at cs.uu.nl
Fri Jul 17 04:26:27 EDT 2009


There is stil something not clear in your description.

>m> I'm using multiprocessing to spawn several subprocesses, each of which
>m> uses a very large data structure (making it impractical to pass it via
>m> pipes / pickling). I need to allocate this structure once when the
>m> process is created and have it remain in memory for the duration of
>m> the process. 

I have read this as that every subprocess has its own large
data structure and that there is no connection between these.

But seeing where the discussion is going I guess there might be
different interpretations. So can you enlighten us how the situation is?

1. Each subprocess has a copy of a data structure that is prepared by the
   master process. Therefore you want it to be passed by the fork
   1a. the data structure is constant i.e. the subprocess doesn't change it
   1b. the subprocess makes changes in its copy
2. Each subprocess has a seperate data structure not equal to the others
3. Something else.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org



More information about the Python-list mailing list