[New-bugs-announce] [issue20660] Starting a second multiprocessing.Manager causes INCREF on all object created by the first one.

[mythsmith]

New submission from mythsmith:

I seems that upon the start of a second manager, all objects referenced in the first one gets an INCREF. On the third start, all objects created by the first and the second manager get another INCREF. And so on. I cannot understand why the start of a totally new manager, in a new process, will cause all these INCREF about object hosted on other managers - which cause the program to take forever to start/stop.

This small script fully reproduces the behaviour, tested in python 2.7 and 3.2:

from __future__ import print_function
import multiprocessing, logging
# # Activate multiprocessing logging
mplog = multiprocessing.get_logger()
mplog.setLevel(multiprocessing.util.DEBUG)
mplog.addHandler(logging.StreamHandler())
objs=[]
def newman(n=50):
    global objs
    m=multiprocessing.Manager()
    print('created')
    for i in range(n):
        objs.append(m.Value('i',i)) 
    return m    

print('#### first man')
m1=newman()

print ('#### second man')
m2=newman()

print ('#### third man')
m3=newman(0)

(Output is attached)

After the start of the first manager, the logger prints out the messages relative to the creation of the first 50 objects.

But when the second manager is starting - before any object was created by it - the logger prints out exactly 50 INCREF messages.Then follows the messages relating to the creation of the 50 new objects on manager 2.

When the third manager starts - before any object was created by it - 100 more INCREF messages are printed.

No object creation message is seen after m3 creation, as I passed 0 to the newman() function.

When the program ends, a similar amount of DECREF messages is printed.

It seems that, when I start a new manager, it creates a reference to all objects referenced by previous managers. In a big application this translates into extremely slow startup/shutdown.

----------
components: Library (Lib)
files: output.txt
messages: 211421
nosy: mythsmith
priority: normal
severity: normal
status: open
title: Starting a second multiprocessing.Manager causes INCREF on all object created by the first one.
type: behavior
versions: Python 2.7, Python 3.2
Added file: http://bugs.python.org/file34121/output.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20660>
_______________________________________