Getting references to objects without incrementing reference counters

John Nagle nagle at
Mon Nov 15 22:06:45 CET 2010

On 11/14/2010 11:08 AM, Artur Siekielski wrote:
> Hi.
> I'm using CPython 2.7 and Linux. In order to make parallel
> computations on a large list of objects I want to use multiple
> processes (by using multiprocessing module). In the first step I fill
> the list with objects and then I fork() my worker processes that do
> the job.
> This should work optimally in the aspect of memory usage because Linux
> implements copy-on-write in forked processes.

    There used to be a memory leak when using Pickle to talk
to subprocesses.  See what I wrote at


The Pickle module has an optimization scheme and cache which
requires that both sender and receiver keep a reference to each
object transmitted.  Cache clearing wasn't well worked out for
Pickle object reuse, and the receive end had a memory leak.

    Did that get fixed?

				John Nagle

More information about the Python-list mailing list