Getting references to objects without incrementing reference counters

[Artur Siekielski]

On Nov 15, 1:03 am, de... at web.de (Diez B. Roggisch) wrote:
> You don't say what data you share, and if all of it is needed for each
> child. So it's hard to suggest optimizations.

Here is an example of such a problem I'm dealing with now: I'm
building large index of words in memory, it takes 50% of RAM. After
building it I want to enable querying it using multiple processors
enabling fast handling of multiple clients at once (communication is
done with sockets). So I create the index in the parent process and
than fork children that handle client queries. But after a short time
only traversing the index by children makes them use much memory,
resulting shortly in Out Of Memory Errors.

> And AFAIK there is no
> built-in way of doing what you want. It's complex and error-prone.

Having read-only objects is complex and error-prone?

> Maybe mmap + (struct|pickle) help, if what you need can be formulated in a way
> that traversing the whole data piecewise by explicitly
> marshaling-demarshaling data?

In case of this word index it is impossible... And probably would be
too slow. Using mmap and mapping the object structure into structs
should probably work...