[Python-ideas] multiprocessing IPC
shibturn at gmail.com
Mon Feb 13 01:31:42 CET 2012
On 12/02/2012 8:33pm, Sturla Molden wrote:
> It seems that on Linux /tmp is backed by shared memory.
> Which sounds rather strange to a Windows user, as the raison d'etre for
> tempfiles is temporary storage space that goes beyond physial RAM.
In reality /tmp is backed by swap space, so physical RAM does not impose
a limit. Anonymous mmaps are also backed by swap space.
> I've also read that the use of ftruncate in this context can result in
Isn't that if you truncate the file to a smaller size *after* it has
been mapped. As far as I am aware, using ftruncate to set the length
*before* it can be mapped for the first time is standard practice and
>> Below is Blob class which seems to work. Note that the process which
>> created the blob needs to wait for the other process to unpickle it
>> before allowing it to be garbage collected.
> I would look at kernel refcounts before unlinking. (But I am not that
> familiar with Linux.)
Even if you have automatic refcounting like on Windows, you still need
to cope with lifetime management issues. If you put an object on a
queue it may be a long time before the target process will unpickle the
object and increase its refcount, and you must not decref the object
until it has, or else it will disappear.
I don't know how to get the ref count for a file descriptor on Unix.
(And posix shared memory does not seems to get a refcount either, even
though System V shared memory does.)
More information about the Python-ideas