[Python-ideas] multiprocessing IPC

[Mike Meyer]

On Sun, Feb 12, 2012 at 3:33 PM, Sturla Molden <sturla at molden.no> wrote:
> Den 12.02.2012 16:20, skrev shibturn:
>> But if his /tmp is a tmpfs file system (which it usually is on Linux) then
>> I think it is entirely equivalent.  Or he could create the file in /dev/shm
>> instead.
> It seems that on Linux /tmp is backed by shared memory.
> Which sounds rather strange to a Windows user, as the raison d'etre for
> tempfiles is temporary storage space that goes beyond physial RAM.

That's what /tmp was created for on Unix as well. But we've since
added virtual memory for that same purpose. Modern kernel virtual
address spaces are bigger than disks, and the IO and VM subsystem
buffer caches have similar performance, and may even share buffers. So
the major difference between memory-backed and fs-backed /tmp is that
an fs-backed one survives a reboot, which creates security issues on
multiuser systems.

In theory, you could create a file on a memory-backed /tmp that's
bigger than any data structure your process can hold.  But modern
software tends to use /tmp for things that need to be shared between
processes (unix-domain sockets, lock files, etc), and legacy software
is usually quite happy with a few tens of megabytes on /tmp. So it's
rather common for a systems per-process virtual address limit to be
bigger than /tmp.

    <mike