[Python-ideas] Application awareness of memory storage classes

[Robert Collins]

On 19 May 2016 at 10:41, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Terry Reedy wrote:
>>
>> I have long thought that I should be able to randomly access data on disk
>> the same way I would access that same data in RAM, and not have to fiddle
>> with seek and so on.
>
>
> I gather Multics was like that -- the only way to access a
> file was to map it into memory, like mmap() in unix.
>
> I believe that some unix kernels also work that way
> internally, with the file I/O layer being built on
> top of the VM layer.
>
> I can see some problems with trying to build that sort
> of thing too deeply into Python, though. One of them
> is what happens when two processes map in the same file
> of Python objects. Some kind of super-GIL that works
> between processes would be needed to manage the
> reference counts.

Another consideration is how to deal with processes that crash (or
machines that crash).

With transient memory, reference counts are always correct whenever
the GIL is released...or the process has just segfaulted/kill
-9/whatever.

If reference counts are stored in NVRAM, then when a process fails
while while it holds a reference from its transient memory to the
NVRAM stored object, those references will leak.

For me at least, I'd want to treat NVRAM as a form of disk, and not
try to use transient ram idioms with it - they seem unsafe.

-Rob