[Python-ideas] Application awareness of memory storage classes

[Terry Reedy]

On 5/16/2016 8:35 PM, R. David Murray wrote:
> I'm currently working on a project for Intel involving Python and directly
> addressable non-volatile memory.  See https://nvdimm.wiki.kernel.org/
> for links to the Linux features underlying this work, and http://pmem.io/
> for the Intel initiated open source project I'm working with whose intent
> is to bring support for DAX NVRAM to the application programming layer
> (nvml currently covers C, with C++ support in process, and there are
> python bindings for the lower level (non-libpmemobj) libraries).

How is non-volatile NVRAM different from static SRAM?

> tldr: In the future (and the future is now) application programs will
> want to be aware of what class of memory they are allocating:

What I want is to be, if anything, less aware.  I remember fiddling with 
register declarations in C.  Then is was discovered that compilers can 
allocate registers better than move people, so that 'register' is 
deprecated for most C programmers.  I have never had to worry about the 
L1, L2, L3 on chip caches, though someone has to.

I have long thought that I should be able to randomly access data on 
disk the same way I would access that same data in RAM, and not have to 
fiddle with seek and so on.  Virtual memory is sort of like this, except 
that it uses the disk as a volatile* cache for RAM objects.  (* Volatile 
in the sense that when the program ends, the disk space is freed for 
other use, and is inaccessible even if not.)  Whereas I am thinking of 
using RAM as a cache for a persistent disk object.  A possible user API 
(assuming txt stored as a python string with k bytes per char):

cons = str(None, path="c:/user/terry/gutenburg/us_constitution.txt")
# now to slice like it was in ram
preamble = cons[:cons.find(section_marker)]

Perhaps you are pointing to being able to make this possible, from the 
implementer side.

The generic interfaces would be bytes(None, path=) (read only) and 
bytearray(None, path=) (read-write).

A list does not seem like a good candidate for static mem, unless insert 
and delete are suppressed/unused.

[snip]

If static objects were **always** aligned in 4-byte boundaries, then the 
lowest 2 bits could be used to indicate memory type.  To not slow down 
programs, this should be supported by the CPU address decoder.  Isn't 
Intel thinking/working on something like this?

-- 
Terry Jan Reedy