[Python-ideas] Application awareness of memory storage classes
tjreedy at udel.edu
Tue May 17 17:07:59 EDT 2016
On 5/16/2016 8:35 PM, R. David Murray wrote:
> I'm currently working on a project for Intel involving Python and directly
> addressable non-volatile memory. See https://nvdimm.wiki.kernel.org/
> for links to the Linux features underlying this work, and http://pmem.io/
> for the Intel initiated open source project I'm working with whose intent
> is to bring support for DAX NVRAM to the application programming layer
> (nvml currently covers C, with C++ support in process, and there are
> python bindings for the lower level (non-libpmemobj) libraries).
How is non-volatile NVRAM different from static SRAM?
> tldr: In the future (and the future is now) application programs will
> want to be aware of what class of memory they are allocating:
What I want is to be, if anything, less aware. I remember fiddling with
register declarations in C. Then is was discovered that compilers can
allocate registers better than move people, so that 'register' is
deprecated for most C programmers. I have never had to worry about the
L1, L2, L3 on chip caches, though someone has to.
I have long thought that I should be able to randomly access data on
disk the same way I would access that same data in RAM, and not have to
fiddle with seek and so on. Virtual memory is sort of like this, except
that it uses the disk as a volatile* cache for RAM objects. (* Volatile
in the sense that when the program ends, the disk space is freed for
other use, and is inaccessible even if not.) Whereas I am thinking of
using RAM as a cache for a persistent disk object. A possible user API
(assuming txt stored as a python string with k bytes per char):
cons = str(None, path="c:/user/terry/gutenburg/us_constitution.txt")
# now to slice like it was in ram
preamble = cons[:cons.find(section_marker)]
Perhaps you are pointing to being able to make this possible, from the
The generic interfaces would be bytes(None, path=) (read only) and
bytearray(None, path=) (read-write).
A list does not seem like a good candidate for static mem, unless insert
and delete are suppressed/unused.
If static objects were **always** aligned in 4-byte boundaries, then the
lowest 2 bits could be used to indicate memory type. To not slow down
programs, this should be supported by the CPU address decoder. Isn't
Intel thinking/working on something like this?
Terry Jan Reedy
More information about the Python-ideas