[Python-ideas] Application awareness of memory storage classes

[Nick Coghlan]

On 24 May 2016 at 17:59, M.-A. Lemburg <mal at egenix.com> wrote:
> On 24.05.2016 04:35, Nick Coghlan wrote:
>> On 24 May 2016 at 05:33, R. David Murray <rdmurray at bitdance.com> wrote:
>>> On the other hand, the fact that *all* in-block persistent object state
>>> gets restored on block abort, regardless of where the exception occurred,
>>> could be somewhat confusing to Python programmers.
>>
>> It occurs to me that you may want to talk to Mike Bayer and start
>> digging into the way SQL Alchemy session objects work, as what you're
>> describing here is starting to sound a *lot* like working with the SQL
>> Alchemy ORM:
>>
>> - you have stateful objects declared as classes (SQL Alchemy models)
>> - you have volatile state in memory (the SQL Alchemy session)
>> - you have transactional commits to persistent storage (also the session)
>>
>> The difference would be that instead of working with a backing
>> relational data store, you're working directly with persistent memory.
>> It would also be interesting to see how much of this could be emulated
>> with mmap and memoryview, permitting such code to have a slower
>> fallback in cases where actual NVRAM wasn't available.
>
> SQLAlchemy relies on underlying Python DB-API modules to
> work with the database, so in addition to the above you
> also have state in objects of those DB-API modules and
> these usually reference objects or buffers in the low-level
> database interface libraries to which Python has no direct
> access.

I'm not talking about using SQL Alchemy itself for this (as you say,
the implementation details of the underlying persistence model are all
wrong), I'm talking about studying the way SQL Alchemy session
specifically, and the way it manages the in-memory state while a
transaction is open, and then flushes that state to the database when
the transaction is completed. It's a thought prompted by a
presentation Mike gave at PyCon US 2013:
https://speakerdeck.com/pyconslides/sqlalchemy-session-in-depth-by-mike-bayer

Now, David might investigate that, and decide it's more complexity
than is needed given persistent RAM, but it's the nearest
already-in-existence thing I know to what I believe he's aiming to
build, and it also handles the distinction between persistent state
(database row for SQL Alchemy, memory allocated in NVRAM for this) and
ephemeral state (elements dynamically allocated in normal RAM for
both).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia