On 24.05.2016 04:35, Nick Coghlan wrote:
On 24 May 2016 at 05:33, R. David Murray
wrote: On the other hand, the fact that *all* in-block persistent object state gets restored on block abort, regardless of where the exception occurred, could be somewhat confusing to Python programmers.
It occurs to me that you may want to talk to Mike Bayer and start digging into the way SQL Alchemy session objects work, as what you're describing here is starting to sound a *lot* like working with the SQL Alchemy ORM:
- you have stateful objects declared as classes (SQL Alchemy models) - you have volatile state in memory (the SQL Alchemy session) - you have transactional commits to persistent storage (also the session)
The difference would be that instead of working with a backing relational data store, you're working directly with persistent memory. It would also be interesting to see how much of this could be emulated with mmap and memoryview, permitting such code to have a slower fallback in cases where actual NVRAM wasn't available.
SQLAlchemy relies on underlying Python DB-API modules to work with the database, so in addition to the above you also have state in objects of those DB-API modules and these usually reference objects or buffers in the low-level database interface libraries to which Python has no direct access. As soon as you have memory in use which is not fully managed by Python, I don't think there's any way to implement transactions on memory in a meaningful way. The possible side effect in the unmanaged blocks would render such transactions meaningless, since a rollback in those would still leave you with the changes in the unmanaged blocks (other parts of the system). Now, back on topic: for writing to NVRAM, having a transaction mechanism in place does make sense, but it would have to be clear that only the bits stored in NVRAM are subject to the transaction. The reason here being that a failure while writing to NVRAM could potentially cause your machine to no longer boot. For volatile RAM, at worst, the process will die, but not have much effect on other running parts of the system, so there is less incentive to have transactions (unless, of course, you are deep into STM and want to work around the GIL :-)). Given that Armin Rigo has been working on STM for years, I'd suggest to talk to him about challenges and solutions for transactions on memory. My take on all this would be to work with NVRAM as block rather than single memory cells: allocate a lock on the NVRAM block try: copy the block into DRAM run manipulations in DRAM block write back DRAM block finally: release lock on NVRAM block so instead of worrying about a transaction failing while manipulating NVRAM, you only make sure that you can lock the NVRAM block and provide an atomic "block write to NVRAM" functionality. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 24 2016)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/