[Persistence-sig] Re: ACID, savepoints, and exceptions (was re: "Straw Man"transaction API)

[Jeremy Hylton]

>>>>> "PJE" == Phillip J Eby <pje@telecommunity.com> writes:

  PJE> During the past week, I've been writing a TransactionService
  PJE> for PEAK, specifically designing it to allow
  PJE> interaction/adaptation to the new ZODB4 transaction API, and
  PJE> extending it to support the multi-prepare and durable
  PJE> subscription models that I need for my applications and
  PJE> framework projects.

Glad to hear you're making progress.  I've been completely swamped
with other things, so I haven't had any time for ZODB4 since we last
talked. 

  PJE> In my API I've standardized on a 'CannotRevertException' when
  PJE> rollback to a savepoint is not possible, and added a
  PJE> 'NullSavepoint' object which can be returned by an object that
  PJE> has nothing to do on rollback.

NullSavepoint is just an implementation convenience, right?

  PJE> An open issue that needs to be addressed, however, is the
  PJE> question of rolling back more than once to the same savepoint.
  PJE> In some ways, it's a very handy capability, but I'm not sure
  PJE> which databases support this. 

Let me ask the question the other way:  Of the databases that support
savepoints, which ones don't support this?

  PJE>                                I'm therefore inclined to say we
  PJE> should explicitly say that a savepoint can be rolled back at
  PJE> most once (since some savepoints may not be able to be rolled
  PJE> back).

I want savepoints that can be returned to multiple times.  If a
database supports savepoints at all, I don't see why it wouldn't
support multiple rollbacks.  (If it didn't, an adapter could
just call savepoint() as part of finishing each rollback().)  Multiple
rollbacks is necessary to support nested transactions.

  PJE> Another open issue: what happens if a rollback fails?  Is the
  PJE> transaction "hosed" at that point?

I think it is.

  PJE>                                     What if five data managers
  PJE> roll back, and the sixth one fails?

Exactly.  If you can't be sure each of the data managers is in a
consistent state, you need to abort the transaction.

  PJE>                                      This suggests adding a
  PJE> 'canRollback()' method to the interface, such that a rollback
  PJE> aggregator can check that its aggregated savepoints can
  PJE> actually be rolled back, so that "CannotRevert" errors don't
  PJE> cause the transaction to be hosed.  

It's probably good to have some way to query this, although I feel
like the predicate methods for testing features haven't worked out all
that well in the ZODB3 storage api.  What about that client code has
access to would support the canRollback() method?  It seems like it
depends on which objects are participating in the transaction.

I tend more towards an ask for forgiveness (AFF) than a look before
you leap (LBYL).  If savepoint() returned None when it wasn't possible
to rollback, that would be good enough, no?  The clients know, for
their specific transaction, whether rollback is going to work.  The
savepoint() presumably hasn't caused too much extra work in those
cases. 

  PJE>                                     However, the issue of
  PJE> another type of exception occurring during rollback still must
  PJE> be addressed.

Yes.

  >> I think I'm also in favor of the new abort semantics.  ZODB3
  >> would abort the transactions -- call abort() on all the data
  >> managers -- if an error occurred during a commit.  The new code
  >> requires that the user do this instead.  I think that's better,
  >> because it leaves the state of the objects intact if the code
  >> wants to analyze what went wrong before retrying the transaction.

  PJE> The interesting question here again is, is the transaction
  PJE> "hosed"?  Should there be a flag that says, "you can't do
  PJE> anything to this transaction but abort it"?

Yes, and yes.

  PJE> To put it in broader terms, if *any* exception is thrown during
  PJE> execution of a transaction-related method, should we consider
  PJE> the transaction unrecoverable?

Yes.  If a resource manager raises an unexpected exception, you've got
no idea what state its in or whether it can/has committed the data.

  PJE> I'm inclined to say yes, because I can think of too many code
  PJE> paths in both my and the ZODB4 transaction code where it
  PJE> becomes nearly impossible to guarantee a "clean" state when an
  PJE> exception occurs.  By definition, if code called by the
  PJE> transaction system raises an exception, it is announcing that
  PJE> it cannot satisfy its contract with the transaction.
  PJE> Therefore, the transaction cannot be certain of satisfying its
  PJE> contract with the application for a clean commit.

Right.

  PJE> Another issue here is clean aborts.  If an error is raised by a
  PJE> data manager during abort, what should the semantics be?  Older
  PJE> ZODB transaction classes wrap every data manager abort call in
  PJE> a try-except that ensures that *all* the abort methods get
  PJE> called, even if several of them raise errors.  The new ZODB4
  PJE> transaction API doesn't do this, and thus can fail to
  PJE> completely roll back a transaction.

I tried to do as little as possible within the commit() implementation
to deal with errors.  I figured if an error occurs, the client had
better abort the transaction explicitly.  The documentation for ZODB3
said that clients needed to do this, but the implementation didn't
work that way.

Jeremy