[Persistence-sig] A simple Observation API

[Phillip J. Eby]

At 04:02 PM 7/30/02 -0400, Shane Hathaway wrote:
>
>I see now.  From one perspective, this problem is a side effect of keeping
>transaction participants registered between transactions, as you've been
>suggesting.  ZODB 3's transaction manager would normally have no problem
>with this, since DM3 and DM2 would only get added to the transaction once
>DM1 started writing.  The implicit order would solve the problem.
>
>Unfortunately, this solution has a weakness--if some other data manager
>wrote unrelated data to DM3 or DM2 before DM1 wrote its data, the implicit
>order would be incorrect.  Thus the need for transaction agents, which
>guarantee a specific order (if I recall correctly).

Right; this is why I described both ZPatterns and TransactionAgents as
being hacks.  We rely on the use of registration order to force things to
work in most cases, or re-registration.  It's a pretty ugly hack.


>Write-through mode seems like a performance killer for many applications.
>What about this: transaction participants could tell the transaction that
>even though their prepare() method has been called already, they need it
>called again.

The only drawback I'm aware of for that approach, is that it leads to an
infinite loop instead of a stack overflow, in the event you accidentally
create a circular dependency graph.  The infinite loop doesn't produce a
traceback, and thus doesn't show you *how* you created the circularity.

I suppose you could require that the number of times you loop through a
list sending prepare() calls is no greater than some multiplier of the
total number of participants, and then at least you could detect what seems
like a runaway dependency.  Printing out what the loop *was* could be hard,
though, and the information would not show you as directly how the loop
occurred.  But I'm willing to bend on this point, since I think even
accidental circularity is likely to be rare, that when it does occur you're
likely to have known there was a risk of it, and that you'll be likely to
know where to look for where it occurred.  It's a lot different than the
risk of out-of-order commits, which could occur with explicit dependency
management for even very simple scenarios.

Also, I think a different method should be used for the second prepare()
call - perhaps a flush() method.  That way, prepare() won't need to be able
to be called twice during the same commit, which I can see some problems
with.  prepare() could simply call flush(), or perhaps the transaction
could do it.  flush() should be written so as to be usable at any point in
the transaction, since it'll presumably be used to implement savepoints as
well, and in some cases to ensure an underlying DB is up-to-date before
performing a query.

I do like the simplification of not needing a "write-through" mode,
although in reality all we are doing is replacing it with a "re-flush"
mode.  That is, once a participant receives prepare(), it must respond to
any future change notifications by requesting a re-call of flush() by the
transaction.

By the way, I'd still like to have the option of having participants join a
transaction "permanently", in order to avoid all of the state management
code that such things currently require.

With the exception of the above issues, I'm good with this approach.
Brilliant idea, Shane.  :)