Standalone ZODB - Connections and multiple threads

[johndheintz at my-deja.com]

I have a proposal and patch at
http://dev.zope.org/Wikis/DevSite/Proposals/ExplicitTransactions that I
think addresses these issues.  See my comments below.

Jim Fulton wrote:
>
> David Bolen wrote:
> >
> > I'm wondering if someone here might be able to help out with a query
> > about using ZODB standalone without Zope?
>
> Yup.
>
> > ZODB3 is documented as supporting multiple connections to a storage
> > which supports independent object caches for multiple threads.
> > However, although I've looked around at a bunch of documents, I can't
> > seem to find if this is required or simply available.

I am planning of releasing template code for how I am using ZODB with
omniORB.  This code needs to deal with the ORB calling into different
Connection caches of Persistent object from a different thread for each
CORBA request.

I currently don't have the requirement to do thread-syncronization
within the objects from the same DB Connection, but I do have the
problem of not being able to control which thread is doing the work.

> It isn't required, but it is by far the common usage pattern.
>
> Hm, I guess that the current implementation actually makes it
> hard to institute a different policy.

I think that the current implementation makes if very hard to even
override get_transaction() because the Transaction module is delete from
the namespace after import.  (See my proposal for how much I would like
to change it ;-)

> > That is, if I'm
> > using ZODB in a multi-threaded Python application, must I create
> > separate connections for each thread, or can they share a single
> > connection.
>
> Multiple threads *can* share a single connection, but
> if you do this, you'll need to:
>
>   - Perform whatever locking is required to serialize
>     access to the shared objects,
>
>   - You'll need to override get_transaction with a version
>     that doesn't manage transactions by thread.
>
> > In my case, the application is already securing itself against
> > simultaneous access to the objects that I'm making persistent, so I
> > don't really think I need the distinct connections, but I ran into one
> > case where it seemed that the thread within which a
> > get_connection().commit() was performed was important to getting the
> > object changes really committed.
>
> The logic for the Persistent mix-in class calls get_transaction
> to get the current transaction when it needs to
> register that an object has changed.  The standard get_transaction
> manages transaction objects per thread.  In addition, the
> function is cached the first time it is called, to avoid
> global lookups each time.

The ExplicitTransactions proposal is based on the claim that this logic
is not necessary and makes it overly difficult to implement other
transaction policies.  The change that the patch makes is to have the
Persistent mix-in class lookup the appropriate Transaction from its
Connection object, not from a global function.  It also re-exposes the
Transaction module so client code can directly create Transactions that
DB Connections can be associated with.

I think that this change makes it more intuitive when multiple threads
are being used to modify Persistent object from multiple DB
Connections.  (This is the circumstance when using CORBA).


> You can replace get_transacton by installing a different version
> in builtins, but you will need to do so immediately after importing
> ZODB, to make sure your version gets cached.
>
> > If I did create separate connections, my understanding is that the
> > object stores referenced beneath the root obtained from each
> > connection would be distinct in memory, but in my case the persistent
> > object information is truly being shared among the execution threads
> > (with appropriate locking), so I'd think having separate copies of
> > them would only complicate things.
>
> Why do your threads actually need to share the same object (copy)?

For my application it is CORBA that controls the threads and every
different CORBA Request can be run from a different thread.

> Jim
>
> --
> Jim Fulton           mailto:jim at digicool.com
> Technical Director   (888) 344-4332              Python Powered!
> Digital Creations    http://www.digicool.com     http://www.python.org
>
> Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
> address may not be added to any commercial mail list with out my
> permission.  Violation of my privacy with advertising or SPAM will
> result in a suit for a MINIMUM of $500 damages/incident, $1500 for
> repeats.

Thanks,
John

--
. . . . . . . . . . . . . . . . . . . . . . . .

John D. Heintz | Senior Engineer

1016 La Posada Dr. | Suite 240 | Austin TX 78752
    T 512.633.1198 | jheintz at isogen.com

w w w . d a t a c h a n n e l . c o m


Sent via Deja.com http://www.deja.com/
Before you buy.