Standalone ZODB - Connections and multiple threads

johndheintz at my-deja.com johndheintz at my-deja.com
Sun Nov 19 22:27:10 EST 2000


I have a proposal and patch at
http://dev.zope.org/Wikis/DevSite/Proposals/ExplicitTransactions that I think
addresses these issues.  See my comments below.

Jim Fulton wrote:
>
> David Bolen wrote:
> >
> > I'm wondering if someone here might be able to help out with a query
> > about using ZODB standalone without Zope?
>
> Yup.
>
> > ZODB3 is documented as supporting multiple connections to a storage
> > which supports independent object caches for multiple threads.
> > However, although I've looked around at a bunch of documents, I can't
> > seem to find if this is required or simply available.

I am planning of releasing template code for how I am using ZODB with
omniORB.  This code needs to deal with the ORB calling into different
Connection caches of Persistent object from a different thread for each CORBA
request.

I currently don't have the requirement to do thread-syncronization within the
objects from the same DB Connection, but I do have the problem of not being
able to control which thread is doing the work.

> It isn't required, but it is by far the common usage pattern.
>
> Hm, I guess that the current implementation actually makes it
> hard to institute a different policy.

I think that the current implementation makes if very hard to even override
get_transaction() because the Transaction module is delete from the namespace
after import.  (See my proposal for how much I would like to change it ;-)

> > That is, if I'm
> > using ZODB in a multi-threaded Python application, must I create
> > separate connections for each thread, or can they share a single
> > connection.
>
> Multiple threads *can* share a single connection, but
> if you do this, you'll need to:
>
>   - Perform whatever locking is required to serialize
>     access to the shared objects,
>
>   - You'll need to override get_transaction with a version
>     that doesn't manage transactions by thread.
>
> > In my case, the application is already securing itself against
> > simultaneous access to the objects that I'm making persistent, so I
> > don't really think I need the distinct connections, but I ran into one
> > case where it seemed that the thread within which a
> > get_connection().commit() was performed was important to getting the
> > object changes really committed.
>
> The logic for the Persistent mix-in class calls get_transaction
> to get the current transaction when it needs to
> register that an object has changed.  The standard get_transaction
> manages transaction objects per thread.  In addition, the
> function is cached the first time it is called, to avoid
> global lookups each time.

The ExplicitTransactions proposal is based on the claim that this logic is
not necessary and makes it overly difficult to implement other transaction
policies.  The change that the patch makes is to have the Persistent mix-in
class lookup the appropriate Transaction from its Connection object, not from
a global function.  It also re-exposes the Transaction module so client code
can directly create Transactions that DB Connections can be associated with.

I think that this change makes it more intuitive when multiple threads are
being used to modify Persistent object from multiple DB Connections.  (This
is the circumstance when using CORBA).


> You can replace get_transacton by installing a different version
> in builtins, but you will need to do so immediately after importing
> ZODB, to make sure your version gets cached.
>
> > If I did create separate connections, my understanding is that the
> > object stores referenced beneath the root obtained from each
> > connection would be distinct in memory, but in my case the persistent
> > object information is truly being shared among the execution threads
> > (with appropriate locking), so I'd think having separate copies of
> > them would only complicate things.
>
> Why do your threads actually need to share the same object (copy)?

For my application it is CORBA that controls the threads and every different
CORBA Request can be run from a different thread.

> Jim
>
> --
> Jim Fulton           mailto:jim at digicool.com
> Technical Director   (888) 344-4332              Python Powered!
> Digital Creations    http://www.digicool.com     http://www.python.org
>
> Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
> address may not be added to any commercial mail list with out my
> permission.  Violation of my privacy with advertising or SPAM will
> result in a suit for a MINIMUM of $500 damages/incident, $1500 for
> repeats.

Thanks,
John

--
. . . . . . . . . . . . . . . . . . . . . . . .

John D. Heintz | Senior Engineer

1016 La Posada Dr. | Suite 240 | Austin TX 78752
    T 512.633.1198 | jheintz at isogen.com

w w w . d a t a c h a n n e l . c o m


Sent via Deja.com http://www.deja.com/
Before you buy.



More information about the Python-list mailing list