Going ZODB

[David Bolen]

Steven Haryanto <steven at haryan.to> writes:

> I am curious as to how many people in this list actually
> use ZODB in production environment and what is the typical
> uses of it (especially ZODB with relational/berkeleydb
> storage). Aside from along with Zope, of course.

We use it as the persistance store for an internal (it's production
internally, but not something shipped to customers) scheduling system
that handles execution of our network management tasks.  The main
scheduler is a process that can be attached to by any number of agents
on a collection of execution machines, and the scheduler dispatches
tasks as appropriate based on a variety of parameters, priorities,
resource requirements and so on.

The ZODB object store holds any of our objects related to the state of
the system.  We're not fancy - we commit changes fairly tightly with
where they are made, and we manage our own interlocked access to the
data from the multiple scheduler threads.  But it works great to
ensure that we can recover from failure at pretty much any point to
precisely the same scheduling state we were in before.  It's not all
that different to simply unpickling the information at startup and
pickling it at shutdown except that we have continuous persistence
(well, at the granularity of any significant state change) in the case
of failure.  Since we're not versioning our objects, we have a
periodic cycle where we pack the database to free space.

Due to this usage, moving away from ZODB if we had to isn't hard - we
just remove the commit stuff and read/write the state ourselves.  But
unless we had fairly rapid checkpoints, we'd be at more risk than with
what we do with ZODB now.  And we're also getting some memory benefits
with ZODB in terms of how it cleans out objects as appropriate and
then transparently brings them back in.  So when we start up, we don't
have all the history objects in memory - but they are brought in
transparently to the code should an administrative request need them.

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/