I'm pleased to announce the release of ZODB 3.2 beta 1. We have completed all the new features planned for ZODB 3.2 and have wrung out a number of bugs. I expect this release is still a bit rough, so I expect to do another beta release. ZODB 3.2 has a number of new features and improvements over ZODB 3.1: - improve performance and stability of ZEO - new ZEO authentication protocol - new configuration language, ZConfig, for databases, storages, and ZEO servers - many bug fixes This release of ZODB has been tested with Python 2.1.3, 2.2.3, and 2.3b1. It should also be compatible with Zope 2.6; Zope users should be able to install the new ZODB code on top of an existing Zope 2.6 installation. You can download the source release and Windows installers for 2.1 and 2.2 from http://www.zope.org/Products/ZODB3.2. I've included the most recent entries from NEWS.txt below. Jeremy What's new in ZODB3 3.2 beta 1 ============================== Release date: 30-May-2003 ZODB ---- Invalidations are now processed atomically. Each transaction will see all the changes caused by an earlier transaction or none of them. Before this patch, it was possible for a transaction to see invalid data because it saw only a subset of the invalidations. This is the most likely cause of reported BTrees corruption, where keys were stored in the wrong bucket. When a BTree bucket splits, the bucket and the bucket's parent are both modified. If a transaction sees the invalidation for the bucket but not the parent, the BTree in memory will be internally inconsistent and keys can be put in the wrong bucket. The atomic invalidation fix prevents this problem. A number of minor reference count fixes in the object cache were fixed. That's the cPickleCache.c file. It was possible for a transaction that failed in tpc_finish() to lose the traceback that caused the failure. The transaction code was fixed to report the original error as well as any errors that occur while trying to recover from the original error. The "other" argument to copyTransactionsFrom() only needs to have an .iterator() method. For convenience, change FileStorage's and BDBFullStorage's iterator to have this method, which just returns self. Mount points are now visible from mounted objects. Fixed memory leak involving database connections and caches. When a connection or database was closed, the cache and database leaked, because of a circular reference involving the cache. Fixed the cache to explicitly clear out its contents when its connection is closed. The ZODB cache has fewer methods. It used to expose methods that could mutate the dictionary, which allowed users to violate internal invariants. ZConfig ------- It is now possible to configure ZODB databases and storages and ZEO servers using ZConfig. ZEO & zdaemon ------------- ZEO now supports authenticated client connections. The default authentication protocol uses a hash-based challenge-response protocol to prove identity and establish a session key for message authentication. The architecture is pluggable to allow third-parties to developer better authentication protocols. There is a new HOWTO for running a ZEO server. The draft in this release is incomplete, but provides more guidance than previous releases. See the file Doc/ZEO/howto.txt. The ZEO storage server's transaction timeout feature was refactored and made slightly more rebust. A new ZEO utility script, ZEO/mkzeoinst.py, was added. This creates a standard directory structure and writes a configuration file with mostly default values, and a bootstrap script that can be used to manage and monitor the server using zdctl.py (see below). Much work was done to improve zdaemon's zdctl.py and zdrun.py scripts. (In the alpha 1 release, zdrun.py was called zdaemon.py, but installing it in <prefix>/bin caused much breakage due to the name conflict with the zdaemon package.) Together with the new mkzeoinst.py script, this makes controlling a ZEO server a breeze. A ZEO client will not read from its cache during cache verification. This fix was necessary to prevent the client from reading inconsistent data. The isReadOnly() method of a ZEO client was fixed to return the false when the client is connected to a read-only fallback server. The sync() method of ClientStorage and the pending() method of a zrpc connection now do both input and output. The short_repr() function used to generate log messages was fixed so that it does not blow up creating a repr of very long tuples. Storages -------- FileStorage has a new pack() implementation that fixes several reported problems that could lead to data loss. Two small bugs were fixed in DemoStorage. undoLog() did not handle its arguments correctly and pack() could accidentally delete objects created in versions. Fixed trivial bug in fsrecover that prevented it from working at all. FileStorage will use fsync() on Windows starting with Python 2.2.3. FileStorage's commit version was fixed. It used to stop after the first object, leaving all the other objects in the version. BTrees ------ Trying to store an object of a non-integer type into an IIBTree or OIBTree could leave the bucket in a variety of insane states. For example, trying b[obj] = "I'm a string, not an integer" where b is an OIBTree. This manifested as a refcount leak in the test suite, but could have been much worse (most likely in real life is that a seemingly arbitrary existing key would "go missing"). When deleting the first child of a BTree node with more than one child, a reference to the second child leaked. This could cause the entire bucket chain to leak (not be collected as garbage despite not being referenced anymore). Other minor BTree leak scenarios were also fixed. Tools ----- New tool zeoqueue.py for parsing ZEO log files, looking for blocked transactions. New tool repozo.py (originally by Anthony Baxter) for performing incremental backups of Data.fs files. The fsrecover.py script now does a better job of recovering from errors the occur in the middle of a transaction record. Fixed several bugs that caused partial or total failures in earlier versions.