From jim@zope.com Tue Jul 2 19:25:55 2002 From: jim@zope.com (Jim Fulton) Date: Tue, 02 Jul 2002 14:25:55 -0400 Subject: [meta-sig] Proposal: persistence-sig Message-ID: <3D21F033.8000204@zope.com> Problem A number of projects are underway to provide persistence mechanisms for Python. These efforts have a number of common requirements, including: - Transparency Applications should not have to explicitly track object changes or save objects. Applications should not have to explicitly query most objects. Typically, some "root" objects will be explicitly retrieved and other objects will be retrieved through normal Python object traversal. Application objects should not contain storage code, such as SQL statements or file operations, needed to provide their persistence. Application code may need to contain code to generate persistence-related events, but even these should be automated to the degree possible, for example by monitoring attribute accesses. - Transactional storage Data should be stored transactionally with support for rolling back changes. Support for (optional) nested transactions should be anticipated. - Effective memory usage Applications should not use undue amounts of memory. Often, persistence applications use databases that are too large to fit into memory. Objects should be loaded only when necessary and should be removed from memory when they are no longer needed. Most of these efforts are focused on providing persistence using relational database data. The efforts are, for the most part, proceeding independently. Each will attempt to address the above requirements independently, with much duplication of effort. The Zope Object Database (ZODB) has satisfied the above requirements for some time. ZODB is currently undergoing a transition from ZODB 3, which was based on ExtensionClass, to ZODB 4, which is based on Python 2.2 new-style classes. As a part of this effort, the ZODB persistence and transaction frameworks are being factored out of ZODB into separate packages, with the hope that they will be of use to other persistence-based frameworks. It will be a huge duplication of effort if each of the various persistence projects has to address the above requirements independently. Worse, the resulting systems will have independent frameworks that are unlikely to interoperate. Objects built for one framework will need to be rewritten to work with another. Proposal A new persistence-SIG is proposed to explore and, if possible, produce persistence and transaction frameworks that can be used for a variety of persistence implementations, including relational database-based persistence and the ZODB. Coordinator: Jeremy Hylton Conclusion: When 1.0 versions of the frameworks are delivered, or September 1, 2003, whichever is sooner. Deliverables: PEPs documenting the frameworks created and software implementing key parts of the frameworks. Assuming that a satisfactory framework can be defined, then the framework and core implementations should be included in standard Python distributions. Scope The scope of this SIG includes common frameworks for: - Transaction coordination - Basic persistence management, in particular observing object changes (to know when an object needs to be saved) and access (to know when objects are used so that unused objects are removed from memory), and - Activation and caching, to move objects into memory when needed and out of memory when no longer needed. The scope does *not* include: - Concurrency control. This is the responsibility of specific data managers that are plugged into the frameworks. The transaction manager simply tracks object changes and coordinates the activities of data managers to commit (or rollback) changes in an atomic fashion. - Query languages. Individual data managers or applications may provide query facilities. While it would be cool to have a common query facility for Python, and I would support such a project, that would be a different project than this one. - Integrity constraints. Some data managers, such as those based on relational data, will need to provide facilities for low-level integrity checks, while others, such as ZODB, will not. Similarly, garbage collection is the responsibility of the data manager, not the framework. Application-level integrity systems would also be interesting, but would not depend on a persistence system. Jim -- -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From Nicolas.Chauvat@logilab.fr Wed Jul 3 11:35:42 2002 From: Nicolas.Chauvat@logilab.fr (Nicolas Chauvat) Date: Wed, 3 Jul 2002 12:35:42 +0200 (CEST) Subject: [meta-sig] persistence SIG Message-ID: I vote +1 for the persistence SIG and will be very happy to see the following people and projects cooperate : ZODB from Zope CAPS from AB Strakt ?? from inQual ?? from Roeland Rengelink POD from pypod.sf.net others I probably forgot... -- Nicolas Chauvat http://www.logilab.com - "Mais oł est donc Ornicar ?" - LOGILAB, Paris (France) From Sebastien.Bigaret@inqual.com Wed Jul 3 11:55:08 2002 From: Sebastien.Bigaret@inqual.com (Sebastien Bigaret) Date: 03 Jul 2002 12:55:08 +0200 Subject: [meta-sig] persistence SIG In-Reply-To: Nicolas Chauvat's message of "Wed, 3 Jul 2002 12:35:42 +0200 (CEST)" References: Message-ID: <87hejhrshf.fsf@bidibule.brest.inqual.bzh> Nicolas Chauvat writes: > I vote +1 for the persistence SIG So do I: +1 And since I'm quite closely related to that project ``?? from inQual'= ' I will be happy to cooperate! -- S=E9bastien Bigaret From hpk@trillke.net Wed Jul 3 12:30:47 2002 From: hpk@trillke.net (holger krekel) Date: Wed, 3 Jul 2002 13:30:47 +0200 Subject: [meta-sig] persistence SIG In-Reply-To: ; from Nicolas.Chauvat@logilab.fr on Wed, Jul 03, 2002 at 12:35:42PM +0200 References: Message-ID: <20020703133047.G10628@prim.han.de> [Nicolas Chauvat Wed, Jul 03, 2002 at 12:35:42PM +0200] > I vote +1 for the persistence SIG and will be very happy to see the > following people and projects cooperate : > > ZODB from Zope > CAPS from AB Strakt > ?? from inQual > ?? from Roeland Rengelink > POD from pypod.sf.net > others I probably forgot... I'd really like participants not to focus too much into bringing one particular implementation into the standard lib. It should be something like a common subset of required functionalities. Other than that i am definitely +1 holger From paul@zope.com Wed Jul 3 12:38:53 2002 From: paul@zope.com (Paul Everitt) Date: Wed, 03 Jul 2002 07:38:53 -0400 Subject: [meta-sig] Re: persistence SIG References: Message-ID: <3D22E24D.6070404@zope.com> Nicolas Chauvat wrote: > I vote +1 for the persistence SIG and will be very happy to see the > following people and projects cooperate : > > ZODB from Zope > CAPS from AB Strakt > ?? from inQual > ?? from Roeland Rengelink > POD from pypod.sf.net > others I probably forgot... MetaKit is a popular one: http://www.equi4.com/metakit/python.html --Paul From lac@strakt.com Wed Jul 3 12:59:28 2002 From: lac@strakt.com (Laura Creighton) Date: Wed, 03 Jul 2002 13:59:28 +0200 Subject: [meta-sig] Re: persistence SIG In-Reply-To: Message from Paul Everitt of "Wed, 03 Jul 2002 07:38:53 EDT." <3D22E24D.6070404@zope.com> References: <3D22E24D.6070404@zope.com> Message-ID: <200207031159.g63BxSLM007135@ratthing-b246.strakt.com> > Nicolas Chauvat wrote: > > I vote +1 for the persistence SIG and will be very happy to see the > > following people and projects cooperate : > > > > ZODB from Zope > > CAPS from AB Strakt > > ?? from inQual > > ?? from Roeland Rengelink > > POD from pypod.sf.net > > others I probably forgot... > > MetaKit is a popular one: > > http://www.equi4.com/metakit/python.html > > --Paul > AB Strakt is interested in persistent objects. Has anybody told A M Kuchling and the rest of the MEMS Exchange people as well as Irmen De Jong (author of Pyro) about a persistent object SIG? Laura Creighton From Nicolas.Chauvat@logilab.fr Wed Jul 3 12:50:11 2002 From: Nicolas.Chauvat@logilab.fr (Nicolas Chauvat) Date: Wed, 3 Jul 2002 13:50:11 +0200 (CEST) Subject: [meta-sig] persistence SIG In-Reply-To: <20020703133047.G10628@prim.han.de> Message-ID: > I'd really like participants not to focus too much into bringing one > particular implementation into the standard lib. It should be > something like a common subset of required functionalities. I think Jim Fulton did a good job at defining what were the components of a persistence framework and how beneficial it would be to have these components standardized separately. My personal opinion is that a "persistence framework" is not the kind of thing that you can sell to a client and what you can not sell to a client is to be considered "raw material" or "commodities" and will benefit everyone if it is stable and very low cost. Having it part of the standard library or part of an "official persistence extension module" (e.g. NumericPython or PIL) doesn't make much of a difference. Same goes for OS, web server, etc. When your clients want a website, they do not care whether it runs on Apache, Zope or something else. They want the web site to fit their needs, look pretty and be fast and secure. If providers can rely on Apache or Zope for security and speed, they'll compete on prettiness and fitting clients' needs using skills and in-house developed software: things that money can buy. -- Nicolas Chauvat http://www.logilab.com - "Mais oł est donc Ornicar ?" - LOGILAB, Paris (France) From smenard@bigfoot.com Wed Jul 3 14:30:54 2002 From: smenard@bigfoot.com (Steve Menard) Date: Wed, 03 Jul 2002 09:30:54 -0400 Subject: [meta-sig] persistence SIG In-Reply-To: <20020703133047.G10628@prim.han.de> References: Message-ID: <5.1.0.14.0.20020703092955.0352cea8@pop.videotron.ca> At 01:30 PM 7/3/2002 +0200, holger krekel wrote: >[Nicolas Chauvat Wed, Jul 03, 2002 at 12:35:42PM +0200] > > I vote +1 for the persistence SIG and will be very happy to see the > > following people and projects cooperate : > > > > ZODB from Zope > > CAPS from AB Strakt > > ?? from inQual > > ?? from Roeland Rengelink > > POD from pypod.sf.net > > others I probably forgot... > > >I'd really like participants not to focus too much >into bringing one particular implementation into >the standard lib. It should be something like a common >subset of required functionalities. > >Other than that i am definitely > >+1 > > holger I am not exactly sure what the persistence-SIG is, or exactly what this vote entails. I would definitely love to cooperate though. Steve From Nicolas.Chauvat@logilab.fr Wed Jul 3 14:42:42 2002 From: Nicolas.Chauvat@logilab.fr (Nicolas Chauvat) Date: Wed, 3 Jul 2002 15:42:42 +0200 (CEST) Subject: [meta-sig] Re: persistence SIG In-Reply-To: Message-ID: On Wed, 3 Jul 2002, Nicolas Chauvat wrote: > I vote +1 for the persistence SIG and will be very happy to see the > following people and projects cooperate : > > ZODB from Zope > CAPS from AB Strakt > ?? from inQual > ?? from Roeland Rengelink > POD from pypod.sf.net > others I probably forgot... I think I forgot Mike Olson from Fourthought, that implemented 4ODF. -- Nicolas Chauvat http://www.logilab.com - "Mais oł est donc Ornicar ?" - LOGILAB, Paris (France) From tim.one@comcast.net Thu Jul 4 07:21:07 2002 From: tim.one@comcast.net (Tim Peters) Date: Thu, 04 Jul 2002 02:21:07 -0400 Subject: [meta-sig] persistence SIG In-Reply-To: <5.1.0.14.0.20020703092955.0352cea8@pop.videotron.ca> Message-ID: [Steve Menard] > I am not exactly sure what the persistence-SIG is, or exactly what this > vote entails. I would definitely love to cooperate though. That's fine: we never know *exactly* what a SIG will do before it starts up. The point of asking is to see whether there's enough interest to *get* it started. The response to this has been unusually high. If it follows the pattern of other high-profile SIGs, it will have a vigorous life until someone actually produces code. Then everyone will run away frightened <0.9 wink>. the-types-sig-went-thru-that-at-least-twice-ly y'rs - tim From sdrees@ieee.org Thu Jul 4 10:01:49 2002 From: sdrees@ieee.org (Stefan Drees) Date: Thu, 4 Jul 2002 11:01:49 +0200 Subject: [meta-sig] persistence SIG In-Reply-To: ; from tim.one@comcast.net on Thu, Jul 04, 2002 at 02:21:07AM -0400 References: <5.1.0.14.0.20020703092955.0352cea8@pop.videotron.ca> Message-ID: <20020704110149.A6061@sdrees2.de> On Thu, Jul 04, 2002 at 02:21:07AM -0400 - a wonderful day - Tim Peters wrote: > ... it will have a vigorous life until someone actually > produces code. Then everyone will run away frightened ... Let's see. You can count me in. Here is my +1 for the persistence-SIG. All the best, s t e f a n. -- Stefan Drees, sdrees@acm.org, www.sdrees.de. From smenard@bigfoot.com Thu Jul 4 14:27:28 2002 From: smenard@bigfoot.com (Steve Menard) Date: Thu, 04 Jul 2002 09:27:28 -0400 Subject: [meta-sig] persistence SIG In-Reply-To: References: <5.1.0.14.0.20020703092955.0352cea8@pop.videotron.ca> Message-ID: <5.1.0.14.0.20020704092616.03365510@pop.videotron.ca> At 02:21 AM 7/4/2002 -0400, Tim Peters wrote: >[Steve Menard] > > I am not exactly sure what the persistence-SIG is, or exactly what this > > vote entails. I would definitely love to cooperate though. > >That's fine: we never know *exactly* what a SIG will do before it starts >up. The point of asking is to see whether there's enough interest to *get* >it started. The response to this has been unusually high. > >If it follows the pattern of other high-profile SIGs, it will have a >vigorous life until someone actually produces code. Then everyone will run >away frightened <0.9 wink>. > >the-types-sig-went-thru-that-at-least-twice-ly y'rs - tim LOL. Well, it certainly has MY support. And I don't believe people run away frightened, I think they're just too happy to have working code and are a busy hiding somewhere using it :) Steve From rengelin@strw.leidenuniv.nl Fri Jul 5 12:55:39 2002 From: rengelin@strw.leidenuniv.nl (Roeland Rengelink) Date: Fri, 5 Jul 2002 13:55:39 +0200 (MEST) Subject: [meta-sig] persistence SIG Message-ID: <200207051155.g65Btdt05307@sloten.strw.leidenuniv.nl> Nicolas Chauvat Wed, Jul 03, 2002 at 12:35:42PM +0200] > I vote +1 for the persistence SIG and will be very happy to see the > following people and projects cooperate : > > ZODB from Zope > CAPS from AB Strakt > ?? from inQual > ?? from Roeland Rengelink > POD from pypod.sf.net > others I probably forgot... I'm indeed very interested in cooperating, mark me for one +1. The ?? in my case should read astro-wise (www.astro-wise.org). The project we're working on includes a prototype for persistence on top of a relational db. See http://www.strw.leidenuniv/~rengelin/persistence.pdf for a specification of that interface (source code not available yet). Cheers, Roeland Rengelink -- Sterrewacht Leiden, Leiden University, P.O. Box 9513, NielsBohrweg 2, 2300 RA Leiden, The Netherlands, Tel (+31) 715275595 (work), 206393300 (home), 715275819 (fax) Email: rengelin@strw.leidenuniv.nl From barry@zope.com Mon Jul 8 21:03:58 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Jul 2002 16:03:58 -0400 Subject: [meta-sig] New Persistence SIG created Message-ID: <15657.61486.222685.665859@anthem.wooz.org> As recently discussed on meta-sig@python.org, we have created a new SIG focussed on producing a common persistence and transactional framework for Python programs. This SIG is called persistence-sig@python.org. For more information on the SIG, its mission, and deadlines see http://www.python.org/sigs/persistence-sig/ To join the mailing list see http://mail.python.org/mailman-21/listinfo/persistence-sig -Barry From barry@zope.com Mon Jul 8 21:14:33 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 8 Jul 2002 16:14:33 -0400 Subject: [meta-sig] SIG charters Message-ID: <15657.62121.520364.556758@anthem.wooz.org> Many of the sig deadlines have expired. See www.python.org/sigs Of these, the following appear to have no (or not much) meaningful recent activity: do-sig import-sig types-sig I propose to retire these sigs and extend all others for another year. Objections? -Barry From guido@python.org Tue Jul 9 21:50:59 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 09 Jul 2002 16:50:59 -0400 Subject: [meta-sig] SIG charters In-Reply-To: Your message of "Mon, 08 Jul 2002 16:14:33 EDT." <15657.62121.520364.556758@anthem.wooz.org> References: <15657.62121.520364.556758@anthem.wooz.org> Message-ID: <200207092050.g69KoxW04101@odiug.zope.com> > Many of the sig deadlines have expired. See www.python.org/sigs > > Of these, the following appear to have no (or not much) meaningful > recent activity: > > do-sig > import-sig > types-sig > > I propose to retire these sigs and extend all others for another year. +1 --Guido van Rossum (home page: http://www.python.org/~guido/)