-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote:

...

Also, the reason for removal may yet disappear if jcrea steps in an continues to make updates.

OK, but none of his changes have received a code review, so if we are going to go down the whole "disciplined" route about it being an rc then we should back out all of Jesus' changes for both 2.6 and 3.0, which puts us back to the same instability issues.

I was wondering if somebody could write a "TO DO" list of things need to keep bsddb in the stdlib. Then I could work to trying to comply :). Yes, we are all very busy guys, but still... - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSL/kPZlgi5GaxT1NAQLu4AP/VSHPYOCQgQYFJsdi2MWXBpyY7TyC5XgT Ks2uilXru/hsKQcegn8G6z/53Bt0Uu+oXZSQaZ51V8VnwDXEqaZ+GnKK+S2ky9m0 yomgMlKIZZJsOVd6X4HbLtrVYVKX8wQ224X/yCkw27OLzLIE9IDbUzEjC3+/A7mD 9IJu3B6IaLA= =ZA8p -----END PGP SIGNATURE-----

On Thu, Sep 4, 2008 at 11:30 AM, Brett Cannon wrote:

Another thing to keep in mind with the whole shelve/dbm.any argument is that for 3.1 there is nothing saying we can't change shelve and the dbm package to allow 3rd-party code to register with the dbm package such that bsddb can be used as needed behind the scenes.

-Brett

Exactly. That is what I think should really happen here.

>> Related but tangential question that we were discussing on the >> pygr[0] mailing list -- what is the "official" word on a scalable >> object store in Python? We've been using bsddb, but is there an >> alternative? And what if bsddb is removed? Brett> Beyond shelve there are no official plans to add a specific Brett> object store. Unless something has changed while I wasn't looking, shelve requires a concrete module under the covers: bsddb, gdbm, ndbm, dumbdbm. It's just a thin layer over one of them that makes it appear as if you can have keys which aren't strings. Skip

On Thu, Sep 04, 2008 at 10:29:10AM -0400, Tony Nelson wrote: -> At 6:10 AM -0500 9/4/08, skip@pobox.com wrote: -> > >> Related but tangential question that we were discussing on the -> > >> pygr[0] mailing list -- what is the "official" word on a scalable -> > >> object store in Python? We've been using bsddb, but is there an -> > >> alternative? And what if bsddb is removed? -> > -> > Brett> Beyond shelve there are no official plans to add a specific -> > Brett> object store. -> > -> >Unless something has changed while I wasn't looking, shelve requires a -> >concrete module under the covers: bsddb, gdbm, ndbm, dumbdbm. It's just a -> >thin layer over one of them that makes it appear as if you can have keys -> >which aren't strings. -> -> I thought that all that was happening was that BSDDB was becoming a -> separate project. If one needs BSDDB with Python2.6, one installs it. -> Aren't there other parts of Python that require external modules, such as -> Tk? Using Tk requires installing it. Such things are normally packaged by -> each distro the same way as Python is packaged ("yum install tk bsddb"). -> -> Shipping an application to end users is a different problem. Such packages -> should include a private copy of Python as well as of any dependent -> libraries, as tested. Why? On Mac OS X, for example, Python comes pre-installed -- not sure if it comes with Tk yet, but the next version probably will. On Windows there's a handy few-click installer that installs Tk. Is there some reason why I shouldn't be relying on those distributions?? Requiring users to install anything at all imposes a barrier to use. That barrier rises steeply in height the more packages (with versioning issues, etc.) are needed. This also increases the tech support burden dramatically. I'm happy to be told that bsddb is too much of a maintenance burden for Python 2.6/3.0 to have -- especially since it's gone from 3.0 now ;) -- but I don't think the arguments that *it won't matter that it's not there* have been very credible. There's a BIG difference between things that come with Python and things that are add-ons. Right now, I'm teaching an intro programming course using Python. It doesn't seem like the students are going to need to install *anything* other than base Python in order to play with full networking libraries & sqlite databases, among other features. And, for me and for them, that's really great. I don't think the convenience of "batteries *included*" should be underestimated. --t -- C. Titus Brown, ctb@msu.edu

On Thu, Sep 04, 2008 at 11:01:35AM -0400, Tony Nelson wrote: -> At 7:37 AM -0700 9/4/08, C. Titus Brown wrote: -> >On Thu, Sep 04, 2008 at 10:29:10AM -0400, Tony Nelson wrote: -> ... -> >-> Shipping an application to end users is a different problem. Such packages -> >-> should include a private copy of Python as well as of any dependent -> >-> libraries, as tested. -> > -> >Why? On Mac OS X, for example, Python comes pre-installed -- not sure -> >if it comes with Tk yet, but the next version probably will. On Windows -> >there's a handy few-click installer that installs Tk. Is there some -> >reason why I shouldn't be relying on those distributions?? -> -> Yes. An application is tested with one version of Python and one version -> of its libraries. When MOSX updates Python or some other library, you are -> relying on their testing of your application. Unless you are Adobe or -> similarly large they didn't do that testing. Perhaps you have noticed the -> threads about installing a new Python release over the Python that came -> with an OS, and how bad an idea that is? This is the same issue, from the -> other side. I have to say I've never had problems with a stock install of Python on either Mac OS X or Windows (shockingly enough :). I think this is good advice for applications that rely on external libraries, but I just don't see any problems with relying on Python 2.5 to contain all the things that normally come with Python 2.5. It seems like you're pushing a pretty sharp dichotomy (trichotomy?) -- - Python library/core developers should compile it all. - Python app developers can rely on what they install from binaries themselves, but not rely on it to be present on anyone else's machine or OS. - End users should be given a complete clean install of Python in a different location for each application they're using, even if those applications depend only on the stdlib. This seems surprisingly complicated to me (and unnecessary, in my limited experience) -- but it does validate my decade-old decision to avoid writing end-user applications in Python, sadly enough. It ends up being less work to distribute and support a C/C++ app on Windows and Mac OS X, for crikey's sake! --t -- C. Titus Brown, ctb@msu.edu

On Sep 4, 2008, at 8:10 AM, C. Titus Brown wrote:

I have to say I've never had problems with a stock install of Python on either Mac OS X or Windows (shockingly enough :). I think this is good advice for applications that rely on external libraries, but I just don't see any problems with relying on Python 2.5 to contain all the things that normally come with Python 2.5.

...

...

import bsddb Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/

There can be subtle differences between a "stock" Python and the system Python on Mac OS X 10.5. For example, Mac OS X compiles against EditLine instead of GNU Readline. From "man python" on Mac OS X: """ The Python inteterpreter supports editing of the current input line and history substitution, similar to facilities found in the Korn shell and the GNU Bash shell. However, rather than being implemented using the GNU Readline library, this Python interpreter uses the BSD EditLine library editline(3) with a GNU Readline emulation layer. ... For example, the rlcompleter module, which defines a completion function for the readline modules, works correctly with the EditLine libraries, but needs to be initialized somewhat differently: ... """ Fairly rare that you'd trip over this minor difference though - EditLine is more a problem on Mac OS X when trying to compile your own Python, since you need to install and link against GNU Readline. However, all does not seem to be right with the bsddb module on the system Python 2.5 on Mac OS X 10.5: $ /usr/bin/python Python 2.5.1 (r251:54863, Jan 17 2008, 19:35:17) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. python2.5/bsddb/__init__.py", line 51, in <module> import _bsddb ImportError: No module named _bsddb

...

...

Kevin Teague wrote:

There can be subtle differences between a "stock" Python and the system Python on Mac OS X 10.5.

Also there can be different versions of Python installed in different versions of MacOSX. So if you distribute an app that relies on the system Python, at the least you have to test it against all the Python versions it's likely to encounter, and possibly provide different versions of the app for different versions of MacOSX. Another thing to keep in mind is that if you use something like py2app or py2exe, it doesn't include the whole Python distribution, just the stdlib modules the app actually uses. So it's not as bad as including a whole Python installation for every app. -- Greg

I don't think the convenience of "batteries *included*" should be underestimated.

Yeah, but bsddb is one of those exploding batteries. I've used it for years, and have had lots and lots of problems with it. Having SQLite in there is great; now we need implementations of anydbm and shelve which use it. Bill

I thought that all that was happening was that BSDDB was becoming a separate project. If one needs BSDDB with Python2.6, one installs it.

No, not in the way you mean it.

Aren't there other parts of Python that require external modules, such as Tk?

It's different. BSDDB (the Sleepycat-then-Oracle implementation) never was part of Python; this hasn't changed. What *has* changed is that the bsddb module (i.e. the Python wrapper) is now not part of Python anymore, either, due to it being maintained separately. This is as if Tkinter was removed from Python. Regards, Martin

On Thu, Sep 04, 2008 at 03:23:22PM +0200, Jesus Cea wrote: -> -----BEGIN PGP SIGNED MESSAGE----- -> Hash: SHA1 -> -> Brett Cannon wrote: -> >> Related but tangential question that we were discussing on the pygr[0] -> >> mailing list -- what is the "official" word on a scalable object store -> >> in Python? We've been using bsddb, but is there an alternative? And -> >> what if bsddb is removed? -> > -> > Beyond shelve there are no official plans to add a specific object store. -> -> If you are storing million of objects, you'd better use a transactional -> storage, able to survive diskfulls or code/computer crashes. We're using a write-once-read-many pattern of access, and it is simply a cache of a separate file (that remains around), so no, we don't better use a transactional storage :). -> I will maintain "bsddb" as a separate (downloadable via PYPI) package -> whatever the fate of bsddb in Python stardard library be. So bsddb is a -> pretty safe bet, even if you need to install it separately. Since I/we want to distribute pygr to end-users, this is really not a pleasant prospect. Also often the installation of Python itself goes much more smoothly than the installation of separately compiled binary packages, for all the obvious reasons (compiler/OS versions, lib versions, etc. etc.) -> Compared to sqlite, you don't need to know SQL, you can finetuning (for -> example, using ACI instead of ACID, deciding store by store), and you -> can do replication and distributed transactions (useful, for example, if -> your storage is bigger than a single machine capacity, like my case). If -> you combine Berkeley DB with Durus, for example, all of this is -> abstracted and you simply use "regular" python objects. I agree. I like bsddb for just this reason and I'd like to continue being able to use it! I think that there are many reasons why having such a thing in the stdlib is really useful and I also think it's worth exploring the ramifications of taking it out... --t -- C. Titus Brown, ctb@msu.edu

On Thu, Sep 04, 2008 at 07:01:47PM +0200, Jesus Cea wrote: -> -----BEGIN PGP SIGNED MESSAGE----- -> Hash: SHA1 -> -> C. Titus Brown wrote: -> > Since I/we want to distribute pygr to end-users, this is really not a -> > pleasant prospect. Also often the installation of Python itself goes -> > much more smoothly than the installation of separately compiled binary -> > packages, for all the obvious reasons (compiler/OS versions, lib -> > versions, etc. etc.) -> -> I agree. I can check the library with Solaris 10 and several flavors of -> Linux, but I'm particularly worried about Windows support. I'm unable to -> provide precompiled libs, and 99.999% of windows users don't know what a -> "compiler thing" is, let alone being able to compile anything by themselves. I believe I might be able to help you with this. More off-list, in a few weeks; if anyone else needs full Windoze access, Watch This Space, as they say. (Yes, I know access is not enough -- you really want someone to be paying attention on Windows, too. I'm working on a project or two there; access to large quantities of talented students is opening up some ideas :) --titus -- C. Titus Brown, ctb@msu.edu

Doesn't SQLlite still have a 4gb cap? I'd personally prefer an open source solution (if that's Berkeley so be it but there's plenty out there... MySQL for one)

On Thu, Sep 04, 2008 at 01:07:23PM -0400, Jeff Hall wrote:

Doesn't SQLlite still have a 4gb cap?

http://sqlite.org/limits.html Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

me> I suggested in another message (perhaps on another thread) that me> maybe a dbm.sqlite module would be worth having. http://bugs.python.org/issue3783 Skip

-On [20080904 16:22], C. Titus Brown (ctb@msu.edu) wrote:

I agree. I like bsddb for just this reason and I'd like to continue being able to use it! I think that there are many reasons why having such a thing in the stdlib is really useful and I also think it's worth exploring the ramifications of taking it out...

And having to install bsddb from an external source disables your use, how? -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Infinite Dreams, I can't deny them, Infinity is hard to comprehend...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Oleg Broytmann wrote:

-- SQLite is public domain; the licensing terms of Berkeley DB[1] are not friendly to commercial applications: "Our open source license ... permits use of Berkeley DB in open source projects or in applications that are not distributed to third parties." I am not sure if using of PyBSDDB in commercial applications is considered "using of Berkeley DB in open source projects";

I can't comment on this. I'm not a lawyer.

-- SQLite has a pretty stable API and a pretty stable on-disk format; for bsddb one needs to do dump/reload on every major release;

Not at all. The worst thing you would need to do is a "db_upgrade", an in-place operation. Lately it is pretty harmless and fast (like upgrading the log format, not the database file format). A stable fileformat is useful for long term support, but an evolving format allows improvements. Following your reasoning, Python should be keep in 1.0 era, for compatibility sake.

-- SQLite implements a subset of SQL - a powerful query language;

Yes, a declarative language completely unrelated to Python.

-- SQLite is extensible - one can write his/her own functions and aggregates, e.g.; PySQLite allows to write these functions in Python; PySQLite also allows to write data conversion functions that converts between Python and SQL data types;

bsddb 4.7.4 (available next month) will allow to subclass DB/DBEnv, etc. objects, so you can implement the logic you wish there. Until that, you can do proxy/delegation (that is the way I'm doing 3.0 compatibility, BTW).

-- a program can attach a few databases at once thus distributing loads between a number of disks, including network mounts.

That is an OS issue. Any program get the benefice. The problem is not disk capacity. Any modern machine can scale disk without bound via iSCSI, for example (god bless ZFS!). The issue is replication for redundancy, load sharing and high availability. These things are available in bsddb 4.7.3 (that is, in Python 2.6). How do you scale traffic demand in SQLite?. I can keep adding machines to solve read requests, without sharing any disk between them. I can launch 64 bsddb processes in a single 64 CPU machine to manage (read/write) a single shared database. I don't know if SQLite can do that. Berkeley DB can.

Durus (and ZODB) has an index of all objects, the index is stored in memory AFAIK - a real problem if one has millions of objects. Does bsddb help to mitigate the problem?

Latest Durus has not that issue, but you always can use another project of mine: Berkeley DB Backend Storage Engine for DURUS http://www.jcea.es/programacion/durus-berkeleydbstorage.htm This code (with some private enhancements for replication and distributed transactions) manages a nearly 200 Terabytes Durus repository without any sweat (~2^35 objects stored there). In this particular instance, distributed transactions allows me to partition data between several machines, with no sharing between them, and replication allows redundancy and routing read requests to a less loaded machine (the writes goes to the "master" machine, replication from there is transparent). SQLite is a good product. I just dislike "SQL fits all" model, as I already said in another message. Using a SQL storage to save persistent Python objects is ugly and SQL language is of no use there. You just need "something" safe, scalable, configurable and being able to give you opaque objects when an OID (Object ID) is presented. Compare a terabyte Python "shelve" object, ease of use, transparence, etc., with keeping the objects in a SQL database server. Just my opinion, of course. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSMAdiJlgi5GaxT1NAQJxkwP/emM8dDKbnhxme76Nm3bXhA89NwCgQNQi ojO0wkVZZ8ypUBNKwGM8PyDzGYoWGnh7VgylGb2bsPt67bCxrHjcXBNPaYrMN/fw AETLlJUrhu9J17jPWKA+JU1FmC9oX34Ki580qMXI9nR51LVLU/1H6nM+KgA0slnn uG3xvm5chfk= =M46E -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Oleg Broytmann wrote:

Once I upgraded Python on a Windows computer... I think it was 2.2 to 2.3 upgrade - and all my bsddb databases stopped working. I cannot call this "improvement". I didn't have db_upgarde on that computer (or I didn't know about it). Does PyBSDDB have db_upgrade in the distribution? Does Python distribution have db_upgrade?

I can't comment about bsddb status before my maintenance era (since March or so). But current release: http://www.jcea.es/programacion/pybsddb_doc/db.html#upgrade """ [jcea@tesalia Modules]$ grep -i upgrade * _bsddb.c:DB_upgrade(DBObject* self, PyObject* args) _bsddb.c: if (!PyArg_ParseTuple(args,"s|i:upgrade", &filename, &flags)) _bsddb.c: err = self->db->upgrade(self->db, filename, flags); _bsddb.c: MAKE_ENTRY(nupgrade); _bsddb.c: {"upgrade", (PyCFunction)DB_upgrade, METH_VARARGS}, _bsddb.c: ADD_INT(d, DB_UPGRADE); _bsddb.c: ADD_INT(d, DB_LOCK_UPGRADE); _bsddb.c: ADD_INT(d, DB_LOCK_UPGRADE_WRITE); _bsddb.c: ADD_INT(d, DB_LOCK_UPGRADE); """

...

Following your reasoning, Python should be keep in 1.0 era, for compatibility sake.

Python? No. But persistent data structures? Yes! How many different pickle data formats there were since Python 1.0? What is the oldest pickle format modern Python can read? (Just using pickle as an example.)

Modern bsddb can read relic bsddb data; just do a "db_upgrade".

...

...

-- SQLite implements a subset of SQL - a powerful query language; Yes, a declarative language completely unrelated to Python.

Sometimes being unrelated to Python is advantage.

No argument here. My brain just work that way :). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSMAwLJlgi5GaxT1NAQIKzgP5AeHeGF52DtGs/KMssduQczPnoH5ndgME /265foN/qp/GM4kgunoOTPGd9kREVmxgduVaY9yNvVkQNH0WW+t+y41CIcwL36lG EWXb+9eeAkBm7C0fFLwYZnqDva9/n9Ax7SkXHl+SOerL9Eq6rXzFXyHcTfyZtu8i uI4q4n7nHQA= =upd8 -----END PGP SIGNATURE-----

On Sun, Sep 07, 2008 at 11:34:37AM -0700, Gregory P. Smith wrote:

You could probably have built the bsddb185 module and loaded your data from that and rewritten it using the new bsddb module.

I built bsddb185, loaded old data, exported it to... I don't remember now, but I clearly remember I stopped using bsddb.

The lesson for python: when that happens lets write the code to make the transition between formats trivial.

For me the lesson is different - do not include modules in the stdlib that relies on unstable 3rd party libraries. I consider bsddb unstable. sqlite is more stable, but PySQLite... there are many minor releases between Python releases; my humble opinion is it'd be better to have one external PySQLite module than two (PySQLite and sqlite3). Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

On Sun, Sep 7, 2008 at 3:43 PM, Steve Holden wrote:

Oleg Broytmann wrote:

...

On Sun, Sep 07, 2008 at 11:34:37AM -0700, Gregory P. Smith wrote:

...

You could probably have built the bsddb185 module and loaded your data from that and rewritten it using the new bsddb module.

I built bsddb185, loaded old data, exported it to... I don't remember now, but I clearly remember I stopped using bsddb.

...

The lesson for python: when that happens lets write the code to make the transition between formats trivial.

For me the lesson is different - do not include modules in the stdlib that relies on unstable 3rd party libraries. I consider bsddb unstable. sqlite is more stable, but PySQLite... there are many minor releases between Python releases; my humble opinion is it'd be better to have one external PySQLite module than two (PySQLite and sqlite3).

Unfortunately this advice should have been taken several years ago. The fact is that there are almost certainly Python users who rely on the presence of the bsddb module for production work, and simply removing it without deprecation is bound to upset those users.

Those users would first have to port their code to Python 3.0. That task is a lot larger than dealing with a separate download of bsddb. It is not being removed from 2.6.

I'm particularly concerned that it appears that normal procedures have been circumvented to enable its removal from 3.0. Since we have at least one developer committed to ongoing support that seems both harsh and unnecessary.

3.0 breaks a lot of things. Most of the library reorg may have been discussed more than this particular removal, but that doesn't mean that changes won't come as a surprise for most users. In this case, a completely compatible module is available as a 3rd party download. That's a lot less sever than the complete abandonment than the fat of many other modules. It's just a matter of source code packaging. Vendors can completely remove the difference in their packaging of the binaries. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, Sep 4, 2008 at 7:33 AM, Oleg Broytmann wrote:

SQLite is public domain; the licensing terms of Berkeley DB[1] are not friendly to commercial applications: "Our open source license ... permits use of Berkeley DB in open source projects or in applications that are not distributed to third parties." I am not sure if using of PyBSDDB in commercial applications is considered "using of Berkeley DB in open source projects";

Wow, I hadn't realized that it was such a restrictive license. When I see "Berkeley" I think "BSD license". -- Curt Hagenlocher curt@hagenlocher.org

On Thu, Sep 4, 2008 at 7:33 AM, Oleg Broytmann wrote:

On Thu, Sep 04, 2008 at 03:23:22PM +0200, Jesus Cea wrote:

...

Compared to sqlite, you don't need to know SQL, you can finetuning (for example, using ACI instead of ACID, deciding store by store), and you can do replication and distributed transactions (useful, for example, if your storage is bigger than a single machine capacity, like my case).

Let me raise the glove. Compared to bsddb:

-- SQLite is public domain; the licensing terms of Berkeley DB[1] are not friendly to commercial applications: "Our open source license ... permits use of Berkeley DB in open source projects or in applications that are not distributed to third parties." I am not sure if using of PyBSDDB in commercial applications is considered "using of Berkeley DB in open source projects";

FWIW, many years ago in the past when I asked sleepycat about this (long before oracle bought them) they said that python was considered to be the application. Using berkeleydb via python for a commercial application did not require a berkeleydb license. But my legal advice is worth as much as the paper its printed on. Always ask your own lawyer and oracle about such things. -gps

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michele Simionato wrote:

I do not use bsddb directly, but I use shelve which on Linux usually takes advantage of bsddb. Does removing bsddb mean that I will not be able to read shelve files written with Python 2.5 with Python 3.0? That would be quite disturbing to me.

Seems so. If bsddb is actually unavailable in Python 3.0, you would need to download/install it from PYPI. I'm committed to keep bsddb alive, in a way or another :). I'm thinking that if bsddb is discarded in Python 3.0, shelve probably drop it also, so installing bsddb externally will not "magically" make it available to 3.0 shelve. I can't comment about the python-dev plans here. PS: Remember that if you are installing bsddb as a separate package, its name will be "bsddb3", not "bsddb". - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSL/jJZlgi5GaxT1NAQIzJAP8CiIxpoz553NDr+/1GkAfzs3W6fu1uBuo XsCmbqkiOMe9fPOBNZnlfoBnGz4C4nlOlOzQV3RexRXBiWKOqUUg4DlJOJtrAMUO ZNtcz9JyvjzmVNZMezgCsfjkEhzNABbCe9mXHQVCR5zlNZVUKpTW7A06/1eX/gMv ECJqQta662o= =6XBZ -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Coghlan wrote:

While that will still be visible to some degree due to the presence of the 2.x version of the bsddb code in Python 2.6, I don't think it will be quite the same as it would have been with the 3.x version also being readily available as part of the standard 3.0 install.

Since 2.6 intention seems to mark this module as deprecated, I guess 2.x bsddb presence in stock python will finish in 2.7. Moreover, I'm working just now improving 2.x/3.x conversion code in pybsddb. I think this code will be available in bsddb 4.7.4, and it will not be integrated in Python 2.6 (that will include 4.7.3.minor releases, if we keep the criteria of "only stability and security fixes in 2.6.x"). If the idea is to keep bsddb alive in 2.x, I don't see the point of not keeping the 3.0 version, because the issues used to justify the removal persist: I'm the only maintainer, little code review, buildbot issues, etc. (I would like a comprehensive list, to be able to improve those deficiencies). In fact, if we keep bsddb in 2.x, the pressure to keep it in 3.x will be higher.

Regardless, given that the removal of bsddb from the 3.0 branch is now a done deal in svn, I suggest that all we can do is monitor how much

Any version control system can revert that with a single command :). All I can say is that current bsddb code (in my personal repository) passes all tests in current compiled python3.0 binary, called with the "-bb" parameter flag (the "-bb" flag was something I learned yesterday).

but in the meantime there *are* real benefits to decoupling the maintenance cycles (those wanting to get the most out of Jesus's ongoing work in exposing more of the bsddb API are probably going to want to be using the external pybsddb releases anyway rather than waiting however many months it ends up being until we get to 2.7/3.1).

The cycles are actually decoupled since I toke over the bsddb maintenance (I've released a new version per month). So the release cycles are not an issue. The main issue here is 3.0 support, that I worked over the last couple of months. It is done now. It couldn't be done faster because I was learning 3.0 internals on-the-fly (there are NO docs about C module migration; my experience there could be valuable) and 3.0 was a moving target (still is). For example, when I left to summer holiday bsddb worked flawless in Python 3.0b2. It failed in 3.0b3 because threading renames done in python 3.0. So, Python 3.0 is not waiting for bsddb to be ready, because it already is (since yesterday). And future Python releases won't suffer because we won't have any other major architectural reengineering of Python in a long long time (I hope!). That is, future Python releases would take whatever bsddb is available at that time. No wait. No dependent release cycles. With my current release schema of "a release per month", I can track python evolution with little effort. For example, Python 2.5 to 2.6 was pretty painless, even with the "PyBytes_*" ugliness.

As far as the corporate scenarios go: if a corporate environment is *so* screwed up as to simultaneously permit a migration from Python 2.x to 3.0 of an internal project that relies on bsddb, while at the same time banning those performing the migration from installing the 3.0 compatible version of pybsddb, then they have much bigger problems than which modules and packages we are choosing to include in the standard library.

Agreed. I was thinking about bsddb removal in 2.7. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSMAEnplgi5GaxT1NAQIrKgP/YAp45HUSG8Q+M355LTVqlcLMLkycpooc fflW0MlQ3zZV307VBUbGo9urkS6h1pYhYByivApylhVqj8D4x8OEmMZk0lX8cegG LYSBzs/sBeyxWWva6r5D9/4DsgJe9ZHqaBLMpy6ipPNVtUbMS61VTNovb3wP+f72 EnSIf9k/glM= =QxRo -----END PGP SIGNATURE-----