[DB-SIG] WHat's the status of DB modules and datetime.py supp ort?

[M.-A. Lemburg]

Guido van Rossum wrote:
>>Since I've always tried to keep up the spirit of the DB API
>>spec ever since I was editor of the 2.0 version, you probably
>>feel that I'm protective of it.
>>
>>It is true that I've invested
>>a considerable amount of time into discussing the various
>>issues raised with 1.0 on this list and after 2.0 was released
>>amended it various times (without changing the version number)
>>in fully backwards compatible ways (e.g. by adding standardized
>>extensions that we discussed on this list some time ago).
>>
>>However, that is *not* the reason of why I think making the
>>DB API more restrictive is a bad thing. The spirit of the DB
>>API spec has always been to give developers a guideline in
>>creating new database modules while not making the implementation
>>too burdensome. As a result we have quite a few database modules
>>out there which implement the spec, including some of the extensions.
>>They also often come with a multitude of new developments
>>that are permitted by the spec (e.g. customized row objects
>>that behave like a sequence).
>>
>>By making the spec more restrictive we would not only make
>>life of the developers harder, but also break backwards
>>compatibility of most database modules that work perfectly
>>OK today.
> 
> For Python itself, making the life of the developers harder is
> usually considered okay if it makes the life of the users easier.

Not so in the DB-SIG world :-)

We believe that it is more important to have a multitude of
database modules for various different databases than to have
just a few for database backends that provide enough features
to actually implement all the requirements.

We also think that supporting multiple Python versions is
a good thing and that more specific design decisions (e.g.
the mapping between database and Python data types) should
be left to the database module implementor.

And last not least, it has always been the aim of the
DB API to provide enough freedom of choice for developers
to add functionality that complies to the standard, but
extends it in various different useful ways.

> I'm curious if the life of the users isn't made *considerably* harder
> by some gratuitous differences between DB-API implementations, when
> a user needs to switch database vendors or (a slightly different
> case!) needs to support multiple vendors simultaneously.

That hard part in supporting multiple backends lies in
the differences at the SQL level (different syntax, data
types, restrictions, etc.), not the database interfacing
level.

> Yes, you can write all your code using the facilities to inquire
> DB-API features, but in practice nobody does that until they are
> forced to consider multiple versions and realize just how different
> they are.  In a few months on this list I've seen many folks post
> about this.
> 
> (Of course, you can argue that the database vendors are a major source
> of the differences -- the differences in SQL syntax are indeed
> staggering!  But I'm not sure if that gives DB-API developers an
> excuse -- otherwise we might as well have no standard at all.)

Hmm, that contradicts the success of the DB API specifications
1.0 and 2.0 which have enabled Python developers to write
good working code for more than 7 years.

>>My aim has always been to see what database developers
>>come up with and then standardize those new ideas in a
>>way that makes life easier for the user. The database
>>extensions are a good example of this practice and I
>>have strong belief that this method of evolving the
>>standard has proven to be much more of a success than
>>trying to break compatibility every year or so by trying
>>to stay bleeding edge all the time.
>>
>>Many of the things people have asked about in the past
>>are really not intended to be implemented at the level
>>the DB API is working at. Object relational wrappers,
>>ADO style access, etc. can easily be programmed on top
>>of the DB API and indeed have been implemented quite
>>successfully for a number of database modules out there.
> 
> But the creation of such wrappers in a vendor-agnostic way is harder
> because of the wide variation at the DB-API level.

Have you tried any of them ? Adding support for new
DB-API compatible drivers is usually just a matter
of overriding a few driver specific access or connect
methods.

>>If someone feels we need a standard for these wrappers,
>>I'd suggest to first look at the question whether there
>>will ever be more than just one or two such wrappers
>>written. I've been in the database business for quite
>>some time now and to be honest, every single application
>>has had slightly different requirements for the database
>>abstraction layer - no single API would have captured
>>all cases. Of course, that's just my personal opinion.
>>If people like the existing OR-wrappers, they should
>>simply use them.
> 
> I agree, in practice the needed higher-level abstractions are
> incredibly application-specific, and generic abstraction modules are
> of limited value to "real" applications.
> 
> But this argues for *more* standardization between DB-API modules:
> everyone who develops a serious application that eventually gets to
> curse the gratuitous differences -- between vendor SQL as well as
> between DB-API modules.
> 
> We can't fix the vendor's SQL, but we *can* try to agree on more
> similarity between DB-API modules. 

Can you give some examples of what you find so incredibly
disturbing about the current set of database modules ? Where
are the differences that you are talking about and what
makes you think that making a specification less flexible
would result in better database modules ?

> It's clear to me now that this
> would have to be DB-API 3.0, and that it will take some years before a
> 3.0-compliant module is available for most db vendors -- that's okay
> with me.  Standardization takes a long time.

Right. It took us 7 years so far and we have learned a lot
in that time.

> For my part, I'd be happy to have a public C API added to the datetime
> module starting with Python 2.4 (we could even backport it to Python
> 2.3.4).  I'd appreciate some help, but if this is the *only* think
> keeping datetime from eventually becoming the de-facto standard for
> representing database date and time values, I'd do it myself.

Note that the DB API specification allows database module
authors to use datetime types to represent date and time
coming from the database and accepting it on input.
There's no need to change the DB API to have database
authors use datetime, e.g. as configurable option.

The reason that there isn't much support for it
is that most database modules are written in C and without
an easy to use C API it's not easy to convince authors to
provide such an option (I've tried to do that myself for
mxODBC but then refrained from adding support simply because
the overhead of calling out to Python would have slowed
down result set access considerably).

Another downside is that datetime types don't support
int/float slots providing Unix ticks values. Adding these
would make it a lot easier to at least use datetime
types as input to .execute() since many database modules
accept integers/floats and interpret them as Unix ticks
values.

>>>my oppinion is that the part of the dbapi related to typecasts is
>>>flacky. 
>>
>>Probably because it only defines them for the input side
>>and not the output side :-)
>>
>>It is certainly a weak point in the spec and we should
>>do something about it for the next revision.
> 
> Great!  I don't know this part, but it seems there's consensus that a
> new version should be developed.  Let's get started!

Not a new version.... a new revision will do nicely. We don't
need revolution, just evolution.

Happy New Year, BTW :-)
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 02 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::