Why database modules are incomplete

[Paul Boddie]

Dave Cole <djc at object-craft.com.au> wrote in message news:<m3k81h1u3d.fsf_-_ at vole.object-craft.com.au>...
> >>>>> "Paul" == Paul Boddie <paul at boddie.net> writes:
> 
> Paul> That's not to say that one doesn't have a certain feeling of
> Paul> unease when evaluating Python modules for serious database use,
> Paul> however. It does seem like a reflection of the open source
> Paul> community at large when there are lots of seemingly incomplete
> Paul> modules covering certain narrow, fashionable areas, and few
> Paul> modules covering other areas.
> 
> Boy, would I ever like to be able to complete my modules...

Isn't that true for most of us? ;-)

> The problem with producing free software is that you can only dedicate
> so much time to it before the consulting business goes belly-up.  It
> would make my year if someone who used one of my modules approached me
> and said that they would sponsor some development on it.

Indeed. Whilst sponsorship was seen as one of the big ways forward in
commercialising open source, only large organisations like IBM seem to
have the luxury of pursuing this aggressively, and even then most of
their funding seems to be spent in-house.

> In the absence of such a sponsor, I am limited to my spare time (such
> as it is) - the rest of my time goes into making sure that I do not
> lose my business, house, car, family, ...

Understood. But as I pointed out in the thread from which the message
you respond to came from, the usefulness of your module in your
business, and your reliance on that module, should be seen as
guarantees that you take its development seriously. It is for this
reason that I took exception to Mr Wilson's "Maverick" accusations.

> Paul> I mean no offence to any database module author when I use the
> Paul> term "incomplete"; my definition of "completeness" may differ
> Paul> from that of any given developer in this case, and presumably
> Paul> most database authors are satisfied with what they have
> Paul> written. Moreover, I wouldn't like to make any criticism of the
> Paul> reliablity of any module without having seen it first. I
> Paul> sometimes get concerned that serious commercial users are likely
> Paul> to have views on issues of functionality and reliability even
> Paul> more extreme than my own, however.
> 
> If the module is almost good enough surely you should be entering a
> discussion with the module author to work out how it can be made good
> enough.

Indeed. And in that original thread I later raise this point.

Actually, I think I did try your Sybase module out about a year ago or
so, by the way. What held me back there was, I think, the lack of
certain required libraries with my Sybase system, but I may have got
it working in the end. Sorry if I never communicated that!

An interesting point emerges here, though. The process of module
discovery can be somewhat stressful. It's quite likely that many
people give up "too easily" while building extensions, for example,
and move on to the next candidate module. In my Sybase adventures, I
tried your module and ctsybase first; ctsybase was easiest to get
working but lacked support for bind/bound variables. Since I can't
abide working without them, I moved over to mxODBC, even though the
effort in getting ODBC working can be significant, but at that point I
had the possibility of choosing other database systems and was
satisfied enough not to do any more evaluation of other modules.

> In my case, I can always dedicate spare time for development, so while
> funding would be nice, it is not essential.  What is more important is
> to be able to find people who are willing to exercise the module in
> their environment and work with me to resolve problems.

I certainly accept this. It can be disheartening to release something
which seems interesting and exciting to oneself, only to be met with
near silence in response. In practical terms, I suppose such a
reception means that the module never gets developed in ways that some
potential users would like, and a rift emerges between module authors
who must address their immediate needs and potential users who
consider those modules to marginally lack relevance to them to be of
any real use.

> In the entire history of the Sybase module(s) I have developed, I have
> only been contacted by four people who experienced a usage problem
> with the module.  There are two possible interpretations to that:
> 
> 1- I write awesome code which has only the most obscure bugs.

I think obscure bugs are the things that worry people most
particularly about database modules. ;-)

> 2- People who experience problems do not bother contacting me.
> 
> Although it would be nice to think that option 1 is true, I suspect
> that option 2 is closer to the truth.

There's another possibility: people may experience problems but may
not have your level of skill to diagnose or investigate them. I accept
that this isn't a good enough excuse when, in this case, there is
someone taking responsibility for the module, but then in periods of a
maintainer not having enough time to work on the module whilst doing
paid work, development and fixes are understandably delayed. Whilst it
seems harsh and ungrateful, developers are often impatient creatures
who only report problems when they happen and frequently give up
quickly, choosing the path of least resistance (to the next candidate
module).

> Paul> However, if you are referring to a standard API for databases,
> Paul> then one already exists. Unfortunately, most modules aren't
> Paul> compliant with the most recent version of that API as far as I
> Paul> can tell.
> 
> Time or money is the solution to this in my case.

Of course, and no-one can or should be blamed for this. One big
problem in the development of database modules in particular is the
rarity of developers motivated enough and with enough resources to
help people like you out. Not many people in the past have had access
(or the inclination for ideological or other reasons to access) Sybase
database systems whilst developing interfaces to the native libraries.
Indeed, even those who have access to such systems may find libraries
and headers missing.

What this means in general is that some projects are likely to be
driven by a single developer without "peers". A good way to make this
situation more bearable for such developers and for the community
would definitely go down well.

Paul