[Chicago] Random book notes

[Martin Maney]

Thanks to Peter for halfway reading my mind during a time when I was
too busy to give this the attention it deserved.  This is all a bit
tangled - I keep getting my own vision tangled with what Django does
and doesn't do, and I guess I've never told the story behind that
coherently.  So, trying to keep it not too long...

Sometime last year, while working on the officially approved (but not
officially funded, hence "spare" time) lab\\\ facilities scheduling
application, I came to realize that I was repeating myself far too
often in several places.  The most obvious was probably between the
database definition, which at the time was implemented as SQL
statements; the (re)loader code; and various places in the (web) UI
code.  About this time last year I was working on a sketch of a
Python-based way of getting the basics stated once, and then either
generating (eg., SQL to create the database) or simply using (loader
code, UI code) the information about field types and sizes.  By the
time of Adrian's presentation last summer, I had realized that this
subproject would likely be as large as the user-visible application
that it was first intended to support.

So I was moderately depressed by that, feeling that whatever the long
term gains it was just too big a bite to take, when Django burst into
my ken.  At first glance it did so much of what I had been struggling
with that I was ecstatic.  As I learned more about Django, it turned
out that a lot of things were just Not Right in terms of what I was
looking for.  If there's a common cause for these disappointments it's
probably that Django is designed for a common type of use: an
application that is solely hosted over the web, so it uses in-database
constraints in a very limited fashion (nothing much aside from foreign
keys and not-null) and does most other sanity checking in cdoe that
exists only in its own framework.

My point of view was that the web interface was at best a compromise,
one adopted to make it easier for a large group of casual users to get
easy access to the application.  It wasn't really my first choice for
the app's interface, merely the most expedient.  For practical reasons
the web UI was the one I put most of my effort into, but there was
always some background maintenance and bulk updates that were done
completely separately.  The tools for these were really crufty, as I
was the sole user of them, though that was once again expediency -
easier, for now, to do some scut work myself than to provide a
user-friendly interface for these things.  But they served to remind me
that the web interface wasn't the beginning and end of the thing.

Thus, Django's approach, with most consistency checks implemented in
the web-oriented framework, was distasteful to me.  Sure, in theory I
could probably get access to those checks in a way that would be
useable by non-web code, but given the way its buried in a framework
that's all about the web interface I didn't feel this was a direction I
would wish to go.  I this connection I want to point to Guido's short
blog post at http://www.artima.com/weblogs/viewpost.jsp?thread=152104
Frameworks aren't toolkits, they're skeletons.  Trying to use Django
that way would feel like carrying along an entire skeleton in order to
make use of a few ribs.  Not impossible, but awkward and clumsy, with
the added bonus that the interfaces I'd need are largely framework
internals, hence more likely to change (or even disappear) than I would
be comfortable with.

Okay, this is too long and starting to wander.  On with it!

On Fri, May 05, 2006 at 01:04:17PM -0500, Peter Fein wrote:
> CONSTRAINTs, I suppose.  CHECK, UNIQUE (for non-PK), partial indexes, etc.

And other data validation by way of functions, triggers, etc, yep.

> I don't use ORMs, but usually use a mix of application and database validation 
> techniques.  Some constraints (UNIQUE, say) cannot be implemented by the 
> application (b/c of transaction semantics).  Others (data integrity for an 
> individual row) are often better done in the app, where you have a 
> friendlier/more powerful language to work in.

I disagree only in that I'm not comfortable with the assumption that
important constraints can sensibly be placed in only the application
side.  OTOH, I'm all in favor of having both sides check for many
things - the app needs to check a lot of things in order to provide
good/timely error feedback to the user rather than letting things get
caught by the database constraints alone.  But in general I'd rather
have them only in the db than only in the app or framework, since that
way you only lose (probably) quality of error reporting, not quality of
data.  And you don't lose important protections when accessed other
than through the app.

The only problem I have with duplicating checks is the need to state
them more than once.  :-(

> It all depends on how much you care about data correctness/purity.  I care 
> quite a lot...

I can only add that it's perfectly reasonable to care more or less
strongly about data correctness, based upon the reasonable needs of the
applications.  But it's not at all reasonable to insist that what's
good enough for, say, a blog, will suffice for all applications.  Not
that we might all agree about just what degree of care a blog calls
for, either.  :-)

-- 
It isn't that secrets are never needed in security.
It's that they are never desirable.  -- Whitfield Diffie