Thoughts on language-level configuration support?
jfager at gmail.com
Wed Apr 1 06:15:19 CEST 2009
On Mar 31, 10:44 pm, "Rhodri James" <rho... at wildebst.demon.co.uk>
> On Tue, 31 Mar 2009 07:06:50 +0100, jfager <jfa... at gmail.com> wrote:
> > On Mar 30, 9:31 pm, "Rhodri James" <rho... at wildebst.demon.co.uk>
> > wrote:
> >> On Mon, 30 Mar 2009 16:59:12 +0100, jfager <jfa... at gmail.com> wrote:
> >> > It's the configuration problem. Right now you would use something
> >> > like ConfigParser or optparse to populate some configuration object,
> >> > which you would then pass around and extract values from. This would
> >> > provide two advantages over these approaches: it would separate "what
> >> > can be configured" from "the mechanism by which it is configured" -
> >> > i.e., I, programmer, don't have to make a decision about what the end
> >> > user has to do to give me those values.
> >> This would be a interesting idea, but ultimately no more than a veneer
> >> over the current set of configuration possibilities. Quite how such a
> >> system would tell whether to get configuration data from command line
> >> parameters, a config file somewhere, or just pull bytes off the Z-wave
> >> from Mars, I'm not at all clear.
> > Not a veneer over; a foundation under. As I acknowledged in my blog
> > post, this would require a small bit of bootstrapping, to configure
> > which end-user configuration system was used. But this would simply
> > point to an adapter, that would map from the desired configuration
> > interface into the canonical standard api provided by the language
> > system.
> By magic: see later
> > The problem with the current set of configuration possibilities is
> > that there's nothing really constant between them, unless the
> > programmer explicitly codes it, even though they're basically
> > accomplishing the same thing. There's nothing amazingly special about
> > this proposal, it's just saying: that basic thing that we do in a lot
> > of different ways, let's make as much of that as possible standard. I
> > think that cutoff point is below the end-user interface, since there
> > are a lot of valid ways to want to actually inject config data
> > (command line, local files, from a db, etc.), but there's no reason
> > that the end-user interfaces can't rest on top of a lower-level common
> > format that the programmer can rely on.
> There is actually. The reason is that those different configuration
> interfaces are actually accomplishing different things. You're lumping
> together both config files (wherever they may be and whatever format
> they may be in) with command line parameters and GUI command interfaces
> when the one is providing long-term defaults and the other short-term
> The level of detail that you can have in a config file is something that
> you never want to see on a command line. When you do, as with say "gcc",
> the results are rather mind-numbing. This distinction is something that
> your reduction of configuration to a single black box fails to capture.
The fact that you 'can' expose something through a particular
interface doesn't mean you 'have to'. If command line args don't make
sense for a particular program, nobody is would be forced to use
them. That's the point - the interface the end user works with is
flexible, it's what they want to deal with.
> >> > And it would allow the
> >> > configurable surface of the program to be discoverable; I wouldn't
> >> > have to poke through code or documentation that was maintained
> >> > separate from the source code.
> >> Except that you still do.
> > No, you don't. If you do, then this proposal hasn't been implemented
> > (and the little code snippet I provided in my post, please don't
> > consider that a full implementation, it was just a sketch I threw
> > together in a really short time). I'm not saying discoverability
> > magically arises; I'm saying that it's one the main points that would
> > be deliberately put into place.
> Yes you do. This spread-about inline style of configuration is in no
> sense discoverable to the casual reader, who isn't going to know that
> a particular config option even exists unless he or she chances upon
> the line where it's registered.
"Discoverable", as in built-in tools that let you have the following
conversation: "Program, tell me all the things I can configure about
you" - "Okay, here they all are". No digging through the source
> I'm not commenting on your code
> snippet here -- or I'd be here all night, and you really wouldn't
> like me at the end of it --
:) Dude, that was all of 15 minutes of coding (not counting the
diversions spent poking around the bytecode tools for the first
> but I am commenting on the usage style
> it's obviously built to support. That usage style is much less
> visible than any of the common (or uncommon, for that matter) styles
> of getting config input, and discourages documentation habits that
> would otherwise mitigate it.
> >> You've just specified a different way in
> >> which you have to do this, one that's a good deal less visible in
> >> the code
> > Why would it be less visible? If it's syntax, you would know exactly
> > where it was just by looking. I would argue that it's more clear - no
> > more boilerplate code spent shuffling around config objects that you
> > peel values out of, just write the code that does the work you're
> > actually trying to accomplish, with a small bit of syntax to mark that
> > the end user can provide an alternate value.
> As I said, it's less visible because it's decentralised. The moment
> you do that, you lose all hope of centralising the documentation, and
> the visibility of your options plummets.
The tools provided by the language do the centralization for you, that
was one of the main points.
> >> and a lot less self-documenting because it's spread out all
> >> over the place.
> > Actually, you get the best of both worlds. You get to see clearly in
> > the code how the configured values are actually used, and you get the
> > gathered summary of configurable values from the "discoverability"
> > implementation.
> What "discoverability"? You still haven't told me anything to make
> me believe that this configuration setup is discoverable at all without
> external tools.
The whole point is that the tools for this would be provided by the
> >> It also has the major disadvantage from my point of
> >> view of requiring Python to do magic in the background to figure out
> >> just what is being configured.
> > If it's built into the language, it stops being magic, and turns into
> > "how it works".
> It remains magic when it requires the language to invent things on the
> fly that the programmer has no means of discovering until the code is
> running, by which time it's generally too late. For example:
> spam = conf(1)
> How does that translate into a command-line option? -s? --span?
> Both? -S, even? --first-config-option-I-came-across? If the first,
> what happens when:
> span = conf(0)
> shows up to initialise a class attribute some time later? What
> restrictions can be put on the value you get back? What can the
> help system say about this, or do we have to go back to doing all
> that by hand? Now translate all those questions into the very
> different environment of a config file. Repeat with a database,
> and all it's quirks. By the time your colossus has acquired
> enough parameters to at least hint at the desirable answers to
> these questions, you've effectively duplicated the interfaces to
> all of the config mechanisms you're trying to replace and you've
> still lost a whole heap of flexibility.
Yes, you're right, the code that actually injects the configuration
isn't trivial. I never intended to imply that it was. But it would
probably only have to be written once (people would only write their
own if they had a special need). The win is that the underlying code
doesn't have to change just because the end-user configuration format
> Rhodri James *-* Wildebeeste Herder to the Masses
More information about the Python-list