[Twisted-Python] twisted server config: command line, environment or Axiom?
Hi, I have a bunch of configuration for my service. Things like endpoints, location of the root axiom store… Right now I grab these from the environment. That means that my twistd IServiceMaker plugin's options is pretty much by definition empty. I'm wondering if more mature twistd users have already learned the hard way what the best way to manage this is. I see three possible solutions: 1. environment variables as I use them now: portable, simple. 2. command line options: still pretty portable and simple. additional benefit: easy discoverability of options 3. axiom store: has the benefit that there's a single source of truth for everything, i.e. I copy a store and I can recreate the service based on that store exactly, using just the store Like I said, I currently use (1), it looks like (1) and (2) are more or less equivalent, I'm not sure if (3) is a good idea. I'm not really unhappy with (1), it's just that finally writing my ServiceMaker class made me contemplate the options. (3) has the downside that it conflates user data with configuration data. I'm not sure that's a good thing or not. It sounds very similar to axiomatic, which I've heard of, seen, but never played with. I already use environment variables to decide whether or not to run certain tests ("live" functional high level tests, with real disk and network IO, that take way longer than the unit tests to run). I'm going to keep it that way, because there's no obvious way of passing configuration to the tests other than the environment. cheers lvh
On Wed, Aug 08, 2012 at 03:58:27PM +0200, Laurens Van Houtven wrote:
Hi,
I have a bunch of configuration for my service. Things like endpoints, location of the root axiom store? Right now I grab these from the environment. That means that my twistd IServiceMaker plugin's options is pretty much by definition empty.
I'm wondering if more mature twistd users have already learned the hard way what the best way to manage this is. I see three possible solutions:
1. environment variables as I use them now: portable, simple. 2. command line options: still pretty portable and simple. additional benefit: easy discoverability of options
Both, preferrably. Defining all of your configuration in terms of t.p.usage.Options is not a bad thing at all. It's human readable, and scriptable. And it maps directly to ENV vars or INI files (etc). If you have strange configuration requirements, (like binary files), I think those belong in their own seperate file(s). E.g. SSL certificates, PW files, etc
3. axiom store: has the benefit that there's a single source of truth for everything, i.e. I copy a store and I can recreate the service based on that store exactly, using just the store
Yeah, I hate that kind of configuration. Configuration is/should-be diffable with standard tools, if possible.
Like I said, I currently use (1), it looks like (1) and (2) are more or less equivalent, I'm not sure if (3) is a good idea. I'm not really unhappy with (1), it's just that finally writing my ServiceMaker class made me contemplate the options.
(3) has the downside that it conflates user data with configuration data. I'm not sure that's a good thing or not. It sounds very similar to axiomatic, which I've heard of, seen, but never played with.
Well as indicated *I don't like it*, but really that is only due to lack of tooling support... :)
I already use environment variables to decide whether or not to run certain tests ("live" functional high level tests, with real disk and network IO, that take way longer than the unit tests to run). I'm going to keep it that way, because there's no obvious way of passing configuration to the tests other than the environment.
Yeah. Plain ENV vars are probably the best bet there. -- Cheers, -E
On Wed, Aug 8, 2012 at 8:58 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Hi,
I have a bunch of configuration for my service. Things like endpoints, location of the root axiom store… Right now I grab these from the environment. That means that my twistd IServiceMaker plugin's options is pretty much by definition empty.
I'm wondering if more mature twistd users have already learned the hard way what the best way to manage this is. I see three possible solutions:
1. environment variables as I use them now: portable, simple. 2. command line options: still pretty portable and simple. additional benefit: easy discoverability of options 3. axiom store: has the benefit that there's a single source of truth for everything, i.e. I copy a store and I can recreate the service based on that store exactly, using just the store
Never, ever store config data and user data in the same place. Likewise, if you're going to persist config data, it should be in some kind of plain text format. 1) and 2) are fine, though I prefer a simple text file format as well, and I like to have multiple levels of configuration which are overridable. e.g. one or more config files (maybe in /etc, in /home/<user>, the pwd) which override each other in a specified manner, which are in turn overridable by command line options and/or env vars whether any of this is appropriate for your case, and how much complexity you need obviously depends on your application Kevin Horn
On 01:58 pm, _@lvh.cc wrote:
Hi,
I have a bunch of configuration for my service. Things like endpoints, location of the root axiom store… Right now I grab these from the environment. That means that my twistd IServiceMaker plugin's options is pretty much by definition empty.
I'm wondering if more mature twistd users have already learned the hard way what the best way to manage this is. I see three possible solutions:
1. environment variables as I use them now: portable, simple. 2. command line options: still pretty portable and simple. additional benefit: easy discoverability of options 3. axiom store: has the benefit that there's a single source of truth for everything, i.e. I copy a store and I can recreate the service based on that store exactly, using just the store
This doesn't really seem like a Twisted question... but... axiom is my favorite configuration persistence tool.
Like I said, I currently use (1), it looks like (1) and (2) are more or less equivalent, I'm not sure if (3) is a good idea. I'm not really unhappy with (1), it's just that finally writing my ServiceMaker class made me contemplate the options.
(3) has the downside that it conflates user data with configuration data. I'm not sure that's a good thing or not. It sounds very similar to axiomatic, which I've heard of, seen, but never played with.
It doesn't conflate user data with configuration data any more than any other system. Keep a database for your configuration and keep your user data somewhere else (perhaps in a separate axiom store). Jean-Paul
I already use environment variables to decide whether or not to run certain tests ("live" functional high level tests, with real disk and network IO, that take way longer than the unit tests to run). I'm going to keep it that way, because there's no obvious way of passing configuration to the tests other than the environment.
cheers lvh
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Wed, Aug 8, 2012 at 3:58 PM, Laurens Van Houtven <_@lvh.cc> wrote:
(3) has the downside that it conflates user data with configuration data. I'm not sure that's a good thing or not. It sounds very similar to axiomatic, which I've heard of, seen, but never played with.
As exarkun mentions in his reply, you don't have to store the configuration and user data in the same Axiom store. If you're interested, the way "axiomatic start" works is that it locates all IService powerups on the store you pass it (which would be the "site store", in Mantissa), and attaches those to the application. I think most of the interesting IService implementations are in Mantissa, not Axiom, but axiomatic itself does not depend on Mantissa and of course there's no reason you can't write your own IService powerups. -- mithrandi, i Ainil en-Balandor, a faer Ambar
participants (5)
-
Eric P. Mangold -
exarkun＠twistedmatrix.com -
Kevin Horn -
Laurens Van Houtven -
Tristan Seligmann