Hmmm. Interesting context for the assignment Glyph just gave me - come up with a friendly and robust config system for servers. I'm just coming up to speed, so I have nothing to contribute yet, but I'm eager and this is a good set of questions to begin with.
Sounds like one problem to solve is to differentiate between the initial state with which you start/construct a server and the runtime state that evolves during the use of the server, which is pickled. We need a robust and easy way to a) establish the initial state (a la configure a server) and b) enable that state to be saved and re-used in the future (e.g. kept in CVS or other source control) while c) not mucking with the persistability of the runtime state.
As I said, I am a twisted neophyte, so I'm going to spend much time learning and listening and experimenting for a while first, but thanks for brining the topic up.
-----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Kevin Turner Sent: Saturday, July 28, 2001 2:26 PM To: twisted-python@twistedmatrix.com Subject: [Twisted-Python] Configuration, Persistance and Version Control
So, Tv and itamar were on channel today, asking about Twisted philosophy, persistance, and configurability. Lacking the presence of anyone closer to the pure Twisted light, I tried to be a good little minion and make the case... But since I'm the guy who doesn't quite believe in .spl files, I wasn't able to be terribly convincing.
And now I could try to play his side of the argument with you guys, but ya know, it'd be so much easier for me to just let his words do the talking. Scrollback excerpts:
<itamar|home> anyone who knows how twisted's config mechanism works? <itamar|home> typically, glyph is on 24 hours a day unless I actually want to talk to a twisted developer <Acapnotic> Wait, Twisted has a config mechanism? <itamar|home> well, its server are persistent <Acapnotic> right <itamar|home> you coifgure them and then can pickle them to disk <Acapnotic> right <itamar|home> my question is: how do I configure them? <itamar|home> can this mechanism be used with other software packages? <Acapnotic> You load them up, set them, and re-pickle them. You can also set them through the interactive python interface on the telnet server, say <Acapnotic> Now somehow, Glyph claims that this is easier than having configuration files. <Acapnotic> I am not yet convinced of that, however. <itamar|home> I am <itamar|home> my software right now needs either something like that or an XML config file <itamar|home> which'll end up being a pickle equivalent anyway, so there's no point
[...]
<Acapnotic> itamar: Well, so far as I've been able to tell, there's not overmuch magic there. 'mkfooserver' just creates an instance of a server object and pickle.dumps it, twistd just pickle.loads it and invokes its the run() method. <itamar|home> Acapnotic: well, my configs are pretty complex <Tv> Acapnotic: But all that does is make your "config settings" into "command line settings". <Tv> I don't see it making things very easy. <Tv> And think about version upgrades.
[...]
<Tv> I see trouble with the approach; mostly losing the unix ubiquitousness (sp?). I can grep my config files; I store them in CVS. <Tv> DavidC_: You want every program to contain some UI or command mechanism to fully configure it. <Tv> There's a lot of things I'd wish someone would spell out about twisted. One thing is "how do I version control my configuration?" <Tv> And don't give a trivial example.. <Tv> I have multi-hundred-line config files here. <Tv> Shared between n hosts, via CVS. <Tv> I'd love to hear more of the twisted philosophy..
[...]
<Tv> Personally, I have _needed_ programmability in config files many times.
[...]
<Tv> Acapnotic: Oh, I like version control. I wish I could put more things into it. <Tv> Have you ever screwed up a config file? <Tv> Have you ever downgraded something, and found out it won't understand the new config file, and had to recreate the config to match the older version? <Tv> The same reasons you version control source code.
<Acapnotic> tv: you should post to the list at twisted-python@twistedmatrix.com <Tv> Acapnotic: I'm not following any more lists until I have migrated my mail to another domain :) <Acapnotic> tv: bah, you don't have to follow it, just post ;)
[...]
<Acapnotic> Tv: so if you want programmable config, is there a problem with just using python? In which case your configuration process is a script that does "app = pickle.load(...) app['web']. port = 8080 ; app.append(ChickenServer()) ; ... ; pickle.dump(app, ...) ? <Tv> Acapnotic: That may very well be what I'd do. But then the mechanism needs to support _both_ getting config programmatically and state from pickle, at the same time. <Acapnotic> Well, if I follow this train, there is only the state of the application, since it is persistant. That state may be changed or examined programmaticly. <Tv> I see something beautiful there, but.. <Tv> noone has yet explained persistence to me well enough to convince me. <Acapnotic> but what happens when you upgrade? heh. <Tv> What if a buggy program corrupts its state? <Tv> Then you lose your config file also. <Tv> There needs to be something else. <Tv> Something that is less volatile. <DavidC_> what do you mean? both config and state are the same thing. <Tv> No. <Tv> config changes when *I* say it changes. <itamar|home> nah, config is parsed and then "rendered" into state <Tv> program can change it's state as it wants. <Tv> Any bug in program must never change the port to bind to. <Tv> :) <Tv> I see config as sort of a factory for initial program states, if you want to put it that way. <itamar|home> lets say by mistake someone does "del serverInstace. port" <itamar|home> and then shuts down the server <itamar|home> the pickled state is now wrong <DavidC_> well that would be a bug. <itamar|home> the point is that if you have an external, version controleld config file <itamar|home> these bugs don't bite you, or even if they do at least they don't effect the configuration
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python