[Twisted-Python] Configuration, Final Spec
![](https://secure.gravatar.com/avatar/faeb19fbb879e56c1be4a300cfd80ec8.jpg?s=120&d=mm&r=g)
Rewritten, standalone, spec OK, let me first state some axioms: 1. Everything should be configurable the same way, from within twisted. Glyph has mentioned that he's not familiar with what's in bin/ anymore: he just mktelnetserver, and configures from within the telnet server everything. This is what should happen, except not limited to the telnet server. 2. The configuration interface must not be ui specific. This is obvious, right? Command line and web based aren't the only option. No reason why we can't implement a special purpose client/server for configurin g twisted. 3. It should be easy to optimize the configuration for a ui This is like the last one -- since we might deal with very good UIs, we need to give them enough information for using all their abilities to help us. 4. It should be very easy to make an object configurable. This is very important -- the harder it is, the less it will be easy to add *good* code to twisted. This is what may be the single most idiotic thing in Zope (and DC is aware of it! and thinking of how to fix it!). Let's learn from their mistakes: the less methods, the better. The more the learning curve is gradual (not needing to learn a class/method/interface before you need the functionality), the better. OK, so what do we need to do about it? Here's a rough proposition: the configurable *interface*, which will be a class, but not a class people should inherit from, will contain the following methods: .getParameters() --> return a dictionary: name of parameter, Parameter object .setParameter(name, answer) --> notify the application that an answer has been given to a particular question It can throw an InvalidParameter exception with a string for a reason. This is for application-level verification. Note that application-level verification is discouraged. .endParameters() --> the "application" promises that no methods of the object will be called between a series of .setParameter()s and .endParameters(). So, this means that if the UI got a bunch of answers, it will call .setParameter() several times, and then .endParameters(). The UI will *check* for this method's existance, and will not call it if it doesn't exist. .setParent(parent) --> if the object is created inside an InterfaceProperty of some other object, module nested ArrayParameters and DictParameters, .setParent will be called with that object, *if it exists*. Parameter objects are meant to be open ended. They can contain a default. Here is the general interface of the Question, that all objects conform too: .hasValue() --> boolean, whether the Parameter already contains an value .getValue() --> will only work if .hasValue() is true, returns the answer .setValue(val) --> make .hasValue() true .title --> A string, the human-readable description of the property. Objects which can be created by the UI should have an initParameters attribute which should be a sequence of Parameters, which correspond to the mandatory arguments of the __init__ method. Well, my proposal would not be complete if I didn't say what parameters are available. Keep in mind, though, that the set of parameters is *open ended*. That does not violate the light-and-lean guidelines, since a specific Parameter class will only be used if the functionality is needed. Without further ado: class BooleanParameter: class IntParameter: (can have .min and .max) class FloatParameter: class StringParameter: (can have .maxlength) class LongStringParameter: (same as above -- it's a hint to the ui) class InterfaceParameter: (specify interface, valid answers are objects) class ArrayParameter: (an array of the same kind of question) class DictParameter: (a dictionary mapping strings -> same kind of question) The constructor will always take a title as a first argument, and possibly some more parameters if appropriate. I'm thinking of adding some generic methods, the most interesting one is traverse, which either returns a Parameter or throws a ValueError("node parameter"). A class will be said to implement an interface if the interface class is a member of the __implements__ attribute. We can refine it to saying that the interface is a superclass of some member of __implements__: def implements(klass, interface): for member in klass.__implements__: if issubclass(member, interface): return 1 return 0 This leaves open the question of how classes register with the UI at all. A possibly radical solution will be to have the UI scan through sys.modules, and each class will be checked for the __implements__ attribute. A less radical option is to have a register(klass) function which registers it with the UI. register() can be used from within the module, if it is an active participant, or from within an importing module. example: class Server: # note -- not inheriting from anything initParameters = (StringParameter("Server Name"), IntParameter("Port")) def __init__(self, name, port): self.name, self.port = name, port def getParameters(self): name = StringQuestion("Server Name") name.setValue(self.name) port = IntQuestion("Port") port.setValue(self.port) return {'port': port, 'name': name} def setParameter(self, name, answer): # note: no need to use int(answer) # for port: an IntQuestion has an integer as a .getValue() setattr(self, name, answer) -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez@debian.org |http://www.{python,debian,gnu}.org -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez@debian.org |http://www.{python,debian,gnu}.org
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Saturday 12 May 2001 02:32, you wrote:
Rewritten, standalone, spec
[ agreement with axioms ]
IMHO, people *should* inherit from it, so that we can have some visibility on what implements it. I agree that it should provide as little functionality as possible, however.
I don't understand this part of the design. In my humble (but correct) opinion, it is *always* bad API design to force the caller to "promise" something. (Sometimes, other design concerns mitigate this decision, but I don't see one here. However, I have frequently joked about having a neon sign in my office, for design discussions, that says "IT IS NOT THE CALLER'S RESPONSIBILITY" in 10-foot-tall letters) I would much rather see a .getParameters() -> dict / .setParameters(dict). It seems more symmetrical, and if you can "promise" that (A) setParameters may be called some number of times and (B) endParameters() will *always* be called afterwards, we can put that loop into the config side of the code. Any reasons why it can't be done this way?
Of the "Parameter", I take it you mean? :)
Another interesting open question (although not one I'm sure this code has to address) is how to get the respective configuration interfaces to represent object identity. (e.g. I want to mount the *same* web resource on /foo, on /bar)
There is a third option -- make Configurable a metaclass :) However, I prefer the register(klass) way, both for clarity and efficiency. Explicit is better than implicit.
Your code has a bug in it, which is a good example of why I like to actually inherit from interfaces :). You don't define "endParameters". One more question -- why doesn't the Configurable interface class actually do some things, like data validation? It sounds like it'd be possible (isCorrectInput method on the Parameter classes, or somesuch...) On the whole this looks like a good proposal, although I'm still not entirely sure what the final end-user experience of configuration will look like. -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph
![](https://secure.gravatar.com/avatar/faeb19fbb879e56c1be4a300cfd80ec8.jpg?s=120&d=mm&r=g)
On Sat, 12 May 2001, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
No, people should put it in the __implements__ attribute.
Yes. In the usual case, where you don't need a .endParameters(), then the code will be *much* clearer. You also missed the place where I say ".endParameters() will not be called if it doesn't exist". See axiom 4 -- certainly def setParameter(self, name, value): setattr(self, name, value) (which is the usual case) Is much quicker then def setParameters(self, dict): for name, value in dict.items(): setattr(self, name, value) And another good reason -- this makes it much easier when doing application-level validation, to know what failed: # configuring RSA def setParameter(self, name, value): if not isPrime(value): raise ValueError("not prime") setattr(self, name, value)
This code isn't the one that should address it -- this is UI's responsibility. Take *another* look at the way Zope does it.
However, I prefer the register(klass) way, both for clarity and efficiency. Explicit is better than implicit.
Indeed, I've decided I agree, for a different reason -- we do not want spurious options for the user.
Your code has a bug in it, which is a good example of why I like to actually inherit from interfaces :). You don't define "endParameters".
No it doesn't. It's permissible not to have .endParameters(), .setParent() and .initParameters. The first two will not be called if they don't exist.
I've thought about it, and decided against it -- since the UI needs to know so much about the validation anyway, I prefer to putting the knowledge in the UI. Sure, when we realized what is convinient for two UIs, we can have some nice functions to help UIs -- but *outside* the Parameter class. The classical example is that the UI might need to write the validation code in JavaScript, for DHTML forms. Surely, a method written in Python won't help...
On the whole this looks like a good proposal, although I'm still not entirely sure what the final end-user experience of configuration will look like.
Well, I've already gave an example of what it would look like with the command line. Mutatis-mutandis, we can pretty much imagine how to translate it into web-based. Of course, many options still remain, but I don't think I care about UIs: I've specified enough information for the UI -- now the UIs can compete. (We can have 10 different web-based UIs, and choose among them) -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez@debian.org |http://www.{python,debian,gnu}.org
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Sunday 13 May 2001 00:31, you wrote:
But if you declare setParameters as a member of the Configurable class, then you don't have to write every UI to deal with the possible existance or non-existance of endParameters. I am strongly opposed to any API which requires validation on the caller's behalf, for several reasons. A practical concern is that using an RPC like gloop (e.g. an asynchronous one) it is impossible to tell if the object you are calling "has the attribute" endParameters or not. You just have to call it and hope for the best. If setParameters is defined in Configurable (the default implementation being a 2-line loop which does setParameter and then endParameter) then the extra work you're talking about for the configurable author can be avoided. Having an API which requires a garuantee on the caller's behalf (the "caller" in this case being the UI) in a language where exceptions are as common as in Python makes handling of errors somewhat tricky. After all, if setting a bunch of parameters is a non-atomic operation, then what do you do when one of your parameters is invalid, and raises an exception? Does endParameters still have to get called? It seems to me that setParameters is an atomic operation, which should validate, then test. validateParameters could be a method as well, to do extra validation if parameters are dependant upon each other. However, you can then clearly say that setParameter may never raise an exception: if the call to validateParameters succeeds, but setParameter fails, then you are in a known-to-be-inconsistent state and can display an appropriate error message. However, if some of your setParameter calls raise exceptions and others do not, should endParameters be called anyway? Does every UI need to handle exceptions itself?
Zope confuses the hell out of me, both in the UI and the code :)... but I agree that this code probably shouldn't have to deal with that.
Yay, agreement! :)
It's not to help out the UIs, it's to prevent setParameter from being called incorrectly. There are several points of failure here -- * the user typed something incorrect (presumably, your JavaScript example) * something bad came from the transport (WebUI.validate) * there is a bug in/bad data from the UI (Parameter.validate) * there is a bug in the configurable (exceptions caught by Configurable on setParameter) and I think each one has to be treated differently.
It's for this reason that I'd like as much thought to be given to making UIs easy to write as to making Configurable objects easy to write. I think I've changed my mind about Configurable not providing much functionality; The Configurable and Parameter classes should encapsulate as much common behavior as possible. (Although, I admit that there is probably not *much* common functionality.) If they don't for some reason (and I am still not convinced any such reasons exist), the "config" module should at least contain it all, and as a core part of the interface, not as an additional utility library for GUIs. -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Saturday 12 May 2001 02:32, you wrote:
Rewritten, standalone, spec
[ agreement with axioms ]
IMHO, people *should* inherit from it, so that we can have some visibility on what implements it. I agree that it should provide as little functionality as possible, however.
I don't understand this part of the design. In my humble (but correct) opinion, it is *always* bad API design to force the caller to "promise" something. (Sometimes, other design concerns mitigate this decision, but I don't see one here. However, I have frequently joked about having a neon sign in my office, for design discussions, that says "IT IS NOT THE CALLER'S RESPONSIBILITY" in 10-foot-tall letters) I would much rather see a .getParameters() -> dict / .setParameters(dict). It seems more symmetrical, and if you can "promise" that (A) setParameters may be called some number of times and (B) endParameters() will *always* be called afterwards, we can put that loop into the config side of the code. Any reasons why it can't be done this way?
Of the "Parameter", I take it you mean? :)
Another interesting open question (although not one I'm sure this code has to address) is how to get the respective configuration interfaces to represent object identity. (e.g. I want to mount the *same* web resource on /foo, on /bar)
There is a third option -- make Configurable a metaclass :) However, I prefer the register(klass) way, both for clarity and efficiency. Explicit is better than implicit.
Your code has a bug in it, which is a good example of why I like to actually inherit from interfaces :). You don't define "endParameters". One more question -- why doesn't the Configurable interface class actually do some things, like data validation? It sounds like it'd be possible (isCorrectInput method on the Parameter classes, or somesuch...) On the whole this looks like a good proposal, although I'm still not entirely sure what the final end-user experience of configuration will look like. -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph
![](https://secure.gravatar.com/avatar/faeb19fbb879e56c1be4a300cfd80ec8.jpg?s=120&d=mm&r=g)
On Sat, 12 May 2001, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
No, people should put it in the __implements__ attribute.
Yes. In the usual case, where you don't need a .endParameters(), then the code will be *much* clearer. You also missed the place where I say ".endParameters() will not be called if it doesn't exist". See axiom 4 -- certainly def setParameter(self, name, value): setattr(self, name, value) (which is the usual case) Is much quicker then def setParameters(self, dict): for name, value in dict.items(): setattr(self, name, value) And another good reason -- this makes it much easier when doing application-level validation, to know what failed: # configuring RSA def setParameter(self, name, value): if not isPrime(value): raise ValueError("not prime") setattr(self, name, value)
This code isn't the one that should address it -- this is UI's responsibility. Take *another* look at the way Zope does it.
However, I prefer the register(klass) way, both for clarity and efficiency. Explicit is better than implicit.
Indeed, I've decided I agree, for a different reason -- we do not want spurious options for the user.
Your code has a bug in it, which is a good example of why I like to actually inherit from interfaces :). You don't define "endParameters".
No it doesn't. It's permissible not to have .endParameters(), .setParent() and .initParameters. The first two will not be called if they don't exist.
I've thought about it, and decided against it -- since the UI needs to know so much about the validation anyway, I prefer to putting the knowledge in the UI. Sure, when we realized what is convinient for two UIs, we can have some nice functions to help UIs -- but *outside* the Parameter class. The classical example is that the UI might need to write the validation code in JavaScript, for DHTML forms. Surely, a method written in Python won't help...
On the whole this looks like a good proposal, although I'm still not entirely sure what the final end-user experience of configuration will look like.
Well, I've already gave an example of what it would look like with the command line. Mutatis-mutandis, we can pretty much imagine how to translate it into web-based. Of course, many options still remain, but I don't think I care about UIs: I've specified enough information for the UI -- now the UIs can compete. (We can have 10 different web-based UIs, and choose among them) -- "I'll be ex-DPL soon anyway so I'm |LUKE: Is Perl better than Python? looking for someplace else to grab power."|YODA: No...no... no. Quicker, -- Wichert Akkerman (on debian-private)| easier, more seductive. For public key, finger moshez@debian.org |http://www.{python,debian,gnu}.org
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Sunday 13 May 2001 00:31, you wrote:
But if you declare setParameters as a member of the Configurable class, then you don't have to write every UI to deal with the possible existance or non-existance of endParameters. I am strongly opposed to any API which requires validation on the caller's behalf, for several reasons. A practical concern is that using an RPC like gloop (e.g. an asynchronous one) it is impossible to tell if the object you are calling "has the attribute" endParameters or not. You just have to call it and hope for the best. If setParameters is defined in Configurable (the default implementation being a 2-line loop which does setParameter and then endParameter) then the extra work you're talking about for the configurable author can be avoided. Having an API which requires a garuantee on the caller's behalf (the "caller" in this case being the UI) in a language where exceptions are as common as in Python makes handling of errors somewhat tricky. After all, if setting a bunch of parameters is a non-atomic operation, then what do you do when one of your parameters is invalid, and raises an exception? Does endParameters still have to get called? It seems to me that setParameters is an atomic operation, which should validate, then test. validateParameters could be a method as well, to do extra validation if parameters are dependant upon each other. However, you can then clearly say that setParameter may never raise an exception: if the call to validateParameters succeeds, but setParameter fails, then you are in a known-to-be-inconsistent state and can display an appropriate error message. However, if some of your setParameter calls raise exceptions and others do not, should endParameters be called anyway? Does every UI need to handle exceptions itself?
Zope confuses the hell out of me, both in the UI and the code :)... but I agree that this code probably shouldn't have to deal with that.
Yay, agreement! :)
It's not to help out the UIs, it's to prevent setParameter from being called incorrectly. There are several points of failure here -- * the user typed something incorrect (presumably, your JavaScript example) * something bad came from the transport (WebUI.validate) * there is a bug in/bad data from the UI (Parameter.validate) * there is a bug in the configurable (exceptions caught by Configurable on setParameter) and I think each one has to be treated differently.
It's for this reason that I'd like as much thought to be given to making UIs easy to write as to making Configurable objects easy to write. I think I've changed my mind about Configurable not providing much functionality; The Configurable and Parameter classes should encapsulate as much common behavior as possible. (Although, I admit that there is probably not *much* common functionality.) If they don't for some reason (and I am still not convinced any such reasons exist), the "config" module should at least contain it all, and as a core part of the interface, not as an additional utility library for GUIs. -- ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://twistedmatrix.com/users/glyph
participants (2)
-
Glyph Lefkowitz
-
Moshe Zadka