Hi Einar, On Sep 8, 2010, at 6:19 AM, Einar S. Idsø wrote:

I am trying to make one class/service hook onto another class/service to add to its functionality without disturbing the original class/service.

So... you are trying to write a computer program, is what you're saying? ;-)

My situation is this: I currently have a class which at specific intervals queries one server per instance (via t.u.getProcessOutput()) for some basic information. Each instance has a method which analyses its returned data and stores the result it to ts internal data structure. My dilemma is that I wish to leave this class alone, but I also wish to be able to add functionality from external classes based on the result of said method. For instance: If the method analysing the result from getProcessOutput() discovers a certain user on the server being queried, I wish to store the data to a mysql database. Under other circumstances I wish to send a notification to myself over MSN or via mail. I can easily envision myself wanting to respond to the results with other actions as well in the future. How can this be achieved?

It sounds like this is a pretty trivial application of the Publish/Subscribe pattern (or perhaps the Observer) pattern. <http://en.wikipedia.org/wiki/Publish/subscribe> <http://en.wikipedia.org/wiki/Observer_pattern> In less fancy language, "it sounds like you need a 'for' loop".

I can of course modify the original class so that I pass an adbapi.ConnectionPool as optional argument to __init__() and modify the method to check for the existence of such an object before querying the database. I would then also need to add another optional argument which is whatever object needs to be passed to allow the class to send a message to MSN, and still more optional arguments for any such new service I'd like to add. However, as mentioned I'd rather not modify the original class, and doing this seems like too much of a hack. Or is it? Perhaps inheritance is the key, although that would require me to modify existing methods which I know to be working just to tack on some additional functionality at the end.

No, inheritance isn't the key. It never is - composition is the key :). You're right that adding lots of special-purpose arguments to __init__ that each do one thing is a bad idea. Don't do that. Add one general-purpose "list of observers that want to be notified (when X happens)" argument. Then when your getProcessOutput() finishes, just do 'for something in self.observers: something.youJustGotServerStatus(newServerStatus)'. It usually makes sense to define an interface with a couple of interesting methods when you do this, so that you can have more methods. If you've only ever got one interesting type of event for this class and you're sure you won't ever have more, you might just use callable objects instead of methods, since that will let users pass in functions (i.e. 'for observer in observers: observer()').

What are my options here? I took a look at t.p.hook, and it seems like it is perfect for the job: After having instantiated my "global" adbapi.ConnectionPool(), I can just add a pre- or post-hook to any class's method from which I'd like to store data. That way I can make a db-instance which hooks in to any part of my application and stores any result imaginable from any method. However, it seems that t.p.hook is very rarely used, so perhaps there is a better way of achieving what I want?

Please, please do not use t.p.hook. I hope that we delete it one day. The only thing in all of Twisted which uses t.p.hook is twisted.python.threadable, which is also very lightly used. This functionality could easily be implemented using a decorator instead of the awfulness that twisted.python.hook gets up to. If you look at how twisted.python.hook is implemented, you'll notice that it generates strings of Python code at runtime, and compiles and runs them. More generally, please don't try to do what t.p.hook tries to do, and add functionality to a class that you wrote without that class's cooperation. If your application depends on magical external modifications to classes in order to work as expected, it will become flaky and difficult to maintain. For one thing, imagine some new developer reading the code: they're looking at the body of the method, they're looking at the code of the method, and then they're looking at some code in some totally random other spot that is somehow getting invoked when they call it. Imagine the frustration as they try to figure out why that is happening. If you instead just have an explicit list of observers, which are passed in to the class, then if one of them blows up, it will be very easy to figure out where it's blowing up: the line will be observer.youJustGotServerStatus(), there will be a couple of youJustGotServerStatus method somewhere that you can track down bugs to. In some situations, if you're dealing with somebody else's code, that you can't modify for some reason at the source level (for example: you want to work with a released version of the standard library, or an older version of Twisted) it might make sense to patch in some extra functionality from your code. In that case, using something like twisted.python.hook that enforces a particular style (again: do not use twisted.python.hook itself, it is old, crufty code that is unnecessarily complex) might be slightly better than just raw monkey patching. But this is very much something that happens across the boundaries of code that is distributed separately. If you are the maintainer for your class, just change your class, it's much simpler! Of course, there are perfectly good reasons to use the wonktacular insane dynamic features that Python offers you, but mostly those are useful when you're trying to abstract away some concern that the developer writing the code in question shouldn't have to worry about, like a proxy object. It's not a good idea to implement core application functionality in terms of meta-programming hacks: the whole point of adding funky meta-programming hacks is to make the core application logic easier to follow, by removing other concerns from the code :).

I hope this was somewhat comprehensible, and that there is an elegant solution out there. :)

Well, if my answer answers your question, then yes :).

Hi Einar, On Sep 8, 2010, at 6:19 AM, Einar S. Idsø wrote:

I am trying to make one class/service hook onto another class/service to add to its functionality without disturbing the original class/service.

So... you are trying to write a computer program, is what you're saying? ;-)

My situation is this: I currently have a class which at specific intervals queries one server per instance (via t.u.getProcessOutput()) for some basic information. Each instance has a method which analyses its returned data and stores the result it to ts internal data structure. My dilemma is that I wish to leave this class alone, but I also wish to be able to add functionality from external classes based on the result of said method. For instance: If the method analysing the result from getProcessOutput() discovers a certain user on the server being queried, I wish to store the data to a mysql database. Under other circumstances I wish to send a notification to myself over MSN or via mail. I can easily envision myself wanting to respond to the results with other actions as well in the future. How can this be achieved?

It sounds like this is a pretty trivial application of the Publish/Subscribe pattern (or perhaps the Observer) pattern. <http://en.wikipedia.org/wiki/Publish/subscribe> <http://en.wikipedia.org/wiki/Observer_pattern> In less fancy language, "it sounds like you need a 'for' loop".

I can of course modify the original class so that I pass an adbapi.ConnectionPool as optional argument to __init__() and modify the method to check for the existence of such an object before querying the database. I would then also need to add another optional argument which is whatever object needs to be passed to allow the class to send a message to MSN, and still more optional arguments for any such new service I'd like to add. However, as mentioned I'd rather not modify the original class, and doing this seems like too much of a hack. Or is it? Perhaps inheritance is the key, although that would require me to modify existing methods which I know to be working just to tack on some additional functionality at the end.

No, inheritance isn't the key. It never is - composition is the key :). You're right that adding lots of special-purpose arguments to __init__ that each do one thing is a bad idea. Don't do that. Add one general-purpose "list of observers that want to be notified (when X happens)" argument. Then when your getProcessOutput() finishes, just do 'for something in self.observers: something.youJustGotServerStatus(newServerStatus)'. It usually makes sense to define an interface with a couple of interesting methods when you do this, so that you can have more methods. If you've only ever got one interesting type of event for this class and you're sure you won't ever have more, you might just use callable objects instead of methods, since that will let users pass in functions (i.e. 'for observer in observers: observer()').

What are my options here? I took a look at t.p.hook, and it seems like it is perfect for the job: After having instantiated my "global" adbapi.ConnectionPool(), I can just add a pre- or post-hook to any class's method from which I'd like to store data. That way I can make a db-instance which hooks in to any part of my application and stores any result imaginable from any method. However, it seems that t.p.hook is very rarely used, so perhaps there is a better way of achieving what I want?

Please, please do not use t.p.hook. I hope that we delete it one day. The only thing in all of Twisted which uses t.p.hook is twisted.python.threadable, which is also very lightly used. This functionality could easily be implemented using a decorator instead of the awfulness that twisted.python.hook gets up to. If you look at how twisted.python.hook is implemented, you'll notice that it generates strings of Python code at runtime, and compiles and runs them. More generally, please don't try to do what t.p.hook tries to do, and add functionality to a class that you wrote without that class's cooperation. If your application depends on magical external modifications to classes in order to work as expected, it will become flaky and difficult to maintain. For one thing, imagine some new developer reading the code: they're looking at the body of the method, they're looking at the code of the method, and then they're looking at some code in some totally random other spot that is somehow getting invoked when they call it. Imagine the frustration as they try to figure out why that is happening. If you instead just have an explicit list of observers, which are passed in to the class, then if one of them blows up, it will be very easy to figure out where it's blowing up: the line will be observer.youJustGotServerStatus(), there will be a couple of youJustGotServerStatus method somewhere that you can track down bugs to. In some situations, if you're dealing with somebody else's code, that you can't modify for some reason at the source level (for example: you want to work with a released version of the standard library, or an older version of Twisted) it might make sense to patch in some extra functionality from your code. In that case, using something like twisted.python.hook that enforces a particular style (again: do not use twisted.python.hook itself, it is old, crufty code that is unnecessarily complex) might be slightly better than just raw monkey patching. But this is very much something that happens across the boundaries of code that is distributed separately. If you are the maintainer for your class, just change your class, it's much simpler! Of course, there are perfectly good reasons to use the wonktacular insane dynamic features that Python offers you, but mostly those are useful when you're trying to abstract away some concern that the developer writing the code in question shouldn't have to worry about, like a proxy object. It's not a good idea to implement core application functionality in terms of meta-programming hacks: the whole point of adding funky meta-programming hacks is to make the core application logic easier to follow, by removing other concerns from the code :).

I hope this was somewhat comprehensible, and that there is an elegant solution out there. :)

Well, if my answer answers your question, then yes :).