[Twisted-Python] Configuring Twisted logging via plugins
Hello, all. There have been a few questions on this list about how to configure Twisted logging via plugins, as opposed to .tac files. And we now know about the #638 and #3534 tickets. Hopefully, they will be resolved soon. But before this happens, it seems to me there's no other way to handle this than to use a "monkey patching" techinque on Twisted (excluding patching the Twisted source code itself). We need to access the 'application' object when the server starts. We can do this when the 'twistd' code calls setServiceParent on the service returned by the plugin. So if we wrap setServiceParent, we can access the application object and do what we want. See the code: http://gist.github.com/505926 This works for me. I don't know if a better and less dangerous way to achieve the same exists. Maybe there is one? Best Regards, Valeriy
On Aug 5, 2010, at 12:44 AM, Valeriy Zamarayev wrote:
This actually looks pretty ingenious to me! It uses public APIs... mostly. I'd have to think pretty hard about whether this particular interface is actually supported, of course, since you're depending on extra stuff from your service parent which hypothetically may not actually be there. If you're looking for aesthetic advice, it would be a bit nicer to use something like proxyForInterface and wrap the underlying service, rather than setting an attribute on it and clobbering its setServiceParent method. Also... you could write a twistd plugin that could be used directly, instead of code that needs to be adapted / re-written for each different twistd plugin. Your code can scan the IServiceMaker plugin registry just as well as twistd itself can. So you could have twistd my-custom-logger --my-log-option-1 --my-log-option-2 web --path ~/public_html and have the my-custom-logger plugin delegate to all twistd subcommands. That's a pattern which should remain useful even after #638 is fixed. Code to do this properly is left as an exercise for the reader, of course :).
On 05:25 pm, glyph@twistedmatrix.com wrote:
This is probably possible, but not using the twisted.python.usage options parser, which doesn't support multiple subcommands. This solution has been discussed on IRC before and probably the lack of multiple subcommand support in the options parser is the only reason it hasn't been implemented. I think there was some consensus on an alternate way to spell this combination, but I don't remember what it was right now. It's probably in someone's IRC logs. Jean-Paul
On Aug 5, 2010, at 20:25, Glyph Lefkowitz wrote:
The hack is based on a few assumptions: 1. the service parent for a plugin is the application. 2. setServiceParent will be called soon after plugin configuration. I don't know how stable and safe these assumptions are, just hope that by the time something changes, there will be an official way :)
Oh, yes. I'll change this, thanks.
Nice idea. I actually though about writing a single .tac which could configure logging and then find and start whatever plugin is given in the command line. But then I found that a .tac isn't even necessary. Given what Jean-Paul says about option parsing, it is not that straigtforward. I don't need (yet) to configure logging of standard or third party plugins, only my own. So if the goal is just to reuse the code and not do the same with every plugin, it's easy to use a template-method technique, and reuse the hack code across all the plugins (The example in the gist is a bit artificial). Regards, Valeriy
On Aug 5, 2010, at 12:44 AM, Valeriy Zamarayev wrote:
This actually looks pretty ingenious to me! It uses public APIs... mostly. I'd have to think pretty hard about whether this particular interface is actually supported, of course, since you're depending on extra stuff from your service parent which hypothetically may not actually be there. If you're looking for aesthetic advice, it would be a bit nicer to use something like proxyForInterface and wrap the underlying service, rather than setting an attribute on it and clobbering its setServiceParent method. Also... you could write a twistd plugin that could be used directly, instead of code that needs to be adapted / re-written for each different twistd plugin. Your code can scan the IServiceMaker plugin registry just as well as twistd itself can. So you could have twistd my-custom-logger --my-log-option-1 --my-log-option-2 web --path ~/public_html and have the my-custom-logger plugin delegate to all twistd subcommands. That's a pattern which should remain useful even after #638 is fixed. Code to do this properly is left as an exercise for the reader, of course :).
On 05:25 pm, glyph@twistedmatrix.com wrote:
This is probably possible, but not using the twisted.python.usage options parser, which doesn't support multiple subcommands. This solution has been discussed on IRC before and probably the lack of multiple subcommand support in the options parser is the only reason it hasn't been implemented. I think there was some consensus on an alternate way to spell this combination, but I don't remember what it was right now. It's probably in someone's IRC logs. Jean-Paul
On Aug 5, 2010, at 20:25, Glyph Lefkowitz wrote:
The hack is based on a few assumptions: 1. the service parent for a plugin is the application. 2. setServiceParent will be called soon after plugin configuration. I don't know how stable and safe these assumptions are, just hope that by the time something changes, there will be an official way :)
Oh, yes. I'll change this, thanks.
Nice idea. I actually though about writing a single .tac which could configure logging and then find and start whatever plugin is given in the command line. But then I found that a .tac isn't even necessary. Given what Jean-Paul says about option parsing, it is not that straigtforward. I don't need (yet) to configure logging of standard or third party plugins, only my own. So if the goal is just to reuse the code and not do the same with every plugin, it's easy to use a template-method technique, and reuse the hack code across all the plugins (The example in the gist is a bit artificial). Regards, Valeriy
participants (3)
-
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Valeriy Zamarayev