[Twisted-Python] Could Service.startService return a Deferred?
I have a Twisted application with several Services. One of them has a startService method that does some setup asynchronously. This can go wrong if the next service to start makes a call to the first one, and it isn't ready yet. One way round it would be to make all methods on the slow service returned a Deferred (e.g. "self.slow_init.addCallback(self.actual_method)", but it seems a shame to have to do that every time when the situation only occurs when the daemon starts up. So would it be possible for t.a.s.Service.startService to be allowed to return a Deferred? Then the next service would only be started up when the Deferred fired. Peter.
Peter,
I assume you're using a MultiService to hold all of these services.
You could delay adding the dependent services to the MultiService
until the setup service has finished doing what it does.
For instance, something like this (untested):
class SetupService(Service):
def __init__(self):
self.done = defer.Deferred()
def startService(self):
# do things
self.done.callback(None)
def makeService(...):
ms = MultiService()
setup = SetupService()
setup.done.addCallback(addTheOtherServicesTo, ms)
Or maybe there's a better built-in way to do this that I don't know about.
Matt
On Wed, Nov 27, 2013 at 7:58 AM, Peter Westlake
I have a Twisted application with several Services. One of them has a startService method that does some setup asynchronously. This can go wrong if the next service to start makes a call to the first one, and it isn't ready yet. One way round it would be to make all methods on the slow service returned a Deferred (e.g. "self.slow_init.addCallback(self.actual_method)", but it seems a shame to have to do that every time when the situation only occurs when the daemon starts up. So would it be possible for t.a.s.Service.startService to be allowed to return a Deferred? Then the next service would only be started up when the Deferred fired.
Peter.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 02:58 pm, peter.westlake@pobox.com wrote:
I have a Twisted application with several Services. One of them has a startService method that does some setup asynchronously. This can go wrong if the next service to start makes a call to the first one, and it isn't ready yet. One way round it would be to make all methods on the slow service returned a Deferred (e.g. "self.slow_init.addCallback(self.actual_method)", but it seems a shame to have to do that every time when the situation only occurs when the daemon starts up. So would it be possible for t.a.s.Service.startService to be allowed to return a Deferred? Then the next service would only be started up when the Deferred fired.
Probably not. There is some discussion on https://twistedmatrix.com/trac/ticket/5941. Jean-Paul
On Wed, Nov 27, 2013, at 17:13, exarkun@twistedmatrix.com wrote:
On 02:58 pm, peter.westlake@pobox.com wrote: ... So would it be possible for
t.a.s.Service.startService to be allowed to return a Deferred? Then the next service would only be started up when the Deferred fired.
Probably not.
There is some discussion on https://twistedmatrix.com/trac/ticket/5941.
That's helpful, thanks. I hadn't realized that startService was called before the reactor started. Instead, I've passed in a Deferred to each service that needs to wait. How does this look? # -*- mode: python -*- from twisted.application import service from twisted.internet import defer, reactor from twisted.internet.task import deferLater application = service.Application('Dependencies') def report(error): print 'ERROR', error reactor.stop() class Runner(service.Service): def __init__(self, baton): self.baton = baton def startService(self): print 'startService', self.name, reactor.running self.baton.addCallback(lambda ignore:deferLater(reactor, 1.0, self.realStartService)) self.baton.addErrback(report) def realStartService(self): print 'realStartService', self.name, reactor.running baton = defer.succeed('pass me to each service') foo = Runner(baton) foo.setName('foo') foo.setServiceParent(application) bar = Runner(baton) bar.setName('bar') bar.setServiceParent(application) Peter.
On 06:00 pm, peter.westlake@pobox.com wrote:
On Wed, Nov 27, 2013, at 17:13, exarkun@twistedmatrix.com wrote:
On 02:58 pm, peter.westlake@pobox.com wrote: ... So would it be possible for
t.a.s.Service.startService to be allowed to return a Deferred? Then the next service would only be started up when the Deferred fired.
Probably not.
There is some discussion on https://twistedmatrix.com/trac/ticket/5941.
That's helpful, thanks. I hadn't realized that startService was called before the reactor started.
Instead, I've passed in a Deferred to each service that needs to wait. How does this look?
# -*- mode: python -*- from twisted.application import service from twisted.internet import defer, reactor from twisted.internet.task import deferLater
application = service.Application('Dependencies')
def report(error): print 'ERROR', error reactor.stop()
class Runner(service.Service): def __init__(self, baton): self.baton = baton
def startService(self): print 'startService', self.name, reactor.running self.baton.addCallback(lambda ignore:deferLater(reactor, 1.0, self.realStartService)) self.baton.addErrback(report)
I'd be concerned with the state of `Runner` at this point in the process. What happens if the application gets shut down while that `deferLater` is still pending? `stopService` has a harder job because it might need to deal with a service that is partially initialized or a service that is completely initialized (and "partially initialized" may cover a multitude of different states depending on the complexity of your service).
def realStartService(self): print 'realStartService', self.name, reactor.running
baton = defer.succeed('pass me to each service')
foo = Runner(baton) foo.setName('foo') foo.setServiceParent(application)
bar = Runner(baton) bar.setName('bar') bar.setServiceParent(application)
What about something like this instead? @implementer(IService) class Runner(object): ... @classmethod def loadFromWhatever(cls, name): return deferLater(reactor, Runner, name) def __init__(self, name): self.name = name def startService(self): self.running = True print 'realStartService', self.name, reactor.running def parent(service): application.setServiceParent(service) loading = Runner.initializeFromWhatever("foo") loading.addCallback(parent) loading.addCallback(lambda ignored: Runner.initializeFromWhatever("bar")) loading.addCallback(parent) loading.addErrback(stopTheReactorOrWhatever) The advantage I see of this approach is that a `Runner` never exists in the service hierarchy until it is fully initialized, started, and running. If a `Runner` is only partially ready and the process shuts down then its `stopService` method isn't called because it's not part of the service hierarchy. I could definitely imagine a library to help with this kind of thing. For example, perhaps you want the above encapsulated as: asynchronouslySetUpServices(application, [ lambda: Runner.initializeFromWhatever("foo"), lambda: Runner.initialifrFromWhatever("bar")]) And maybe then you want to add in some logic so that if the application gets shut down while some things are still being initialized then you cancel their Deferred. Then you have good cleanup support for the uninitialized case - without complicating `stopService` (the cleanup logic is isolated in the implementation of Deferred cancellation where it belongs - eg, with this `deferLater`-based asynchronousness it's alreay present since deferLater implements cancellation already). Jean-Paul
On Wed, Nov 27, 2013, at 19:20, exarkun@twistedmatrix.com wrote:
On 06:00 pm, peter.westlake@pobox.com wrote: ...
class Runner(service.Service): def __init__(self, baton): self.baton = baton
def startService(self): print 'startService', self.name, reactor.running self.baton.addCallback(lambda ignore:deferLater(reactor, 1.0, self.realStartService)) self.baton.addErrback(report)
I'd be concerned with the state of `Runner` at this point in the process.
What happens if the application gets shut down while that `deferLater` is still pending?
`stopService` has a harder job because it might need to deal with a service that is partially initialized or a service that is completely initialized (and "partially initialized" may cover a multitude of different states depending on the complexity of your service).
...
What about something like this instead?
@implementer(IService) class Runner(object): ... @classmethod def loadFromWhatever(cls, name): return deferLater(reactor, Runner, name)
def __init__(self, name): self.name = name
def startService(self): self.running = True print 'realStartService', self.name, reactor.running
def parent(service): application.setServiceParent(service)
loading = Runner.initializeFromWhatever("foo") loading.addCallback(parent) loading.addCallback(lambda ignored: Runner.initializeFromWhatever("bar")) loading.addCallback(parent) loading.addErrback(stopTheReactorOrWhatever)
The advantage I see of this approach is that a `Runner` never exists in the service hierarchy until it is fully initialized, started, and running.
If a `Runner` is only partially ready and the process shuts down then its `stopService` method isn't called because it's not part of the service hierarchy.
I'll do that, thank you! Could the documentation for Service say something about which methods can be called when? For instance, it would never have occurred to me that setServiceParent could be called after control had passed out of the .tac file and the reactor had started running. I see from the source that it calls startService, but this is definitely the sort of non-obvious tip that it would be helpful to have written down. Likewise the fact that startService normally runs before the reactor starts. Out of interest, was there a reason for not making Runner a subclass of Service? There are some methods of IService that this version doesn't implement.
I could definitely imagine a library to help with this kind of thing. For example, perhaps you want the above encapsulated as:
asynchronouslySetUpServices(application, [ lambda: Runner.initializeFromWhatever("foo"), lambda: Runner.initialifrFromWhatever("bar")])
And maybe then you want to add in some logic so that if the application gets shut down while some things are still being initialized then you cancel their Deferred. Then you have good cleanup support for the uninitialized case - without complicating `stopService` (the cleanup logic is isolated in the implementation of Deferred cancellation where it belongs - eg, with this `deferLater`-based asynchronousness it's alreay present since deferLater implements cancellation already).
I'll add it to my list of things to do one of these years :-) Would you like to put in a ticket with a spec? Thanks for the help, Peter.
This is kind of an ongoing bugaboo for me, too. Buildbot uses services *heavily*, and at runtime (to support reconfigs, etc.). We've had to work around issues like stop being called before asynchronous starts complete using DeferredLocks and other such fun. I filed a bug with some of my issues - https://twistedmatrix.com/trac/ticket/6813 It'd be great to fix some of this in the existing implementation, but #4366 seems to demonstrate that this is impossible without breaking compatibility. So, maybe we need to introduce a new service hierarchy -- perhaps with some ability to use the existing Service classes as child services. I'd be interested to hear thoughts on the matter. Dustin
On Wed, Nov 27, 2013, at 19:20, exarkun@twistedmatrix.com wrote: ...
What about something like this instead?
@implementer(IService) class Runner(object): ... @classmethod def loadFromWhatever(cls, name): return deferLater(reactor, Runner, name)
def __init__(self, name): self.name = name
def startService(self): self.running = True print 'realStartService', self.name, reactor.running
def parent(service): application.setServiceParent(service)
A small detail: should this be service.setServiceParent(application)? Peter.
loading = Runner.initializeFromWhatever("foo") loading.addCallback(parent) loading.addCallback(lambda ignored: Runner.initializeFromWhatever("bar")) loading.addCallback(parent) loading.addErrback(stopTheReactorOrWhatever)
The advantage I see of this approach is that a `Runner` never exists in the service hierarchy until it is fully initialized, started, and running.
If a `Runner` is only partially ready and the process shuts down then its `stopService` method isn't called because it's not part of the service hierarchy.
I could definitely imagine a library to help with this kind of thing. For example, perhaps you want the above encapsulated as:
asynchronouslySetUpServices(application, [ lambda: Runner.initializeFromWhatever("foo"), lambda: Runner.initialifrFromWhatever("bar")])
And maybe then you want to add in some logic so that if the application gets shut down while some things are still being initialized then you cancel their Deferred. Then you have good cleanup support for the uninitialized case - without complicating `stopService` (the cleanup logic is isolated in the implementation of Deferred cancellation where it belongs - eg, with this `deferLater`-based asynchronousness it's alreay present since deferLater implements cancellation already).
Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (4)
-
Dustin J. Mitchell -
exarkun＠twistedmatrix.com -
Matt Haggard -
Peter Westlake