[Web-SIG] Web Site Process Bus

[Robert Brewer]

Chris McDonough wrote:
> There are also non-webbish processes like postgres, mysql, etc. that  
> need to be treated as "part of the application".
> 
> I handle this currently by running all of the processes related to a  
> specific project under a process controller (which happens to be  
> implemented in Python, but that's besides the point, see http:// 
> www.plope.com/software/supervisor2/).  The process controller is  
> responsible for execing the child processes upon its own startup.
> It is also responsible for restarting children if they die,
> capturing their output (if any), and allowing sufficiently
> privileged users to start and stop each one independently.
> The only promise a subprocess must make to be managed is that
> it must be possible to start the process "in the foreground"
> (not under its down daemon manager).
> 
> If a "process bus" is implemented I suspect it should be implemented  
> at this kind of level.

Ah, but there's the rub: we all have different ideas about how to
*implement* IPC and control. Which is why the WSPBus I outlined says
nothing about the mechanisms of message transport, RPC/IPC, or process
or thread boundaries. Instead, it defines the messages themselves, a set
of states for a given site, and a singleton message broker and state
machine. That's it.

If we can get that one piece into place, I think it can be a focal point
for interop between 1) application components, 2) HTTP servers, and 3)
"process controllers" (great term, I think I'll use it from now on). We
can achieve that without specifying how any process controller is
implemented, I think. It's difficult to discuss and reason about,
because CherryPy, Apache, Django, Zope, etcetera etcetera all provide
all three. A common bus should make it easier to decouple, say,
CherryPy's app+server from its process controller, allowing more people
to try out supervisor2 more easily.

> "Actions" could be registered for a specific subprocess types
> to send some input to a pipe file descriptor, send a signal to
> the process, etc.

Yes, and a supervisor2-wspb package could provide a Bus which does that,
then hand a reference to it to each child's components. The "reference"
in this case would be a child-side proxy object, which knows how to send
signals back to the parent process. But the children don't need to know
those transport details--all they have to do is call bus.subscribe,
bus.publish('log'), etc.

> It would also be possible to create some sort of dependency
> map between processes in a configuration, that relate the
> actions of one process to another (restart process A if
> process B is restarted, send a signal S to process C if
> signal T is sent to process D, etc).

Dependencies are a layer that can be built on top of the basic bus.
Since it's the process controller that's calling bus.start, stop and
restart, there's nothing about the WSPBus that stops supervisor2 from
handling dependency graphs on its own. If process A has to *know* that
process B has been restarted, that's a problem (which could be addressed
via custom bus channels), but if only the process controller has to
know, then there's no need to add that to the bus spec, IMO.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org