Best way to spawn process on back end computer

[Robin Becker]

sophie_newbie wrote:
> Hi,
> 
> I'm running a python cgi script on a frontend web server and I want it
> to spawn another script (that takes a long time to run) on a backend
> number crunching server thats connected to the same network. What do
> you think is the best way to do this? I have a few ideas but I'm sure
> there is a "best" way to go about this.


The main problem here is that you'll probably need to detach the job to allow 
the current cgi request to return a response to the client.

The implication of that is that the job either has to be anonymous and requires 
no further attention or you need to provide some means of making the job 
responsive to requests about its status so that a periodic request can be made 
by the web page. That implies that the job can be identified and the creation 
reponse returns the identity. One of the major problems is that the normal www 
user has few privileges and cannot normally write to disk.

I have done this using both external shell scripts to do the main processing and 
detaching and or python scripts that know how to detach. It was not terribly 
easy or obvious.

Another alternative, as Simon's Kamaelia might indicate, is that you might 
consider running a job server to service the cgi script requests on the remote 
host. I have also done this as part of a web application. One of the advantages 
was that the jobserver can run as any user and thus gets access to whatever the 
owner has; additionally by providing a suitable protocol eg  XMLRPC you can test 
the jobserver without going through the web.
-- 
Robin Becker