Threads CGI & database connections...
ianb at colorstudy.com
Tue Oct 7 21:58:59 CEST 2003
On Tuesday, October 7, 2003, at 10:23 AM, Chris Lyon wrote:
> We run a sound effects library site.
> I use a cgi script written in python to copy files from our file
> repository to a publically viewable directory for clients to pick up
> with locations sent to them in E-mails and on a completion web page.
> Unfortunately since some of the files are fairly large (20-30 Mbytes
> is not untypical) it can take a considerable length of time to copy
> the files across.
> So we would like to implement something that would allow us to quickly
> report back that the process has been started and then copy the files
> page to indicate the progress.
> The status page extracts it's information from the database where a
> progress column is populated ( 1 to indicate copy started and 2 to
> indicate copy completed).
> Now this seems to mean threads( it's all hosted on windows2000 ) an
> area I have little ( read none) experience of. I believe I will have
> to maintain a database cursor within the active thread to indicate the
> progress to the database.
CGI scripts are run in a single process, so threads won't work for you.
The response hasn't ended until the process quits. So since you want
to send multiple responses (each time the page is reloaded, to give the
progress as another response) you will have multiple processes for each
Which is to say, when you first run the CGI script that starts the
copying process, that CGI script has to terminate before the copying is
finished. You will have to somehow spawn another process to actually
do the copying. But you have that Windows problem, since Windows
doesn't have fork. There may be some equivalent way to do this, I
don't know -- someone who knows Windows better may be able to answer.
Another alternative to spawning a process is to have some long-running
process waiting around to copy files for people. If you are using a
database you could, for instance, have field that means
please-copy-this-file. This long-running process polls the database,
and when it finds a row with that field set it starts copying (setting
that field to a new value, copying-in-progress). This is what I'd
recommend, as it is pretty straight-forward and translates to different
environments fairly easily. This polling process could use threads or
not, depending on whether you want to copy files in parallel or
serially. (The polling process won't be hooked into the web server in
any way, it'll be a separate program you run on the server.)
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org
More information about the Python-list