multiple instance on Unix
zanesdad at bellsouth.net
Thu Sep 30 14:32:55 CEST 2004
Nigel King wrote:
> I have not explained very well what I wanted.
> I had a program that was called randomly by specific emails arriving
> which asked for certain information. If two or more emails arrived
> simultaneously then procmail asked two or more instances of my program
> to run. These instances interfered with one another, so I needed a
> process to stop that from happening. What my son devised was for the
> first to create a directory and run and when finished to delete the
> directory. The subsequent instances could try to create a directory
> but fail in an atomic piece of code. They would sleep for 1 or more
> seconds and then try again. The first of the subsequent instances that
> tries and succeeds stops the others from succeeding.
> Now, this works but I wondered whether anybody knew of a more standard
> bit of python code was available for ensuring that only one instance
> was processing. mutex does it for threads but not for instances as I
> understand it.
> The specification for a better process would include the ability to
> ensure that the queue was orderly, in other words some sort of FIFO
> would ensure that first served would have been the first to request
> the lock and fail.
> Our solution which does not satisfy the previous paragraph.
> import os, time
> # This program is not thread safe so we must protect it from being
> # trampled over by another copy
> # pause if another email is being processed for half an hour maximum
> t = time.time()+1800
> locked = True
> while locked and time.time() < t:
> locked = False
> except :
> # do everything else
> finally :
> # Removes the thread locking device so that another copy may run
> The timer was in case for any reason finally did not run successfully
> Facundo's solution I have not yet studied.
So, basically you have a working solution. Couple of things, though.
You may want to modify your
if not Locked:
otherwise, if one of them times out, it's going to delete that directory.
Also, you probably want to tighten that except statement down to "except
OSError" so that you are handling the only exception that should be
raised right in that block of code. If something goes batty and raises
another type of exception, you probably want to fall into your filly
condition, log it (are you logging with this application?), then let
finally raise the exception up.
This last one is just a matter of preference, but you could remove most
of those "pass" statements. They're not hurting anything, but it's just
a little cluttered.
OK - not I know that you weren't looking for comments on your code, but
you got it for free, right? ;-) Anyway, to your question. Is there a
better way of doing this than the way you are doing it? Probably, but I
don't know for sure what a good answer is. Frankly, this approach both
scares the pants off of me and gives me an upset stomach. I'm guessing
you must be feeling at least a little of that sentiment or you wouldn't
be asking if there's another way to do this. I've just seen this type
of thing abused and lead to all kinds of weird and undiagnosable
problems. When you're using the filesystem as either a locking
mechanism or as a source of state information, that just smells like the
wrong solution to me, but I'm probably biased against it from experience.
So, what are the alternatives. This may be way over kill, but maybe you
could have procmail kick off a web-services-ish script and talk to a
centralized server process (either XMLRPC or SOAP or something like
that). Then you could put the locking in the server process and not
have to deal with it from the procmail-spawned scripts. And, actually,
if you used something like SimpleXMLRPCServer, you wouldn't have to do a
thing. The first request would be handled and processed, while the
second and subsequent requests would block until the first (or next in
the case of subsequent requests) request finished. But, how do you make
sure you've only got one XMLRPCServer running? ;-)
Another option is a recipe that is on the ASPN Python Cookbook site for
this type of thing:
But at a glance it looked like more of the same of what you've already done.
Another alternative is to use the builtin procmail locking as others
have suggested. I googled on it and the most frequent recurring word
pattern was "procmail lock failure", so I'd be a little afraid of that.
But it'd be worth looking into nonetheless.
I don't know if in *NIX OSes if you can somehow register an instance of
a process and make any subsequent registration block execution until the
running one terminates. That would be ideal.
Anyway, these are my thoughts.
More information about the Python-list