One idea would be to put a spam-style filter on the inbound invite-confirm mail processor, so that phrases like "out of the office" could be caught.

Thanks to Barry for moving the sourceforge tracker to the new list!