
On Tue, 2012-11-06 at 18:02:27 -0800, Mark Sapiro wrote:
Sahil Tandon wrote: ...
Obviously, this would result in a lot of pickles that are constantly opened, edited (and, periodically cleansed), and closed. Is the performance cost/benefit prohibitive?
Whether the cost is prohibitive depends on how many messages per minute, hour, day, etc you process through the list. I think it could work. The 'in-memory dictionary would also work as long as you are running with the default single qrunner per queue except for the rare case where the duplicates are processed one on each side of a restart.
Note as an implementation for the file name (path) derived from the list's internal_name, I would just use a fixed file name, e.g., message-ids.pck in the existing lists/internal_name()/ directory.
Thanks; this custom handler is now in testing and working well so far. I appreciate your help.
-- Sahil Tandon