-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Mark,
I'm sorry I haven't been able to respond to this thread before now,
but I've been traveling and PyCon'ing and either haven't had time or
have had spotty access to email.
On Feb 22, 2007, at 5:24 PM, Mark Sapiro wrote:
The question I would like to discuss, is what is the best way to preserve the queue entry for analysis. The patch just renames the .bak file to .psv and leaves it in the original queue. This could potentially over time accumulate a lot of .psv files in the 'in' or other queues and impact processing.
We can't shunt the entry in the normal way because in some cases at least, shunting has already thrown an exception. I can think of three things to do.
- Just rename the entry .psv and leave it in the original queue.
- Rename it .psv and attempt to move it to the shunt queue.
- Rename it .psv and attempt to move it to the bad queue.
I'm pretty sure we don't want to leave the .psv file in the original
queue. I agree that over time, we'll just end up filling up the
queue directory with files we'll never process, wasting time listing
them, or worse. A major reason for having the shunt queue in the
first place was to not clutter up our processing queues with messages
we couldn't do anything about. Of course, culling our shunt queue is
another issues. :/
OTOH, if we're going to move the offending message to the shunt
queue, then there's not much point in keeping the .psv extension. We
pretty much always know that if the file is in shunt, it's bad, so
maybe we just rename the offender to shunt/blah.txt (assuming it's an
unparsed text file).
The other thing to consider is adding a configuration variable that
let's us limit the size of the files we'll handle. I can't imagine
any scenario under which we'd want to (let alone be able to) handle a
message of a half terabyte. Heck, you have to have a pretty
misconfigured mail system to even allow such a message to get to
Mailman, IMO. I think Postfix for example has a 10MB default size
limit, and even cranking that up by a factor of 10 should allow most
legitimate mail to go through.
So, if we added a size limit configuration variable, we'd have to
stat the file (os.path.getsize()) and just os.rename() the file to
shunt (with a log message) when we see a file over that size limit.
What do you think?
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBReJo43EjvBPtnXfVAQKw8QP/Qm2twCuIkyQV4zXt1+Ei9PKoBpdBSiK9 G/LgwlUz8OCBRkonb+m4NTyMmZJTQUjqcLICvgFzZ9d+dxf5ekVA8KoXM1iJjjnE TQqS88CAgUYcWixkIfS5H1l0lVSKSGsUn8eUAXwmoVZVn3NaqCzhTo1wGV56KATc VesBGYhpGkQ= =juvu -----END PGP SIGNATURE-----