[Bug] 2x decode_header? [was: No subscription is made ...]
Moving to -developers, reply-to set. Please keep Henrik in the loop.
Seems to be the same as http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103
Henrik Rasmussen writes:
Stephen J. Turnbull wrote:
Henrik Rasmussen writes: Of course it's possible that the MUA is just inserting those headers but it's a lie (the MUA thinks that's enough to let it get away with just putting UTF-8 *anywhere*). But note that 0xF8 is illegal in UTF-8 (for several reasons), so most likely that is intended as an ISO 8-bit character. Do you agree that the interpretation of 0xF8 as "ø" is likely here?
I'm not sure. The mail body contains a signature "Nørregaard" in the plain text section and "N=C3=B8rregaard" in the HTML section, but there is no "ø" in the header (and thus nor the e-mail address of the subscriber).
This sounds like a bug in Mailman; if the headers do not contain any characters outside of ASCII, then for some reason Mailman (or the email module that it calls) is trying to decode the headers twice.
Can you provide a verbatim copy of the original message? (If privacy is a problem, send it to Mark Sapiro <mark@sapiro.net>. Mark is the likely person to actually do any fixing; I know how this stuff works in principle but haven't yet actually hacked the core code.) The whole thing, with no changes at all, please (you never know what matters in these things, even changing the .sig could make the bug go away!)
But even so, the fact that Mailman issued a 'new' command, the user should have been put on the list (i'm just speculating here, I'm not that familiar with e-mail handeling and especially not Mailman nor Python).
"Issue" a command and "complete" a command are two different things. I've already archived the earlier posts so I can't evaluate your conjecture offhand.
Stephen J. Turnbull wrote:
Moving to -developers, reply-to set. Please keep Henrik in the loop.
Seems to be the same as http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103
I do not think it is the same bug. Note that bug 1186819 appears to be due to non-ascii in the user's real name and occurs during processing of a web subscribe request, although it may well also occur in processing an email subscribe request too since it comes in mlist.AddMember. The current bug occurs in an email confirmation only and has to do with non-ascii in the message body. Here's my analysis of the current bug as posted in the mailman-users thread: <quote> I looked more closely at the traceback, and I think we have a bug. What happened here is the confirmation was received and processed and the subscription was confirmed. Then there is a bit at the end of cmd_confirm.py that 'eats up' the rest of the message and makes a list of 'unprocessed' lines. It is in this loop that the exception occurs. The loop goes through the lines skipping any that match the 'confirm <token>' command just processed, since adding such a line to the 'unprocessed' lines would be confusing. In this case, the processed confirm command was in the subject, and because of the way we handle subjects, it was a Unicode string. Thus, when we looped through the rest of the lines doing if line.lstrip() == match: with match being a Unicode string, line.lstrip() was assumed ascii and coerced to Unicode for the comparison. This threw the exception when it got to the signature line with a non-ascii character, and even though the subscription was confirmed, the exception caused the saving of the updated list to be skipped and the new member was lost. </quote> I already have a fix for the current bug. I just haven't tested it yet. --- test-mailman-2.1/Mailman/Commands/cmd_confirm.py +++ test-mailman/Mailman/Commands/cmd_confirm.py @@ -90,8 +90,11 @@ match = 'confirm ' + cookie unprocessed = [] for line in res.commands: - if line.lstrip() == match: - continue + try: + if line.lstrip() == match: + continue + except UnicodeError: + pass unprocessed.append(line) res.commands = unprocessed # Process just one confirmation string per message I will also look into bug 1186819. -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Sorry for posting this to the developers list but it's a little more involved then a standard usability issue.
We have 2 mailman servers that we run in a cluster sharing a san volume for archives, data, lists, logs, qfiles. Orginally we didn't have it sharing the data directory but we ran into problems with moderated lists and having the web administration not seeing the moderated messages on the other node.
So we shared the data directory and it solved this problem. But we are running into issues with master-queuerunner.pid file being deleted when mailman is stopped or restarted causing us to have to recreate this file. Obviously we shouldn't be sharing the same pid file for 2 nodes.
My question is where in the configs can I change this filename so that each node can have a unique pid file?
Jeff Kunzelman (DHL) wrote:
So we shared the data directory and it solved this problem. But we are running into issues with master-queuerunner.pid file being deleted when mailman is stopped or restarted causing us to have to recreate this file. Obviously we shouldn't be sharing the same pid file for 2 nodes.
My question is where in the configs can I change this filename so that each node can have a unique pid file?
The file is defined in Defaults.py and can be overridden in mm_cfg.py just like anything else. The default is
PIDFILE = os.path.join(DATA_DIR, 'master-qrunner.pid')
You can redefine this in each mm_cfg.py to be a unique name for each copy.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Stephen J. Turnbull wrote:
Moving to -developers, reply-to set. Please keep Henrik in the loop.
Seems to be the same as http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103
I do not think it is the same bug. Note that bug 1186819 appears to be due to non-ascii in the user's real name and occurs during processing of a web subscribe request, although it may well also occur in processing an email subscribe request too since it comes in mlist.AddMember.
The current bug occurs in an email confirmation only and has to do with non-ascii in the message body.
Here's my analysis of the current bug as posted in the mailman-users thread:
<quote> I looked more closely at the traceback, and I think we have a bug. What happened here is the confirmation was received and processed and the subscription was confirmed. Then there is a bit at the end of cmd_confirm.py that 'eats up' the rest of the message and makes a list of 'unprocessed' lines. It is in this loop that the exception occurs. The loop goes through the lines skipping any that match the 'confirm <token>' command just processed, since adding such a line to the 'unprocessed' lines would be confusing.
In this case, the processed confirm command was in the subject, and because of the way we handle subjects, it was a Unicode string. Thus, when we looped through the rest of the lines doing
if line.lstrip() == match:
with match being a Unicode string, line.lstrip() was assumed ascii and coerced to Unicode for the comparison. This threw the exception when it got to the signature line with a non-ascii character, and even though the subscription was confirmed, the exception caused the saving of the updated list to be skipped and the new member was lost. </quote>
I already have a fix for the current bug. I just haven't tested it yet.
--- test-mailman-2.1/Mailman/Commands/cmd_confirm.py +++ test-mailman/Mailman/Commands/cmd_confirm.py @@ -90,8 +90,11 @@ match = 'confirm ' + cookie unprocessed = [] for line in res.commands: - if line.lstrip() == match: - continue + try: + if line.lstrip() == match: + continue + except UnicodeError: + pass unprocessed.append(line) res.commands = unprocessed # Process just one confirmation string per message
I will also look into bug 1186819.
I am unable to reproduce either bug in Mailman 2.1.10b3 with Python 2.5.1, however I heard from Henrik that the above patch fixed his problem, and I have committed that patch to the 2.1 branch. I added a note to <http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103> saying I can't reproduce it and that it may have been fixed by <http://sourceforge.net/tracker/index.php?func=detail&aid=1235567&group_id=103&atid=300103> and asking for feedback if it is still a current problem. -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Jeff Kunzelman (DHL) -
Mark Sapiro -
Stephen J. Turnbull