[Mailman-Developers] [Bug] 2x decode_header? [was: No subscriptionis made ...]

[Mark Sapiro]

Mark Sapiro wrote:
> Stephen J. Turnbull wrote:
> 
>> Moving to -developers, reply-to set.  Please keep Henrik in the loop.
>>
>> Seems to be the same as
>> http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103
> 
> 
> I do not think it is the same bug. Note that bug 1186819 appears to be
> due to non-ascii in the user's real name and occurs during processing
> of a web subscribe request, although it may well also occur in
> processing an email subscribe request too since it comes in
> mlist.AddMember.
> 
> The current bug occurs in an email confirmation only and has to do with
> non-ascii in the message body.
> 
> Here's my analysis of the current bug as posted in the mailman-users
> thread:
> 
> <quote>
> I looked more closely at the traceback, and I think we have a bug. What
> happened here is the confirmation was received and processed and the
> subscription was confirmed. Then there is a bit at the end of
> cmd_confirm.py that 'eats up' the rest of the message and makes a list
> of 'unprocessed' lines. It is in this loop that the exception occurs.
> The loop goes through the lines skipping any that match the 'confirm
> <token>' command just processed, since adding such a line to the
> 'unprocessed' lines would be confusing.
> 
> In this case, the processed confirm command was in the subject, and
> because of the way we handle subjects, it was a Unicode string. Thus,
> when we looped through the rest of the lines doing
> 
>     if line.lstrip() == match:
> 
> with match being a Unicode string, line.lstrip() was assumed ascii and
> coerced to Unicode for the comparison. This threw the exception when
> it got to the signature line with a non-ascii character, and even
> though the subscription was confirmed, the exception caused the saving
> of the updated list to be skipped and the new member was lost.
> </quote>
> 
> I already have a fix for the current bug. I just haven't tested it yet.
> 
> --- test-mailman-2.1/Mailman/Commands/cmd_confirm.py
> +++ test-mailman/Mailman/Commands/cmd_confirm.py
> @@ -90,8 +90,11 @@
>              match = 'confirm ' + cookie
>              unprocessed = []
>              for line in res.commands:
> -                if line.lstrip() == match:
> -                    continue
> +                try:
> +                    if line.lstrip() == match:
> +                        continue
> +                except UnicodeError:
> +                    pass
>                  unprocessed.append(line)
>              res.commands = unprocessed
>      # Process just one confirmation string per message
> 
> I will also look into bug 1186819.
> 

I am unable to reproduce either bug in Mailman 2.1.10b3 with Python 
2.5.1, however I heard from Henrik that the above patch fixed his 
problem, and I have committed that patch to the 2.1 branch.

I added a note to 
<http://sourceforge.net/tracker/index.php?func=detail&aid=1186819&group_id=103&atid=100103> 
saying I can't reproduce it and that it may have been fixed by 
<http://sourceforge.net/tracker/index.php?func=detail&aid=1235567&group_id=103&atid=300103> 
and asking for feedback if it is still a current problem.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan