[Bug 1685946] [NEW] Mismatch between declared and actual encoding in mail notifications and web pages

[Johannes Rohr]

Public bug reported:

This is about 2.1.22 in Ubuntu Yakkety. mailman is configured with
German (de) as default language and the templates for German and English
are generated.

For some reason, the notifications and web pages in German are all
encoded in latin-1, while the declared encoding both in mail
notification as well as in web pages served is utf-8. This result in
broken display of umlauts until one manually changes the encoding.

See https://lists.uferwerk.org/cgi-bin/mailman/listinfo/

I have tried to manually enforce UTF-8 by adding

 add_language('en', 'English (USA)', 'utf-8', 'ltr')
 add_language('de', 'Deutsch', 'utf-8', 'ltr')
 
We see this misbehaviour since we migrated mailman from one server running Ubuntu 14.4 with mailman  1:2.1.14-3ubuntu0.4 to another server running 16.10 with mailman  1:2.1.22-1ubuntu0.1

On the old server, it worked just fine without any additional tweaking
and on the new server, not tweaking seems to change anything at all.

In the mailman error log I see recurring error messages of the sort:

  Apr 24 15:10:19 2017 (898) list: mitglieder: can't decode "Liste f�r Mitglieder  der Uferwerk e.G." as utf-8
 Apr 24 15:10:19 2017 (901) list: mitglieder: can't decode "Liste f�r Mitglieder  der Uferwerk e.G." as utf-8
 Apr 24 15:10:20 2017 (901) list: mitglieder: can't decode "Liste f�r Mitglieder  der Uferwerk e.G." as utf-8
 
I am not sure why these descriptions were suddenly in latin-1 but I manually converted them to utf-8 now, restarted mailman, and the situation remains unchanged: Both the overview page and the mail notifications are in latin-1 while the headers declare that they are in utf-8, which results in broken rendering of the text

** Affects: mailman
     Importance: Undecided
         Status: New

** Affects: mailman (Ubuntu)
     Importance: Undecided
         Status: New

** Also affects: mailman (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Mailman
Coders, which is subscribed to GNU Mailman.
https://bugs.launchpad.net/bugs/1685946

Title:
  Mismatch between declared and actual encoding in mail notifications
  and web pages

To manage notifications about this bug go to:
https://bugs.launchpad.net/mailman/+bug/1685946/+subscriptions