[moin-user] ListPages macro doesn't work

[Volker Wysk]

Thanks for your detailed message!

I've made a little test data. When I do "locale.setlocale(locale.LC_ALL, "")" 
and ".sort(cmp=locale.strcoll)" at the beginning of the macro, I get this:

    Übb
    Aaaa
    Bbbb
    Ciao
    Vbbb
    c't

The umlaut is at the beginning, and capital letters come first.

With "locale.setlocale(locale.LC_COLLATE, "de_DE.UTF-8")" and 
".sort(cmp=locale.strcoll)", I get:

    Aaaa
    Bbbb
    c't
    Übb
    Ciao
    Vbbb
    
The "Ü" gets sorted betwenn the two "c"/"C"! It's weird.


Am Montag, 4. September 2017, 12:47:44 CEST schrieb Paul Boddie:

> I'm not sure whether I really understand locales better, but here are a few
> things that might help. Firstly, you can get the default locale as follows:
> 
> import locale
> locale.setlocale(locale.LC_ALL, "") # returns the locale string
> 
> This has to be done to make the process's locale information available. It
> is possible that something does this already in Moin, but as mentioned
> before, it is questionable that the process's locale is relevant for a user
> of a Web application. Now you can get the locale details more conveniently.
> 
> For example, to ask for the collation:
> 
> language, charset = locale.getlocale(locale.LC_COLLATE)
> 
> I would think that the collation is the most pertinent locale setting when
> it comes to sorting things. So, it might be more interesting to set this
> based on any details about the user provided by Moin. The
> MoinMoin.user.User object has a language attribute that could work in
> principle, but I'm not convinced that this is enough by itself. More on
> that in a moment.
> 


> Anyway, you can set the collation as follows:
> 
> locale.setlocale(locale.LC_COLLATE, "no_NO") # something I just tested
> 
> And you can apply the locale sorting as follows:
> 
> names.sort(cmp=locale.strcoll)

That's what I've done:
locale.setlocale(locale.LC_COLLATE, "de_DE.UTF-8")

The "de_DE.UTF-8" part seems to be necessary.


> 
> This will correctly sort a sequence of names where Norwegian letters are
> used. It seems that Unicode will work, too.
> 
> Why I don't think the Moin language code is enough is that the locale system
> is rather particular about what you ask it for. However, it seems that you
> can get a proper locale from the Moin language as follows:
> 
> language = request.user.language # will probably work given a request
> localename = locale.normalize(language)
> 
> For me, this yielded "no_NO.ISO8859-1" from "no".

How would I try this? I don't know (hardly) any python.


> A few problems emerge when using locale support for sorting. Firstly, you
> need to have the necessary locales installed for the functions to work.

System-wide (Not just python)? I have that. (My LANG variable is 
"de_DE.UTF-8".)


> Secondly, switching locales affects the entire program, so you have to be
> careful not to cause side-effects, although this is less of a problem in a
> plain CGI environment.
> 
> Another thing noted earlier is that locales are language specific, so if
> your list of page names contains both German names and names using
> non-German characters, the sorting of those other characters may not be as
> desired. Libraries like ICU might try and reconcile different collations,
> but it is probably an open-ended problem. Bindings for Python are available
> here:
> 
> https://pypi.python.org/pypi/PyICU/
> 
> The documentation for the locale functionality is found here:
> 
> https://docs.python.org/2.7/library/locale.html

All this seems to be very complicated. I think I'll only do the ordering which 
ignores case. It's not that important to me.

However, thany you very much for your time.

Bye
Volker