On 31.01.2018 17:36, Guido van Rossum wrote:
On Wed, Jan 31, 2018 at 3:03 AM, Serhiy Storchaka <storchaka@gmail.com <mailto:storchaka@gmail.com>> wrote:
19.01.18 05:51, Guido van Rossum пише:
Can someone explain to me why this is such a controversial issue?
It seems reasonable to me to add new encodings to the stdlib that do the roundtripping requested in the first message of the thread. As long as they have new names that seems to fall under "practicality beats purity". (Modifying existing encodings seems wrong -- did the feature request somehow transmogrify into that?)
In any case you need to change your code. If add new error handler -- you need to change the decoding code to use this error handler:
text = data.decode(encoding, 'whatwgreplace')
If add new encodings -- you need to support an alias table that maps standard encoding names to corresponding names of WHATWG encoding:
aliases = {'windows_1252': 'windows-1252-whatwg', 'windows_1251': 'windows-1251-whatwg', 'utf_8': 'utf-8-whatwg', # utf-8 + surrogatepass ... } ... text = data.decode(aliases.get(normalize_encoding(encoding), encoding))
I don't see an advantage of the second approach for the end user. And of course it is more costly for maintainers, because we will need to implement around 20 new encodings, and adds a cognitive burden for new Python users, which now have more tables of encodings in the documentation.
Hm. As a user, unless I run into problems with a specific encoding, I never care about how many encodings we have, so I don't see how adding extra encodings bothers those users who have no need for them.
There's a reason to prefer new encoding names (maybe augmented with alias table) over a new error handler: there are lots of places where encodings are passed around via text files, Internet protocols, RPC calls, layers and layers of function calls. Many of these treat the encoding as a string, not as a (string, errorhandler) pair. So there may be situations where there is no way in a given API to preserve the need for using a special error handler, while the API would not have a problem preserving just the encoding name.
I already mentioned several reasons why I don't believe it's a good idea to add these encodings to the stdlib as opposed to keeping them on PyPI for those who need them, so won't repeat. One detail I did not mention is that these encodings do not have standard names. WHATWG uses the same names as the original encodings from which they derive - which makes sense for their intended purpose to interpret data coming from web servers, essentially in a decoding only way, but cannot be used for Python since our encodings follow the Unicode standard and don't generate mojibake when encoding. Whatever name would be used in the stdlib would neither be compatible to WHATWG nor to IANA. No other tool outside Python would be able to interpret the encoded data using those names. Given all those issues, I don't see what the benefit would be to add these encodings to the stdlib over leaving them on PyPI for the special use case of reading broken web server data. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 31 2018)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/