
Feb. 4, 2020
11:18 a.m.
New issue 3165: PyPy should set `errors="surrogatepass"` when decoding numpy arrays to unicode scalars https://bitbucket.org/pypy/pypy/issues/3165/pypy-should-set-errors-surrogate... Eric Wieser: See [https://github.com/numpy/numpy/issues/15363](https://github.com/numpy/numpy/issues/15363), where the bug was reported in numpy. The fix in numpy was to replace calls to `PyUnicode_DecodeUTF32` with calls to `PyUnicode_FromKindAndData(PyUnicode_4BYTE_KIND, ...)`. However, it seems that PyPI does not use our code there, and has their own implementation. Assuming `PyUnicode_FromKindAndData` is not available, the equivalent python code would be `some_bytes.decode('utf-32-le', errors='surrogatepass')`.