It's worth comparing the situation with byte arrays. There is no problem
of translating different representations of an element, but there is
still the issue of who owns the memory. The Python buffer protocol
usually solves this problem, so something similar for unicode "arrays"
might suffice.
Exactly my thought on the matter. I have no doubt that between all of us we could design a decent protocol.
The practical problem would be to convince enough people that this is worth doing to actually get the code changed (str being one of the most popular data types traveling across C API boundaries), in the CPython core (which surely has a lot of places to modify) as well as in the vast collection of affected 3rd party modules. Like many migrations it's an endless slog for the developers involved, and in open source it's hard to assign resources for such a project.