[Python-Dev] bytes / unicode

[Nick Coghlan]

On Mon, Jun 21, 2010 at 12:30 PM, P.J. Eby <pje at telecommunity.com> wrote:
> I also find it weird that there seem to be two camps on this subject, one of
> which claims that All Is Well And There Is No Problem -- but I do not recall
> seeing anyone who was in the "What do I do; this doesn't seem ready" camp
> who switched sides and took the time to write down what made them realize
> that they were wrong about there being a problem, and what steps they had to
> take.  The existence of one or more such documents would certainly ease my
> mind, and I imagine that of other people who are less waiting for others'
> libraries, than for the stdlib (and/or language) itself to settle.
>
> (Or more precisely, for it to be SEEN to have settled.)

I don't know that the "all is well" camp actually exists. The camp
that I do see existing is the one that says "without a bug report,
inconsistencies in the standard library's unicode handling won't get
fixed".

The issues picked up by the regression test suite have already been
dealt with, but that suite is unfortunately far from comprehensive.
Just like a lot of Python code that is out there, the standard library
isn't immune to the poor coding practices that were permitted by the
blurry lines between text and octet streams in 2.x.

It may be that there are places where we need to rewrite standard
library algorithms to be bytes/str neutral (e.g. by using length one
slices instead of indexing). It may be that there are more APIs that
need to grow "encoding" keyword arguments that they then pass on to
the functions they call or use to convert str arguments to bytes (or
vice-versa). But without people trying to port affected libraries and
reporting bugs when they find issues, the situation isn't going to
improve.

Now, if these bugs are already being reported against 3.1 and just
aren't getting fixed, that's a completely different story...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia