[Python-Dev] Allowing Unicode literals even without Unicode support?

[Michael Hudson]

Guido van Rossum <guido@python.org> writes:

> Many test modules fail because they use Unicode literals.  It's easy
> enough to skip Unicode-related tests (just test for the existence of
> 'unicode'), but it's hard to avoid having Unicode literals in the text
> at all.  Should we perhaps silently interpret Unicode literals as
> regular string literals when compiling without Unicode support?

Hmm.  It would be nice to have less tests fail in --disable-unicode
builds (there were real bugs hiding in there!).

But I'm not sure that accepting u'' literals is wise, generally
speaking, just because of the confusion it might cause.  Perhaps a
commandline flag?

Anyway, I've been meaning to post here about --disable-unicode builds
for a while, so here's some issues that currently exist.

1) In --disable-unicode builds, it seems that much of the codec
machinery just doesn't exist.  While this was fine once, we now have,
e.g.

>>> "sheesh".encode("rot-13")
'furrfu'

But in a disable unicode build:

>>> "sheesh".encode("rot-13")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/mwh/src/python/dist/src/Lib/encodings/rot_13.py", line 18, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
AttributeError: 'module' object has no attribute 'charmap_encode'

This is why test_string still fails.

I'm not sure what to do about this.

2) Some library modules just don't seem to function without unicode.
Current bad boys seem to include gettext, urllib (well, this just got
fixed I think), minidom, pyexpat and the email package.  I could
imagine the xml modules and the email package just plain not being
supported w/o unicode.  gettext, too, I guess.

In which case:

3) regrtest should probably be taught not to run test_unicode + close
friends in --disable-unicode builds.

Anyone feel like digging into any of the above?

Cheers,
M.

-- 
  You're posting to a Scheme group. Around here, arguing that Java is
  better than C++ is like arguing that grasshoppers taste better than
  tree bark.                        -- Thant Tessman, comp.lang.scheme