[New-bugs-announce] [issue2931] optparse: various problems with unicode and gettext

[Alexey Shamrin]

New submission from Alexey Shamrin <shamrin at gmail.com>:

In the process of trying to use optparse with russian messages, I found
several problems with gettext and unicode handling:

1. optparse.OptionParser.error function doesn't work with unicode argument
2. optparse doesn't work when its error messages are gettext-translated
3. optparse fails running 'prog.py --help > out.txt' with unicode help
(at least on my system: Windows XP, Russian)

I have attached a file demonstrating these problems: test_optparse.py.
You can run it either using nose[1] or directly, manually uncommenting
test_* functions one-by-one.

[1]: http://www.somethingaboutorange.com/mrl/projects/nose/

Here's the result of running `nosetests test_optparse.py`:

EEF
======================================================================
ERROR: OptionParser.error function doesn't work with unicode argument
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"c:\python25\lib\site-packages\nose-0.10.2-py2.5.egg\nose\case.py", line
182, in runTest
    self.test(*self.arg)
  File "C:\work\test_optparse.py", line 10, in test_unicode_error
    optparse.OptionParser().error(russian_text)
  File "C:\Python25\lib\optparse.py", line 1562, in error
    self.exit(2, "%s: error: %s\n" % (self.get_prog_name(), msg))
  File "C:\Python25\lib\optparse.py", line 1551, in exit
    sys.stderr.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
28-34: ordinal not in range(128)

======================================================================
ERROR: optparse doesn't work when its error messages are gettext-translated
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"c:\python25\lib\site-packages\nose-0.10.2-py2.5.egg\nose\case.py", line
182, in runTest
    self.test(*self.arg)
  File "C:\work\test_optparse.py", line 25, in
test_translated_unicode_error_message
    optparse.OptionParser().parse_args(["--unknown"])
  File "C:\Python25\lib\optparse.py", line 1380, in parse_args
    self.error(str(err))
UnicodeEncodeError: 'ascii' codec can't encode characters in position
0-6: ordinal not in range(128)

======================================================================
FAIL: optparse fails running 'prog.py --help > out.txt' with unicode help
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"c:\python25\lib\site-packages\nose-0.10.2-py2.5.egg\nose\case.py", line
182, in runTest
    self.test(*self.arg)
  File "C:\work\test_optparse.py", line 42, in test_redirected_unicode_help
    assert '?????' not in dummy_stdout.getvalue()
AssertionError

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (errors=2, failures=1)

----------
components: Library (Lib)
files: test_optparse.py
messages: 67129
nosy: ash, gward
severity: normal
status: open
title: optparse: various problems with unicode and gettext
Added file: http://bugs.python.org/file10386/test_optparse.py

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2931>
__________________________________