[Python-3000] Pleaswe help with the countdown to zero failing tests in the struni branch!

[Guido van Rossum]

On 8/7/07, Thomas Heller <theller at ctypes.org> wrote:
> Guido van Rossum schrieb:
> > On 8/7/07, Thomas Heller <theller at ctypes.org> wrote:
> >> Guido van Rossum schrieb:
> >> > test_ctypes
> >> > Recently one test started failing again, after Martin changed
> >> > PyUnicode_FromStringAndSize() to use UTF8 instead of Latin1.
> >>
> >> I wanted to look into this and noticed that 'import time' on Windows
> >> doesn't work anymore on my machine.  The reason is that PyUnicode_FromStringAndSize()
> >> is called for the string 'Westeuropäische Normalzeit', and then fails with
> >>
> >> UnicodeDecodeError: 'utf8' codec can't decode bytes in position 9-11: invalid data
> >
> > I'm assuming that's a literal somewhere? In what encoding is it? That
> > function was recently changed to require the input to be UTF-8. If the
> > input isn't UTF-8, you'll have to use another API with an explicit
> > encoding, PyUnicode_Decode().
>
> It's in Modules/timemodule.c, line 691:
>         PyModule_AddObject(m, "tzname",
>                            Py_BuildValue("(zz)", tzname[0], tzname[1]));
>
> According to MSDN, tzname is a global variable; the contents is somehow
> derived from the TZ environment variable (which is not set in my case).

Is there anything from which you can guess the encoding (e.g. the
filesystem encoding?).

> Is there another Py_BuildValue code that should be used?  BTW: There are
> other occurrences of Py_BuildValue("(zz)", ...) in this file; someone should
> probably check if the UTF8 can be assumed as input.

These are all externally-provided strings. It will depend on the
platform what the encoding is.

I wonder if we need to add another format code to Py_BuildValue (and
its friends) to designate "platform default encoding" instead of
UTF-8.

> > I'm pretty sure this change is also responsible for the one failure
> > (as it started around the time that change was made) but I don't
> > understand the failure well enough to track it down. (It looked like
> > uninitialized memory was being accessed though.)
>
> I'm not sure what failure you are talking about here.

When I run test_ctypes I get this (1 error out of 301 tests):

======================================================================
ERROR: test_functions (ctypes.test.test_stringptr.StringPtrTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/google/home/guido/python/py3k-struni/Lib/ctypes/test/test_stringptr.py",
line 72, in test_functions
    x1 = r[0], r[1], r[2], r[3], r[4]
UnicodeDecodeError: 'utf8' codec can't decode byte 0xdb in position 0:
unexpected end of data

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)