[Python-3000] Py3k_struni additional test failures under cygwin

[Joe Smith]

"Guido van Rossum" <guido at python.org> wrote in message 
news:ca471dc20707181113m360db736h2fd079f29f71220 at mail.gmail.com...
> On 7/18/07, Joe Smith <unknown_kev_cat at hotmail.com> wrote:
>>
>> "Guido van Rossum" <guido at python.org> wrote in message
>> news:ca471dc20707181002w64e076aco9a509ec7e4e15b9a at mail.gmail.com...
>> > On 7/17/07, Joe Smith <unknown_kev_cat at hotmail.com> wrote:
>> >> Building Py3k_struni under Cygwin I've noticed a few more tests 
>> >> failing
>> >> than
>> >> the wiki shows.
>> >> These are using SVN revision 56413.
>> >>
>> >> Some spurious errors seem to occur if Python/ is not remaned 
>> >> temporally.
>> >> I
>> >> have not included those. (This is an oddity of the cygwin '.exe'
>> >> autohandling combined with case-insensitivity)
>> >>
>> >>
>> >> Test_coding: Errors. Traceback included at end of message.
>> >> "test test_descr failed -- ['foo\u1234bar'] slots not caught"
>> >> "test test_largefile failed -- got b'z', but expected 'z'"
>> >> test_marshal: Tests that fail are fasiling with a recursion limit
>> >> exceeded
>> >> error.
>> >>
>> >>
>> >>
>> >> Tracebacks:
>> >>
>> >> test test_coding failed -- Traceback (most recent call last):
>> >>   File "/home/Owner/py3k-struni/Lib/test/test_coding.py", line 12, in
>> >> test_bad_c
>> >> oding2
>> >>     self.verify_bad_module(module_name)
>> >>   File "/home/Owner/py3k-struni/Lib/test/test_coding.py", line 20, in
>> >> verify_bad
>> >> _module
>> >>     text = fp.read()
>> >>   File "/home/Owner/py3k-struni/Lib/io.py", line 1186, in read
>> >>     res += decoder.decode(self.buffer.read(), True)
>> >>   File "/home/Owner/py3k-struni/Lib/encodings/ascii.py", line 26, in
>> >> decode
>> >>     return codecs.ascii_decode(input, self.errors)[0]
>> >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 
>> >> 0:
>> >> ordinal
>> >> not in range(128)
>> >
>> > The test_descr and test_largefile failures are reproducible on Ubuntu
>> > and someone will eventually fix them.
>> >
>> > I can't reproduce the test_marshal and test_coding failures; please
>> > investigate more on CYGWIN.
>>
>> For the test coding, apprently the module's contents are intended to be
>> loaded, and then verified that a syntax error occurs when trying to parse
>> the module. However, on cygwin i'm consistantly getting an error on the 
>> line
>> that reads the file. Specificly fp.read().
>>
>> Fp.read() appears to be trying to export a unicode string by interpreting
>> the byte string as ascii. The byte string is most certainly not valid 
>> ascii.
>> So the codec throws an error. I'm guessing for some reason python 
>> normally
>> chose a different codec, but on my cygwin compiles it is choosing ascii. 
>> I'm
>> not sure why. Nor am I sure how to inestigate further.
>
> The encoding defaults to the filesystem encoding or otherwise Latin-1.
> There's an XXX comment in io.py, in TextIOWrapper.__init__, admitting
> this is questionable. I'm guessing CYGWIN has a filesystem encoding
> equal to ASCII? Is this a good idea?

Quite possibly. I know they have wanted to move using the unicode API's to 
support everything,
but that is a pain because of the meathod that windows uses internally to 
support Unicode.

> Maybe the default encoding should always be UTF-8 (matching the source
> code default encoding).
>
> I can also fix it by changing test_coding.py to add encoding="utf-8"
> to the open() call in verify_bad_module().
>
>> Heres a fairly useless loking traceback for test_marshal. Many of the 
>> tests
>> fail with nearly identical tracebacks:
>>
>> #======================================================================
>> #ERROR: test_tuple (test.test_marshal.ContainerTestCase)
>> #----------------------------------------------------------------------
>> #Traceback (most recent call last):
>> #  File "/home/Owner/py3k-struni/Lib/test/test_marshal.py", line 134, in
>> test_tuple
>> #   self.helper(tuple(self.d.keys()))
>> # File "/home/Owner/py3k-struni/Lib/test/test_marshal.py", line 21, in
>> helper
>> #   new = marshal.load(f)
>> #ValueError: recursion limit exceeded
>>
>> For what it's worth here is the fll subtest list and status for
>> test_marshal:
>>
>> #test_bool (test.test_marshal.IntTestCase) ... ERROR
>> #test_int64 (test.test_marshal.IntTestCase) ... ok
>> #test_ints (test.test_marshal.IntTestCase) ... ERROR
>> #test_floats (test.test_marshal.FloatTestCase) ... ERROR
>> #test_buffer (test.test_marshal.StringTestCase) ... ERROR
>> #test_string (test.test_marshal.StringTestCase) ... ERROR
>> #test_unicode (test.test_marshal.StringTestCase) ... ERROR
>> #test_code (test.test_marshal.CodeTestCase) ... ok
>> #test_dict (test.test_marshal.ContainerTestCase) ... ERROR
>> #test_list (test.test_marshal.ContainerTestCase) ... ERROR
>> #test_sets (test.test_marshal.ContainerTestCase) ... ERROR
>> #test_tuple (test.test_marshal.ContainerTestCase) ... ERROR
>> #test_exceptions (test.test_marshal.ExceptionTestCase) ... ok
>> #test_bug_5888452 (test.test_marshal.BugsTestCase) ... ok
>> #test_fuzz (test.test_marshal.BugsTestCase) ... ok
>> #test_loads_recursion (test.test_marshal.BugsTestCase) ... ok
>> #test_patch_873224 (test.test_marshal.BugsTestCase) ... ok
>> #test_recursion_limit (test.test_marshal.BugsTestCase) ... ok
>> #test_version_argument (test.test_marshal.BugsTestCase) ... ok
>>
>> I'm wondering if the recusion limit on my build is getting set too low
>> somehow.
>
> Can you find out what it is? sys.getrecursionlimit().

Hmm...  It is a limit of 1000.
That is probably large enough, no?

Anyway, from some basic testing it looks like marshal is always throwing 
that error when marshal.load() is called.
However, marshal.loads() works fine.

Might this be another encoding related error?