[Python-Dev] Nasty trunk test failures

[Guido van Rossum]

On 3/30/06, Tim Peters <tim.peters at gmail.com> wrote:
> test_tokenize started failing on all the trunk buildbots immediately
> after this seemingly unrelated checkin:

tokenize seems to be mishandling this line:

        assert 6 .__index__() == 6

Note the space between '6' and '.'.

I'm guessing that the untokenization of this somehow drops the space;
this seems to be a bug in untokenize() which probably should add a
safety space after names as well as numbers. Yes, this fixes the
problem:

Index: tokenize.py
===================================================================
--- tokenize.py (revision 43463)
+++ tokenize.py (working copy)
@@ -182,7 +182,7 @@
     for tok in iterable:
         toknum, tokval = tok[:2]

-        if toknum == NAME:
+        if toknum in (NAME, NUMBER):
             tokval += ' '

         if toknum == INDENT:

I'll check this in.

Then there's another (shallow) problem that only occurs when I run
test_tokenize.py directly -- the doctest for decistmt()
has-3.21716034272e-007 but (on my box) this outputs
-3.21716034272e-07. That doesn't seem to bother it when run via
regrtest.py. I'm not sure what's at fault here.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)