[Python-Dev] Better SyntaxError messages

[Ka-Ping Yee]

On Tue, 4 Jul 2000, Ka-Ping Yee wrote:
> It turns out that this should be quite easy.  If it weren't
> past 4am i would be posting a patch instead of just a verbal
> suggestion right now -- but here's how to do it.

Okay, done.  Submitted to SourceForge as patch #100734.  For the patch
itself, see http://sourceforge.net/patch/download.php?id=100734 or
http://sourceforge.net/patch/?func=detailpatch&patch_id=100734&group_id=5470

Here is a sample of the resulting behaviour:

    >>> if 3:
    ... 3
      File "<stdin>", line 2
        3
        ^
    SyntaxError: expected indent here
    >>>  3
      File "<stdin>", line 1
        3
        ^
    SyntaxError: unexpected indent
    >>> if 1:
    ...   3
    ...  4
      File "<stdin>", line 3
        4
        ^
    SyntaxError: dedent does not match any outer indentation level
    >>> if 1:
    ...   if 2:
    ... 3
      File "<stdin>", line 3
        3
        ^
    SyntaxError: unexpected dedent
    >>> if 1:
    ...   if 2:
    ...  3
      File "<stdin>", line 3
        3
        ^
    SyntaxError: dedent does not match any outer indentation level
    >>> 

Also (didn't want to waste pages of text demonstrating it) if you
indent more than 100 levels, you get

    SyntaxError: too many levels of indentation

instead of the current message, "invalid token".


The internal changes are:

    1. Added E_TOODEEP (too many indentation levels) and
       E_DEDENT (no matching outer block for dedent) to errcode.h.

    2. Made PyTokenizer_Get (in tokenizer.c) return E_TOODEEP and
       E_DEDENT instead of printing messages to stderr; added
       corresponding error messages in err_input (in pythonrun.c).

    3. Renamed E_INDENT to E_TABSPACE.

    4. Added int *expected_ret argument to PyParser_AddToken; on
       a syntax error, PyParser_AddToken (in parser.c) places the
       expected token type (if any) into this output parameter.

    5. Added "token" and "expected" to struct perrdetail; made
       parsetok (in parsetok.c) copy the current token into "token"
       and the expected token into "expected" on a syntax error.

    6. Added checks in err_input (in pythonrun.c) to print more
       detailed error messages if err->token or err->expected
       happens to be INDENT or DEDENT when a syntax error occurs.

This leaves the door open for err_input to say more interesting
things about the syntax error if it wants, based on the information
in err->token and err->expected, though i haven't added any more
detailed messages except for the ones about indentation for now.


Look good to you?



-- ?!ng