[Python-Dev] SyntaxError for illegal literals

[Guido van Rossum]

> In support of the argument that bad literals should raise ValueError
> (or a derived exception) rather than SyntaxError, Guido once said:
> 
> > "Problems with literal interpretations
> > traditionally raise 'runtime' exceptions rather than syntax errors."
> 
> This is currently true of overflowing integers and string literals,
> and hence it has also been so implemented for Unicode literals.
> 
> But i want to propose a break with tradition, because some more recent
> thinking on this has led me to become firmly convinced that SyntaxError
> is really the right thing to do in all of these cases.
> 
> The strongest reason is that a long file with a typo in a string
> literal somewhere in hundreds of lines of code generates only
> 
>     ValueError: invalid \x escape
> 
> with no indication to where the error is -- not even which file!
> I realize this could be hacked upon and fixed, but i think it points
> to a general inconsistency that ought to be considered and addressed.
> 
>     1.  SyntaxErrors are for compile-time errors.  A problem with
>         a literal happens before the program starts running, and
>         it is useful for me, as the programmer, to know whether
>         the error occurred because of some computational process,
>         possibly depending on inputs, or whether it's a permanent
>         mistake that's literally in my source code.  In other words,
>         will a debugger do me any good?
> 
>     2.  SyntaxErrors pinpoint the exact location of the problem.
>         In principle, an error is a SyntaxError if and only if you
>         can point to an exact character position as being the cause
>         of the problem.
> 
>     3.  A ValueError means "i got a value that wasn't allowed or
>         expected here".  That is not at all what is happening.
>         There is *no* defined value at all.  It's not that there
>         was a value and it was wrong -- the value was never even
>         brought into existence.
> 
>     4.  The current implementation of ValueErrors is very unhelpful
>         about what to do about an invalid literal, as explained
>         in the example above.  A SyntaxError would be much more useful.
> 
> I hope you will agree with me that solving only #4 by changing
> ValueErrors so they behave a little more like SyntaxErrors in
> certain particular situations isn't the best solution.
> 
> Also, switching to SyntaxError is likely to break very few things.
> You can't depend on catching a SyntaxError, precisely because it's
> a compile-time error.  No one could possibly be using "except ValueError"
> to try to catch invalid literals in their code; that usage, just like
> "except SyntaxError:", makes sense only when someone is using "eval" or
> "exec" to interpret code that was generated or read from input.
> 
> In fact, i bet switching to SyntaxError would actually make some code
> of the form "try: eval ... except SyntaxError" work better, since the
> single except clause would catch all possible compilation problems
> with the input to eval.

All good points, except that I still find it hard to flag overflow
errors as syntax errors, especially since overflow is platform
defined.

On one platform, 1000000000000 is fine; on another it's a
SyntaxError.  That could be confusing.

But you're absolutely right about string literals, and maybe it's OK
if 1000000000000000000000000000000000000000000000000000000000000000000
is flagged as a syntax error.  (After all it's missing a trailing
'L'.)

Another solution (borrowing from C): automatically promote int
literals to long if they can't be evaluated as ints.

--Guido van Rossum (home page: http://www.python.org/~guido/)