[issue1679] tokenizer permits invalid hex integer

[Martin Rinehart]

New submission from Martin Rinehart:

The tokenizer accepts '0x' as an integer zero. The documentation says:
   hexinteger ::= 0x|Xhexdigit+

Stumbled on this testing a tokenizer I wrote in Python for another
language. Expected an Error on "int( '0x', 16 )", but didn't get one.

----------
components: Interpreter Core
messages: 58943
nosy: MartinRinehart
severity: normal
status: open
title: tokenizer permits invalid hex integer
type: compile error
versions: Python 2.6

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1679>
__________________________________