[issue4021] tokenize.detect_encoding(): raise SyntaxError on codecs.lookup() error

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

tokenize.detect_encoding() raises a LookupError() if the charset is 
unknown whereas Python raises a SyntaxError. So this patch mimics 
Python behaviour for tokenize module.

Extra: reuse BOM_UTF8 from the codecs module.

----------
files: tokenize_detect_encoding.patch
keywords: patch, patch
messages: 74205
nosy: haypo
severity: normal
status: open
title: tokenize.detect_encoding(): raise SyntaxError on codecs.lookup() error
versions: Python 3.0
Added file: http://bugs.python.org/file11683/tokenize_detect_encoding.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4021>
_______________________________________