New submission from py.user: https://docs.python.org/3/library/re.html#writing-a-tokenizer There are redundant escapes in the regex: ('OP', r'[+*\/\-]'), # Arithmetic operators Sequence -+*/ is sufficient. It makes the loop to do all steps on every 4 spaces: ('SKIP', r'[ \t]'), # Skip over spaces and tabs Sequence [ \t]+ is faster. Applied patch. ---------- assignee: docs@python components: Documentation, Regular Expressions files: re_ex_tok.diff keywords: patch messages: 223000 nosy: docs@python, ezio.melotti, mrabarnett, py.user priority: normal severity: normal status: open title: In the re's token example OP and SKIP regexes can be improved type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file35951/re_ex_tok.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue21977> _______________________________________