[New-bugs-announce] [issue21977] In the re's token example OP and SKIP regexes can be improved

[py.user]

New submission from py.user:

https://docs.python.org/3/library/re.html#writing-a-tokenizer

There are redundant escapes in the regex:

('OP',      r'[+*\/\-]'),    # Arithmetic operators

Sequence -+*/ is sufficient.

It makes the loop to do all steps on every 4 spaces:

('SKIP',    r'[ \t]'),       # Skip over spaces and tabs

Sequence [ \t]+ is faster.


Applied patch.

----------
assignee: docs at python
components: Documentation, Regular Expressions
files: re_ex_tok.diff
keywords: patch
messages: 223000
nosy: docs at python, ezio.melotti, mrabarnett, py.user
priority: normal
severity: normal
status: open
title: In the re's token example OP and SKIP regexes can be improved
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file35951/re_ex_tok.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21977>
_______________________________________