[New-bugs-announce] [issue27800] Regular expressions with multiple repeat codes

[Martin Panter]

New submission from Martin Panter:

In the documentation for the “re” module, it says repetition codes like {4} and “*” operate on the preceding regular expression. But even though “a{4}” is a valid expression, the obvious way to apply a “*” repetition to it fails:

>>> re.compile("a{4}*")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/proj/python/cpython/Lib/re.py", line 223, in compile
    return _compile(pattern, flags)
  File "/home/proj/python/cpython/Lib/re.py", line 292, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/home/proj/python/cpython/Lib/sre_compile.py", line 555, in compile
    p = sre_parse.parse(p, flags)
  File "/home/proj/python/cpython/Lib/sre_parse.py", line 792, in parse
    p = _parse_sub(source, pattern, 0)
  File "/home/proj/python/cpython/Lib/sre_parse.py", line 406, in _parse_sub
    itemsappend(_parse(source, state))
  File "/home/proj/python/cpython/Lib/sre_parse.py", line 610, in _parse
    source.tell() - here + len(this))
sre_constants.error: multiple repeat at position 4

As a workaround, I found I can wrap the inner repetition in (?:. . .):

>>> re.compile("(?:a{4})*")
re.compile('(?:a{4})*')

The problems with the workaround are (a) it is far from obvious, and (b) it adds more complicated syntax. Either this limitation should be documented, or if there is no good reason for it, it should be lifted. It is not clear if my workaround is entirely valid, or if I just found a way to bypass some sanity check.

My original use case was scanning a base-64 encoding for Issue 27799:

# Without the second level of brackets, this raises a "multiple repeat" error
chunk_re = br'(?: (?: [^A-Za-z0-9+/=]* [A-Za-z0-9+/=] ){4} )*'

----------
components: Regular Expressions
messages: 273107
nosy: ezio.melotti, martin.panter, mrabarnett
priority: normal
severity: normal
status: open
title: Regular expressions with multiple repeat codes
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27800>
_______________________________________