Adding new escapes to regex module

Other regex implementations have escape sequences for horizontal whitespace (`\h` and `\H`) and vertical whitespace (`\v` and `\V`). The regex module already supports `\h`, but I can't use `\v` because it represents `\0x0b', as it does in the re module. Now that someone has asked for it, I'm trying to find a nice way of adding it, and I'm currently thinking that maybe I could use `\y` and `\Y` instead as they look a little like `\v` and `\V`, and, also, vertical whitespace is sort-of in the y-direction. As far as I can tell, only ProgressSQL uses them, and, even then, it's for what everyone else writes as `\b` and `\B`. I want the regex module to remain compatible with the re module, in case they get added there sometime in the future. Opinions?

You seem to be mixing the use \ as the escape for strings and the \ that re uses. Is it the behaviour that '\<unknown>' becomes '\\<unknown>' that means this is a breaking change? Won't this work? ``` re.compile('\v:\\v') # which is the same as re.compile(r'\x0b:\v') ``` Barry

On 2022-08-16 22:14, Barry Scott wrote:
Some languages, e.g. Perl, have a dedicated syntax for writing regexes, and they take `\n` (a backslash followed by 'n') to mean "match a newline". Other languages, including Python, use string literals and can contain an actual newline, but they also take `\n` (a backslash followed by 'n') to mean "match a newline". Thus:
On the other hand:
The problem is that the re and regex modules already have the `\v` (a backslash followed by 'v') sequence to mean "match the '\v' character", so: re.compile('\v') and: re.compile('\\v') mean exactly the same.

16.08.22 23:24, MRAB пише:
I do not like introducing escapes which are not supported in other RE implementations. There is a chance of future conflicts. Java broke compatibility in Java 8 by redefining \v from a single vertical tab character to the vertical whitespace class. I am not sure that it is a good example that we should follow, because different semantic of \v in raw and non-raw strings is a potential source of bugs. But with special flag which controls the meaning of \v it may be more safe. Horizontal whitespace can be matched by [ \t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000] in re or [\t\p{Zs}] in regex. Vertical whitespace can be matched by [\n\x0b\f\r\x85\u2028\u2029]. Note that there is a dedicated Unicode category for horizontal whitespaces (excluding the tab itself), but not for vertical whitespaces, it means that vertical whitespaces are less important. In any case it is simple to introduce special Unicode categories and use \p{ht} and \p{vt} for horizontal and vertical whitespaces.

You seem to be mixing the use \ as the escape for strings and the \ that re uses. Is it the behaviour that '\<unknown>' becomes '\\<unknown>' that means this is a breaking change? Won't this work? ``` re.compile('\v:\\v') # which is the same as re.compile(r'\x0b:\v') ``` Barry

On 2022-08-16 22:14, Barry Scott wrote:
Some languages, e.g. Perl, have a dedicated syntax for writing regexes, and they take `\n` (a backslash followed by 'n') to mean "match a newline". Other languages, including Python, use string literals and can contain an actual newline, but they also take `\n` (a backslash followed by 'n') to mean "match a newline". Thus:
On the other hand:
The problem is that the re and regex modules already have the `\v` (a backslash followed by 'v') sequence to mean "match the '\v' character", so: re.compile('\v') and: re.compile('\\v') mean exactly the same.

16.08.22 23:24, MRAB пише:
I do not like introducing escapes which are not supported in other RE implementations. There is a chance of future conflicts. Java broke compatibility in Java 8 by redefining \v from a single vertical tab character to the vertical whitespace class. I am not sure that it is a good example that we should follow, because different semantic of \v in raw and non-raw strings is a potential source of bugs. But with special flag which controls the meaning of \v it may be more safe. Horizontal whitespace can be matched by [ \t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000] in re or [\t\p{Zs}] in regex. Vertical whitespace can be matched by [\n\x0b\f\r\x85\u2028\u2029]. Note that there is a dedicated Unicode category for horizontal whitespaces (excluding the tab itself), but not for vertical whitespaces, it means that vertical whitespaces are less important. In any case it is simple to introduce special Unicode categories and use \p{ht} and \p{vt} for horizontal and vertical whitespaces.
participants (4)
-
Antoine Pitrou
-
Barry Scott
-
MRAB
-
Serhiy Storchaka