Why not: from itertools import starmap for x in starmap(re.match, *patterns): if x: break On Tue, Jul 21, 2009 at 8:50 PM, MRAB wrote:

Steven D'Aprano wrote:

...

Following the thread "Experiment: Adding "re" to string objects.", I would like to propose the addition of two convenience functions to the re module:

def multimatch(s, *patterns):    """Do a re.match on s using each pattern in patterns,    returning the first one to succeed, or None if they all fail."""    for pattern in patterns:        m = re.match(pattern, s)        if m: return m

def multisearch(s, *patterns):    """Do a re.search on s using each pattern in patterns,    returning the first one to succeed, or None if they all fail."""    for pattern in patterns:        m = re.search(pattern, s)        if m: return m

The rationale is to make the following idiom easier:

m = re.match(s, pattern1) if not m:    m = re.match(s, pattern2)    if not m:        m = re.match(s, pattern3)        if not m:            m = re.match(s, pattern4) if m:    m.group()

which will become:

m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4) if m:    m.group()

Is there any support or objections to this proposal? Any comments?

Extend the current re.match and re.search to accept a tuple of patterns:

   m = re.match((pattern1, pattern2, pattern3, pattern4), s)    if m:        print m.group()

This format is already used by some string methods, eg str.startswith(). _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Gerald Britton

Steven D'Aprano wrote:

On Wed, 22 Jul 2009 06:02:54 pm Ron Adam wrote:

...

One of the needs I've run across is to enable the program user (possibly a non-programmer) to do logical searches on data.

It would be nice if the search patterns specified by the program user could be used directly by the functions. Search functions of this type would take patterns that are more like what you would use for google or yahoo searches instead of the more complex language re requires.

I'm not sure if I understand this correctly. Perhaps you could give an example or two?

Also, please don't overload my simple little proposal with a multitude of new functionality. My proposal is only meant to be a lightweight convenience function. Additional functionality probably belongs as a different function, maybe even a different module.

Yes, it would be a different module and not added directly to the re module. While you are thinking of simplifying re for programmers, I'm thinking of simplified searches for users. A different target and purpose. I think your functions would make this idea easier to do. It would be nice if we could do simple logical searches where. [word1 word2] ;get results with either word1 or word2 [+word1 +word2] ;get results with both word1 and word2 [word1 -word2] ;get results with word1 and not with word2 ["word one" "word two"] ;use quotes to search for phrases And possibly use '*' and '?' as simple wild cards but keep it easy to use and simple. More complex searches should use the re module directly. This would act as a filter for lists and would be suitable for adding a *simple* user search capability to many scrips and applications. An example would be to enhance pydocs search of the summery lines. Currently if you type "modules key", if the key is multiple words, it only searches on the first word. You can not do searches on multiple words or exclude results with certain words. While we could allow regular expression input to work, for many applications it is overkill and it is too complex for many users. For example I would not like to try and teach my parents all the subtleties of regular expressions when they are struggling to understand a lot more basic things. They don't want to learn how to program computers, they just want to get a recipe that has [+chicken +"tomato sauce" -onions]. Ron

Ron Adam "

One of the needs I've run across is to enable the program user (possibly a non-programmer) to do logical searches on data.

It would be nice if the search patterns specified by the program user could be used directly by the functions. Search functions of this type would take patterns that are more like what you would use for google or yahoo searches instead of the more complex language re requires.

It's a proposition of another string matching module ("browsermatch"? :-)) +0.6 *j

Talin wrote:

Steven D'Aprano wrote:

...

Following the thread "Experiment: Adding "re" to string objects.", I would like to propose the addition of two convenience functions to the re module:

def multimatch(s, *patterns): """Do a re.match on s using each pattern in patterns, returning the first one to succeed, or None if they all fail.""" for pattern in patterns: m = re.match(pattern, s) if m: return m

There's a cute trick that you can use to do this that is much more efficient than testing each regex expression individually:

combined_pattern = "|".join("(%s)" % p for p in patterns) combined_re = re.compile(combined_pattern)

m = combined_re.match(string) return m.lastindex

Basically, it combines all of the patterns into a single large regex, where each pattern is converted into a capturing group. It then returns match.lastindex, which is the index of the capturing group that matched. This is very efficient because now all of the patterns are combined into a single NFA which can prune possibilities very quickly.

This works for up to 99 patterns, which is the limit on the number of capturing groups that a regex can have.

[snip] It won't work properly if the patterns themselves contain capture groups.

Georg Brandl wrote:

You might be interested in the undocumented re.Scanner class :)

It looks like it could be interesting, but with no documentation, how are we supposed to tell? Is there *any* information about it available anywhere? -- Greg

2009/7/23 Robert Kern :

On 2009-07-22 19:49, Greg Ewing wrote:

...

Georg Brandl wrote:

...

You might be interested in the undocumented re.Scanner class :)

It looks like it could be interesting, but with no documentation, how are we supposed to tell?

Is there *any* information about it available anywhere?

http://mail.python.org/pipermail/python-dev/2003-April/035075.html

Question: Is there any reason (other than lack of time) why it's undocumented? I'd be willing to write some documentation, but only if it would stand a chance of being accepted - this isn't an itch of mine, so I don't want to spend ages arguing over whether the class should be documented. The source code says "experimental stuff (see python-dev discussions for details)". I've not searched the python-dev archives yet, but it seems to me that it'll never be anything other than experimental if people don't know it's there and try it out... Paul.

So, in 2003 it was discussed but considered not ready for prime time. If six years is not enough to get it ready, I can't imagine what would be. On Thu, Jul 23, 2009 at 1:06 PM, Robert Kern wrote:

On 2009-07-23 11:50, Paul Moore wrote:

...

2009/7/23 Robert Kern:

...

On 2009-07-22 19:49, Greg Ewing wrote:

...

Georg Brandl wrote:

...

You might be interested in the undocumented re.Scanner class :)

It looks like it could be interesting, but with no documentation, how are we supposed to tell?

Is there *any* information about it available anywhere?

http://mail.python.org/pipermail/python-dev/2003-April/035075.html

Question: Is there any reason (other than lack of time) why it's undocumented?

http://mail.python.org/pipermail/python-dev/2003-April/035070.html

-- Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma  that is made terrible by our own mad attempt to interpret it as though it had  an underlying truth."  -- Umberto Eco

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- Gerald Britton

Robert Kern wrote:

On 2009-07-23 16:48, Gerald Britton wrote:

...

So, in 2003 it was discussed but considered not ready for prime time. If six years is not enough to get it ready, I can't imagine what would be.

I'm pretty sure that no one has worked on it.

I worked on it in issue #2636 (turning unnamed capture groups into non-capture groups and forbidding group references and named groups) and also added a generator method.

BJörn Lindqvist wrote:

2009/7/22 Steven D'Aprano :

...

The rationale is to make the following idiom easier:

m = re.match(s, pattern1) if not m: m = re.match(s, pattern2) if not m: m = re.match(s, pattern3) if not m: m = re.match(s, pattern4) if m: m.group()

which will become:

m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4) if m: m.group()

Is there any support or objections to this proposal? Any comments?

I don't like it very much because it would only work for uncompiled patterns. All functions in re has a RegexObject counterpart, but multisearch() and multimatch() obviously would not. For the quoted example I'd usually try to create one regex that matches all four patterns, or use a loop:

for pat in (pattern1, pattern2, pattern3, pattern4): m = re.match(s, pat) if m: m.group() break

re.match and re.search will accept either a string or a compiled pattern as the first argument. Never used it myself, but...

Steven D'Aprano wrote:

On Thu, 23 Jul 2009 07:38:11 am Mark Dickinson wrote:

...

Steven, could you show some examples of real(ish)-world use-cases for one or both of these functions? Preferably including the code that might directly follow a multimatch or multisearch call.

I'm afraid that I don't use regexes anywhere near enough to champion this proposal in the face of serious opposition, or even skepticism. If this isn't a simple enough "no-brainer", then I'm going to have to pass the baton onto somebody else (assuming anyone actually likes this idea).

This idea came about from the thread started by Sean Reifschneider, proposing adding regexes to strings. I thought (and Sean seemed to agree) that these convenience functions would solve his primary use-case. So this proposal isn't scratching an itch I have.

...

It's probably because I haven't used regexes widely enough, but in all the potential examples I can come up with, either

(1) the regexes are similar enough that they can be refactored into a single regex (e.g., just concatenated with '|'), or

(2) they're distinct enough that each regex needs its own handing, so that the multimatch/multisearch would need to be followed by a multiway 'if/elif/else' anyway; in this case, it seems that little is gained.

These are both reasonable approaches. This proposal isn't supposed to solve every multiple-regex-handling problem.

So far support for this has been luke-warm. If anyone really likes this idea, please speak up, otherwise I'll let it drop.

If you want to try multiple regexes until one matches then the approach with re.match accepting a tuple of patterns would, it seems to me, to be the one that requires the smallest change and has the greatest precedence (like str.startwith).

Jan Kaliszewski wrote:

It sounds nice. But why not to use simply:

m = re.match(s, '|'.join(pattern1, pattern2, pattern3, pattern4))

Sorry, I ment of course: m = re.match('|'.join(pattern1, pattern2, pattern3, pattern4), s) ***

    """Do a re.match on string using pattern,     returning dict containing groups which could be got by index or by name."""

I ment: "...returning collections.DefaultDict..." (as you can see in the code following). Regards, *j -- Jan Kaliszewski