Nicolas Fleury wrote:

ottrey@py.redsoft.be wrote:

...

...

...

...

import re2 buf='12 drummers drumming, 11 pipers piping, 10 lords a-leaping' regex='^((?P<verse>(?P<number>\d+) (?P<activity>[^,]+))(, )?)*$' pat2=re2.compile(regex) x=pat2.extract(buf)

If one wanted to match the API of the re module, one should use pat2.findall(buf), which would return a list of 'hierarchical match objects', though with the above, one should really return a list of 'verse' items (the way the regular expression is written).

...

...

...

...

x

{'verse': [{'number': '12', 'activity': 'drummers drumming'}, {'number': '11', 'activity': 'pipers piping'}, {'number': '10', 'activity': 'lords a-leaping'}]}

Is a dictionary the good container or should another class be used? Because in the example the content of the "verse" group is lost, excluding its sub-groups. Something like a hierarchic MatchObject could provide access to both information, the sub-groups and the group itself.

Its contents are not lost, look at the overall dictionary... In any case, I think one can do better than a dictionary.

...

...

x=pat2.match(buf) #or x=pat2.findall(buf)[0] x '12 drummers drumming,' dir(x) ['verse'] x.verse '12 drummers drumming,' dir(x.verse) ['number', 'activity'] x.verse.number '12' x.verse.activity 'drummers drumming'

...would get my vote (or using obj.group(i) semantics I discuss below). I notice that this is basically what the re2 module already does (having read the web page), though rather than...

...

...

pat2.extract(buf).verse[1].activity 'pipers piping'

I would prefer...

...

...

pat2.findall(buf)[1].verse.activity 'pipers piping'

For .verse[1] or .verse[2] to make sense, it implies that the pattern is something like... ((?P<verse>... )(?P<verse>...)) ... which it isn't. I understand that the decision was probably made to make it similar to the case of... ((?P<foo>... (?p<goo>...)+)) ... where multiple matches for goo would require x.foo.goo[i].

Also, should it be limited to named groups?

Probably not. I would suggest using matchobj.group(i) semantics to match the standard re module semantics, though only allow returning items in the current level of the hierarchy. That is, one could use x.verse.group(1) and get back '12', but x.group(1) would return '12 pipers piping'

...

I am wondering what would be the best direction to take this project in.

Firstly is it, (or can it be made) useful enough to be included in the python stdlib? (ie. Should I bother writing a PEP for it.)

And if so, would it be best to merge its functionality in with the re library, or to leave it as a separate module?

And, also are there any suggestions/criticisms on the library itself?

I find the feature very interesting, but being used to live without it, I have difficulty evaluating its usefulness. However, it reminds me how much at first I found strange that only the last match was kept, so I think, FWIW, that on a purist point of vue the functionality would make sense in the stdlib in some way or another.

re2 can be used as a limited structural parser. This makes the re module useful for more things than it is currently. The question of it being in the standard library, however, I think should be made based on the criteria used previously (whatever they were). - Josiah

Josiah Carlson wrote:

re2 can be used as a limited structural parser. This makes the re module useful for more things than it is currently. The question of it being in the standard library, however, I think should be made based on the criteria used previously (whatever they were).

In general, if developers can readily agree that a functionality should be added (i.e. it is "obvious" for some reason), it is added right away. Otherwise, a PEP should be written, and reviewed by the community. In the specific case, Chris Ottrey submitted a link to his project to the SF patches tracker, asking for inclusion. I felt that there is likely no immediate agreement, and suggested he asks on python-dev, and writes a PEP. If this kind of functionality would fall on immediate rejection for some reason, even writing the PEP might be pointless. If the functionality is generally considered useful, a PEP can be written, and then implemented according to the PEP procedures (i.e. collect feedback, discuss alternatives, ask for BDFL pronouncement). I personally think that the proposed functionality should *not* live in a separate module, but somehow be integrated into SRE. Whether or not the proposed functionality is useful in the first place, I don't know. I never have nested named groups in my regular expressions. Regards, Martin

Greetings,

If this kind of functionality would fall on immediate rejection for some reason, even writing the PEP might be pointless. If the [...]

In my opinion the functionality is useful.

I personally think that the proposed functionality should *not* live in a separate module, but somehow be integrated into SRE. Whether or [...]

Agreed. I propose to integrate this functionality into the SRE syntax, so that this special kind of group may be used when explicitly wanted. This would avoid backward compatibility problems, would give each regular expression a single meaning, and would allow interleaving hierarchical/non-hierarchical groups. I offer myself to integrate the change once we decide on the right way to implement it, and achieve consensus on its adoption. Best regards, -- Gustavo Niemeyer http://niemeyer.net

(ie. the re library only returns the ~last~ match for named groups - not a list of ~all~ the matches for the named groups. And the hierarchy of

wrote: those named groups is non-existant in the flat dictionary of matches

that results. )

are you 100% sure that this can be implemented on top of other RE engines (CPython isn't the only Python implementation out there). (generally speaking, trying to turn an RE engine into a parser is a lousy idea. the library would benefit more from a simple parser toolkit than it benefits from more non-standard and highly specialized RE hacks...) </F>