[Patches] [ python-Patches-527371 ] Fix for sre bug 470582
noreply@sourceforge.net
noreply@sourceforge.net
Sun, 17 Mar 2002 05:33:43 -0800
Patches item #527371, was opened at 2002-03-08 14:14
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=527371&group_id=5470
Category: None
Group: None
Status: Open
>Resolution: Accepted
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Fredrik Lundh (effbot)
Summary: Fix for sre bug 470582
Initial Comment:
Bug report 470582 points out that nested groups can
produces matches in sre even if the groups within
which they are nested do not match:
>>> m = sre.search(r"^((\d)\:)?(\d\d)\.(\d\d\d)
$", "34.123")
>>> m.groups()
(None, '3', '34', '123')
>>> m = pre.search(r"^((\d)\:)?(\d\d)\.(\d\d\d)
$", "34.123")
>>> m.groups()
(None, None, '34', '123')
I believe this is because in the handling of
SRE_OP_MAX_UNTIL, state->lastmark is being reduced
(after "((\d)\:)" fails) without NULLing out the now-
invalid entries at the end of the state->mark array.
In the other two cases where state->lastmark is
reduced (specifically in SRE_OP_BRANCH and
SRE_OP_REPEAT_ONE) memset is used to NULL out the
entries at the end of the array. The attached patch
does the same thing for the SRE_OP_MAX_UNTIL case.
This fixes the above case and does not break anything
in test_re.py.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-03-08 19:28
Message:
Logged In: YES
user_id=31435
Assigned to /F -- he's the expert here.
----------------------------------------------------------------------
Comment By: Greg Chapman (glchapman)
Date: 2002-03-08 16:23
Message:
Logged In: YES
user_id=86307
I'm pretty sure the memset is correct; state->lastmark is
the index of last mark written to (not the index of the
next potential write).
Also, it occurred to me that there is another related error
here:
>>> m = sre.search(r'^((\d)\:)?\d\d\.\d\d\d$', '34.123')
>>> m.groups()
(None, None)
>>> m.lastindex
2
In other words, lastindex claims that group 2 was the last
that matched, even though it didn't really match. Since
lastindex is undocumented, this probably doesn't matter too
much. Still, it probably should be reset if it is pointing
to a group which gets "unmatched" when state->lastmark is
reduced. Perhaps a function like the following should be
added for use in the three places where state->lastmark is
reset to a previous value:
void lastmark_restore(SRE_STATE *state, int lastmark)
{
assert(lastmark >= 0);
if (state->lastmark > lastmark) {
int lastvalidindex =
(lastmark == 0) ? -1 : (lastmark-1)/2+1;
if (state->lastindex > lastvalidindex)
state->lastindex = lastvalidindex;
memset(
state->mark + lastmark + 1, 0,
(state->lastmark - lastmark) * sizeof(void*)
);
}
state->lastmark = lastmark;
}
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2002-03-08 14:29
Message:
Logged In: YES
user_id=33168
Confirmed that the test w/o fix fails
and the test passes with the fix to _sre.c.
But I'm not sure if the memset can go too far:
memset(state->mark + lastmark + 1, 0,
(state->lastmark - lastmark) * sizeof(void*));
I can try under purify, but that doesn't guarantee anything.
----------------------------------------------------------------------
Comment By: Greg Chapman (glchapman)
Date: 2002-03-08 14:20
Message:
Logged In: YES
user_id=86307
I forgot: here's a patch for re_tests.py which adds the
case from the bug report as a test.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=527371&group_id=5470