should sre.Scanner be exposed through re and documented?
While moving tests from test_sre to test_re I stumbled upon a simple test for sre.Scanner. This looks fairly cool. Should it be exposed through re and documented? Skip
Skip Montanaro wrote:
While moving tests from test_sre to test_re I stumbled upon a simple test for sre.Scanner. This looks fairly cool. Should it be exposed through re and documented?
I think /F did not consider it ready for general consumption. I believe the approach is cool, but the API would still leave features to be desired. In practical compiler construction, I usually copy the approach, and duplicate it - that gives a very efficient and readily comprehensible scanner. IOW, I would leave it where it is: As a masterpiece of work to get inspiration from, but not as a tool to give out to anybody. Regards, Martin
While moving tests from test_sre to test_re I stumbled upon a simple test for sre.Scanner. This looks fairly cool. Should it be exposed through re and documented?
What's Scanner? --Guido van Rossum (home page: http://www.python.org/~guido/)
On Friday 25 April 2003 12:38 pm, Guido van Rossum wrote:
While moving tests from test_sre to test_re I stumbled upon a simple test for sre.Scanner. This looks fairly cool. Should it be exposed through re and documented?
What's Scanner?
You create a Scanner instance with a list of re's and associated functions, then you use it to scan a string, returning a list of parts which match the given re's. (Actually the matches are run through the associated functions, and their output is what forms the returned list.) Here's the single test case Skip refereed to: def s_ident(scanner, token): return token def s_operator(scanner, token): return "op%s" % token def s_float(scanner, token): return float(token) def s_int(scanner, token): return int(token) scanner = sre.Scanner([ (r"[a-zA-Z_]\w*", s_ident), (r"\d+\.\d*", s_float), (r"\d+", s_int), (r"=|\+|-|\*|/", s_operator), (r"\s+", None), ]) # sanity check test('scanner.scan("sum = 3*foo + 312.50 + bar")', (['sum', 'op=', 3, 'op*', 'foo', 'op+', 312.5, 'op+', 'bar'], '')) Gary Herron
Guido> What's Scanner? Gary already posted the example I was going to (damn phone!)... ;-) I defer to Martin's judgement on this. (I presume his response has passed through your mailbox by now.) I still think it would be nice to demonstrate it somewhere. I'll look and see if there's somewhere some toy script can be squeezed into the Demo directory. Skip
participants (4)
-
"Martin v. Löwis" -
Gary Herron -
Guido van Rossum -
Skip Montanaro