embedding executable code in a regular expression in Python

[Anthra Norell]

Hi, see below ...

----- Original Message -----
From: "Paul McGuire" <ptmcg at austin.rr.com>
Newsgroups: comp.lang.python
To: <python-list at python.org>
Sent: Monday, July 17, 2006 10:09 AM
Subject: Re: embedding executable code in a regular expression in Python


> Avi Kak wrote:
> > Folks,
> >
> > Does regular expression processing in Python allow for executable
> > code to be embedded inside a regular expression?
> >
> > For example, in Perl the following two statements
> >
> > $regex = qr/hello(?{print "saw hello\n"})mello(?{print "saw
> > mello\n"})/;
> > "jellohellomello"  =~  /$regex/;
> >
> > will produce the output
> >
> >   saw hello
> >   saw mello
> >
>
> Not nearly so terse, but perhaps easier to follow, here is a pyparsing
> version.  Pyparsing parse actions are intended to do just what you ask.
>  Parse actions may be defined to take no arguments, just one argument
> (which will be passed the list of matching token strings), 2 arguments
> (the match location and the matching tokens), or 3 arguments (the
> original source string, the match location, and the tokens).  Parse
> actions are very good for transforming input text into modified output
> form, such as the "background-color" to "backgroundColor" transform -
> the BoaConstructor team used pyparsing to implement a version upgrade
> that transformed user source to a new version of wx (involving a
> variety of suh changes).
>
> Here is your jello/mello program, with two variations of parse actions.
>
> -- Paul
>
> from pyparsing import *
>
> instr = "jellorelohellomellofellowbellowmello"
> searchTerm = oneOf( ["jello","mello"] )
>
> # simple parse action, just echoes matched text
> def echoMatchedText(tokens):
>     print "saw", tokens[0]
>
> searchTerm.setParseAction( echoMatchedText )
> searchTerm.searchString(instr)
>
> # modified parse action, prints location too
> def echoMatchedText(loc,tokens):
>     print "saw", tokens[0], "at locn", loc
>
> searchTerm.setParseAction( echoMatchedText )
> searchTerm.searchString(instr)
>
> Prints out:
> saw jello
> saw mello
> saw mello
> saw jello at locn 0
> saw mello at locn 14
> saw mello at locn 31
>
> --
> http://mail.python.org/mailman/listinfo/python-list

One fine example of a pyparse solution!

On my part I had a second thought on the SE solution I proposed. It was
needlessly complicated. Let me try again:

>>> sentence = 'On each occurrence of the word "square" in this text the
variable n will be squared. When it says "double" it will be doubled. At the
end print n % 11.'
>>> def square_n (): global n; n *= n
>>> def double_n (): global n; n += n
>>> def n_mod_11 (): global n; return n % 11
>>> se_definitions = """<EAT>
...  "square=print 'Calling square_n () - ',; square_n (); print 'n is now
%d' % n \n"   # A piece of code for 'square'
 ... "double=print 'Calling double_n () - ',; double_n (); print 'n is now
%d' % n \n"   # Another piece of code for 'double'
 ... ".=\nprint 'n %% 11 is: %d' % n_mod_11 ()\n"   # Another piece of code
for the final dot
... """
>>> from SE import *
>>> Stream_Editor = SE (definitions)
>>> n = 9; exec (Stream_Editor (sentence))
Calling square_n () -  n is now 81
Calling square_n () -  n is now 6561
n % 11 is: 5
Calling double_n () -  n is now 13122
Calling double_n () -  n is now 26244
n % 11 is: 9
n % 11 is: 9

Suppose we now realize that the quoted words "square" and "double" should
not trigger the respective function and neither should the ending dot of
each sentence, except the last one. We fix the defintions in seconds, adding
the exceptions as deletions. The order is immaterial. Targets may be regular
expressions. (The last one is.)

se_definitions = """<EAT>     # Deletes everything except the defined
substitutions
...  "square=print 'Calling square_n () - ',; square_n (); print 'n is now
%d' % n \n"   # A piece of code for 'square'
... "double=print 'Calling double_n () - ',; double_n (); print 'n is now
%d' % n \n"   # Another piece of code for 'double'
... ".=\nprint 'n %% 11 is: %d' % n_mod_11 ()\n"   # Another piece of code
for the final dot
... \""square"=" \""double"="  # Deletes the quoted words
... "~\.\s~="  # Deletes dots followed by white space.
... """
>>> n = 9; exec (SE (se_definitions)(sentence))
Calling square_n () -  n is now 81
Calling double_n () -  n is now 162
n % 11 is: 8

Note 1: SE is no match for pyparse on many accounts. Conversely SE has its
own strengths in the realm of its specialty, in particular ease and speed of
use, versatility and modularity.
      Most any problem can be solved in a variety of different ways. Finding
good approaches is one of the appeals of programming.

Note 2: The SE solution rests entirely on Python's 'exec' functionality and
so here we have another testimony to Python's excellence.

Note 3: I shall include the example in the manual. I hadn't thought of it.
My thanks to the OP for the inspiration. Perhaps he might want to explain
his purpose. I don't quite see the solution's problem.

Frederic