namespace/dictionary quandry

[Peter Otten]

Jack Carter wrote:

> Basically this is to go into a tool that accepts
> commandline arguments, one at a time, but will also
> allow scripting for testing purposes. Thus the desire
> to leverage off python for the commandline interface,
> but not for the whole program.
> 
> The input from the user will be either from the console or
> read from a command file. A simple example of what could
> be entered is:
> 
> bosco=5
> if 1:
>   print bosco
>   attach bosco 9
>   bosco=7
>   print bosco
>   attach bosco 9
> 
> The result I would expect would be for my DoAttach()
> routine to receive the python evaluated value of each
> of the arguments leaving the ones that it doesn't understand
> alone. Whether this happens automagically or by hand I don't
> care as long as I get what right value.
> 
> In the above trying to follow your advice, remembering that
> at this stage of the game I am probably missing the point, this
> is the result I expect:
> 
> johmar % demo.py
>>>> bosco=5
>>>> if 1:
> ...   print bosco
> ...   attach bosco 9
> ...   bosco=7
> ...   print bosco
> ...   attach bosco 9
> ...
> 5
> DoAttach: [5, 9]
> 7
> DoAttach: [7, 9]
> 
> This is what I get:
> 
> johmar % demo.py
>>>> bosco=5
>>>> if 1:
> ...   print bosco
> ...   attach bosco 9
> ...   bosco=7
> ...   print bosco
> ...   attach bosco 9
> ...
> 5
> DoAttach: ['bosco', '9']
> 7
> DoAttach: ['bosco', '9']
> 
> Now I realize that this is probably due to the fact that
> I have the lines:
> 
>                 if white_spaces:
>                     line = front_padding + "myparse." + function + "(" +
> str(args) + ")"
>                 else :
>                     line = "myparse." + function + "(" + str(args) + ")"
> 
> which put make the arguments strings, but that is because I don't
> know how to appropriately pack the "line" for later parsing. Maybe
> that is the crux of my problem. Remember, there will be many commands
> for my tool and the arguments will be variable length and this code
> will not know what variables the gentle use would use.
> 
> Here is the simplified code with hopefully the tabs expanded
> base on your earlier input. Hopefully you'll see the obvious
> error of my way and point it out.
> 
> Thanks ever so much,
> 
> Jack
> 
> **********************************
> demo.py
> **********************************
> #!/usr/bin/env python
> 
> import myparse
> 
> cli = myparse.CLI(globals())
> cli.interact()
> 
> **********************************
> myparse.py
> **********************************
> import code
> import re
> import string
> import sys
> 
>
################################################################################
> #
> # DoAttach
> #
> # Dummy function that I will eventually use to do
> # real stuff.
> #
>
################################################################################
> def DoAttach(args):
> 
>     print "DoAttach:", args
>     pass
> 
> 
> class CLI(code.InteractiveConsole):
>     """Simple test of a Python interpreter augmented with custom
>     commands."""
> 
>     commands = { \
>         "attach" : "DoAttach"
>         }
> 
>     def __init__(self, locals = None):
> 
>         # Call super-class initializer
>         code.InteractiveConsole.__init__(self, locals, "<console>")
> 
>         # Compile regular expression for finding commmands
>         self.regexp = re.compile('[a-z]*')
> 
> 
>     ##################################################################
>     #
>     # interact
>     #
>     # This will read and process input lines from within
>     # my main application as though on a python commandline.
>     #
>     ##################################################################
>     def interact(self):
> 
>         # Set the primary and secondary prompts
>         sys.ps1 = ">>> "
>         sys.ps2 = "... "
> 
>         # Input Loop
>         is_more = 0
>         bosco = 0
>         while 1:
>             try :
>                 # Display the appropriate prompt
>                 if not sys.stdin.isatty():
>                     prompt = ""
>                 elif is_more:
>                     prompt = sys.ps2
>                 else:
>                     prompt = sys.ps1
> 
>                 # Read the next line of input
>                 #self.write("interact 1\n")
>                 line = self.raw_input(prompt)
> 
>                 # TODO: add logging of input line here...
> 
>                 # Process complete lines
>                 if 1:
>                    line = self.process(line)
> 
>                 # Push incomplete lines onto input stack
>                 if line or is_more:
>                     is_more = self.push(line)
> 
>             # Handle CTRL-C
>             except KeyboardInterrupt:
>                 self.write("\nKeyboardInterrupt\n")
>                 is_more = 0
>                 self.resetbuffer()
> 
>             # Handle CTRL-D
>             except EOFError:
>                 self.write("\n")
>                 is_more = 0
>                 self.resetbuffer()
>                 raise SystemExit
> 
>     ##################################################################
>     #
>     # process
>     #
>     # This will determine if the input command is either
>     # from my application's command language or a python
>     # construct.
>     #
>     ##################################################################
>     def process(parent, line):
> 
>         # Attempt to match line against our command regular expression
> 
>         temp_line = string.lstrip(line)
>         len_1 = len(line)
>         len_2 = len(temp_line)
> 
>         white_spaces = len_1-len_2
>         if white_spaces:
>             front_padding = line[0:white_spaces]
> 
>         match = parent.regexp.match(temp_line)
>         if match is not None:
> 
>             #parent.write("process 1\n")
>             # Extract the command and argument strings
>             cmd_string = match.group()
>             arg_string = string.lstrip(temp_line[match.end():])
> 
>             # Find the function for this command in the command dictionary
>             function = parent.commands.get(cmd_string)
> 
>             if function is not None:
> 
>                 # Split argument string into individual arguments
>                 args = string.split(arg_string)
> 
>                 # Convert to Python function-call syntax for this command
>                 if white_spaces:
>                     line = front_padding + "myparse." + function + "(" +
> str(args) + ")"
>                 else :
>                     line = "myparse." + function + "(" + str(args) + ")"
> 
          # let's add some feedback
          print "fed to the snake:", line

>         # Return the line to be processed by Python
>         return line


Now

>>> bosco = 1
fed to the snake: bosco = 1
>>> attach bosco 2
fed to the snake: myparse.DoAttach(['bosco', '2'])
DoAttach: ['bosco', '2']
>>>

You build a function call

myparse.DoAttach(['bosco', '2'])

But what you need would rather be

myparse.DoAttach([bosco, 2])

To achieve that you have to somehow extract (Python-compatible) expressions
for the arguments given in the line

attach bosco 2

which can be arbitrarily complex depending on how you defined your custom
language. Assumming that you use the simplest possible spec, a
space-separated list of already Python-compatible expressions that gives
you

def parseArgs(args):
    return args.split()

def makePythonCall(func, args):
    return "%s([%s])" % (func, ", ".join(args))

and the process() method will become:

    def process(parent, line):

        # Attempt to match line against our command regular expression

        temp_line = string.lstrip(line)
        len_1 = len(line)
        len_2 = len(temp_line)

        white_spaces = len_1-len_2
        if white_spaces:
            front_padding = line[0:white_spaces]

        match = parent.regexp.match(temp_line)
        if match is not None:

            #parent.write("process 1\n")
            # Extract the command and argument strings
            cmd_string = match.group()
            arg_string = string.lstrip(temp_line[match.end():])

            # Find the function for this command in the command dictionary
            function = parent.commands.get(cmd_string)

            if function is not None:

                args = parseArgs(arg_string)
                line = makePythonCall("myparse." + function, args)
                if white_spaces:
                    line = front_padding + line
        print "fed to the snake:", line

        # Return the line to be processed by Python
        return line


Testing it:

>>> bosco = 1
fed to the snake: bosco = 1
>>> attach bosco 2
fed to the snake: myparse.DoAttach([bosco, 2])
DoAttach: [1, 2]
>>> if 1:
fed to the snake: if 1:
...     attach bosco
fed to the snake:       myparse.DoAttach([bosco])
...     bosco = 3
fed to the snake:       bosco = 3
...     attach bosco
fed to the snake:       myparse.DoAttach([bosco])
...
fed to the snake:
DoAttach: [1]
DoAttach: [3]

Works here, but is not very robust:

 >>> attach = 99
fed to the snake: myparse.DoAttach([=, 99])
  File "<console>", line 1
    myparse.DoAttach([=, 99])
                      ^
SyntaxError: invalid syntax
>>> attach(bosco)
fed to the snake: myparse.DoAttach([(bosco)])
DoAttach: [3]
>>> attach bosco "so what"
fed to the snake: myparse.DoAttach([bosco, "so, what"])
DoAttach: [3, 'so, what'] # note the comma

Peter