cmd, readline, and rlcompleter problems

[Peter Kazmier]

I need help debugging a very strange problem.  I'm not a python expert
so it could be something very simple, on the other hand, I did use the
debugger already, but the problem disappears miraculously when I use
the debugger.  It makes no sense to me.

So, I'm basically trying to write my own cmd and rlcompleter.  I've
attached both the module and a small example of its use.  Its almost
identical to the standard cmd however I am trying to offer command
line completion.  

For example, if I have the following commands defined: do_quit and
do_show_version.  You should be able to type just 'qu' and be able to
hit TAB and have it complete to 'quit'.  That works fine of course.
Here is the problem, if a user types 'show ver' it SHOULD complete to
'show version'; however, the readline.completer function only seems to
pass in the current word that needs completing, in this case, 'ver'.
I want to complete based on 'show ver'.  It shouldn't be a problem
because readline provides a function called get_line_buffer() which
returns the contents of the line buffer.  So here's my completer
function: 

    def complete(self, text, state):

        if state == 0:
	    # Without this line, it would only complete on the 
	    # current word, in the above example, that would have
	    # been 'ver'.  But adding this, I should be able to 
	    # complete on the full line, or 'show ver'.
	    text = readline.get_line_buffer()
            self.matches = self.global_matches(text)
        try:
            return self.matches[state]
        except IndexError:
            return None

    # Find the matches, basically, look through this instance's class
    # for any attribute that begins with 'do_', for each attribute
    # that matches, convert underscores to spaces and ditch the 'do'
    # component.  This provides a list of possible completions.
    def global_matches(self, text):
        matches = []
        n = len(text)

        print "DEBUG text =", text
        for word in self.__class__.__dict__.keys():
            if word[0:3] == 'do_':
                word = string.join(string.split(word, '_')[1:], ' ')
                print "DEBUG", word
                if word[:n] == text:
                    matches.append(word)

        return matches

Everything works fine IF I comment out the get_line_buffer() line.
Unfortunately, that would only do completion on the current word.
When I uncomment the get_line_buffer() line, everything seems to work,
but during the 'for word in self.__class__.__dict__.keys()' loop, half
of the keys mysteriously disappear!  The DEBUG output, shows only some
of the keys, not all of them.  Comment out the get_line_buffer() line
and all of the keys() are back!  When I stepped through this in the
debugger, all of the keys() were also there!!  It appears that the
get_line_buffer() call is somehow munging my environment in a weird
way.  Oddly enough, whwen debugging, his munging does not appear.

Could someone PLEASE help me debug this??  I have no idea what I am
doing wrong and the symptoms are like none I've ever seen.  I've
attached two files, cli.py which is my cmd.py replacement, and
shell.py which is a simple test shell.  Run shell.py, and type in 'sh'
and hit TAB, it won't work.  But comment out that get_line_buffer()
line and then do the same.  Viola, it works.  But how come and how do
I get it to work the other way??

Thanks,
Pete

============
shell.py
------------
#!/usr/bin/python

import cli

class CLI(cli.cli):
    def do_quit(self, args):
        """Exit the command interpreter"""
        return 1

    def do_show(self, args):
        """Display information on various components"""
        print "Show function was called:", args
        return None

    def do_show_version(self, args):
        """Display the version of the shell"""
        print "Version is ..."

    def do_show_version_cli(self, args):
        """Display the version of the CLI"""
        print "CLI version is ..."

CLI().cmdloop()

============
cli.py
------------
import string
import readline

# Modified version of cmd.py

class cli:
    prompt = '# '
    
    def __init__(self):
        readline.parse_and_bind("tab: complete")
        readline.set_completer(self.complete)

    def cmdloop(self):
        stop = None
        while not stop:
            try:
                line = raw_input(self.prompt)
            except EOFError:
                print 
                break

            line = self.precmd(line)
            stop = self.onecmd(line)
            stop = self.postcmd(stop, line)

        self.postloop()

    def postloop(self):
        pass

    def precmd(self, line):
        return line

    def postcmd(self, stop, line):
        return stop

    def onecmd(self, line):
        line = string.strip(line)
        if not line:
            return None
        tokens = string.split(line)

        help = 0
        if tokens[len(tokens)-1] == 'help':
            help = 1
            tokens.pop()

        func = None
        for i in range(len(tokens), 0, -1):
            try:
                func = getattr(self, 'do_' + string.join(tokens[:i], '_'))
                args = tokens[i:]
            except AttributeError:
                func = None
            else:
                break

        if help:
            self.do_help(func)
        elif func:
            return func(args)

        return None
        
    def do_help(self, func):
        """Welcome to the help system"""
        if func:
            if func.__doc__:
                print func.__doc__
            else:
                print "No help available"
        else:
            print "No help available"

    def complete(self, text, state):
        
        if state == 0:
            text = readline.get_line_buffer()
            self.matches = self.global_matches(text)
        try:
            return self.matches[state]
        except IndexError:
            return None

    def global_matches(self, text):

        matches = []
        n = len(text)

        print "DEBUG text =", text
        for word in self.__class__.__dict__.keys():
            if word[0:3] == 'do_':
                word = string.join(string.split(word, '_')[1:], ' ')
                print "DEBUG", word
                if word[:n] == text:
                    matches.append(word)

        return matches