[Spambayes-checkins] spambayes/spambayes OptionsClass.py, NONE, 1.1 Options.py, 1.54, 1.55

Mark Hammond mhammond at users.sourceforge.net
Sun Jun 15 21:46:15 EDT 2003


Update of /cvsroot/spambayes/spambayes/spambayes
In directory sc8-pr-cvs1:/tmp/cvs-serv5044

Modified Files:
	Options.py 
Added Files:
	OptionsClass.py 
Log Message:
Split the class into its own file, so it is possible to use other
instances *without* loading the default SpamBayes options.
(Outlook now uses this file format, but stores its UI options in
a different file).

Allow lists to be used for multi-valued options (previously only
tuples were supported.)

Allow the first entry in the "defaults" table to be a sub-class of
Option - this allows custom packing/unpacking - currently only
used by Outlook to manage folder IDs (which are a list of tuples)


--- NEW FILE: OptionsClass.py ---
"""OptionsCore

Classes:
    Option - Holds information about an option
    OptionsClass - A collection of options

Abstract:

This module is used to manage "options" managed in user editable files.
This is the implementation of the Options.options globally shared options
object for the SpamBayes project, but is also able to be used to manage
other options required by each application.

The Option class holds information about an option - the name of the
option, a nice name (to display), documentation, default value,
possible values (a tuple or a regex pattern), whether multiple values
are allowed, and whether the option should be reset when restoring to
defaults (options like server names should *not* be).

The OptionsClass class provides facility for a collection of Options.
It is expected that manipulation of the options will be carried out
via an instance of this class.

To Do:
 o Get rid of the really ugly backwards compatability code (that adds
   many, many attributes to the options object) as soon as all the
   modules are changed over.
 o Once the above is done, and we have waited a suitable time, stop
   allowing invalid options in configuration files
 o Find a regex expert to come up with *good* patterns for domains,
   email addresses, and so forth.
 o str(Option) should really call Option.unconvert since this is what
   it does.  Try putting that in and running all the tests.
 o [See also the __issues__ string.]
 o Suggestions?

"""
# This module is part of the spambayes project, which is Copyright 2002-3
# The Python Software Foundation and is covered by the Python Software
# Foundation license.

__credits__ = "All the Spambayes folk."
# blame for the new format: Tony Meyer <ta-meyer at ihug.co.nz>

__issues__ = """Things that should be considered further and by
other people:

We are very generous in checking validity when multiple values are
allowed and the check is a regex (rather than a tuple).  Any sequence
that does not match the regex may be used to delimit the values.
For example, if the regex was simply r"[\d]*" then these would all
be considered valid:
"123a234" -> 123, 234
"123abced234" -> 123, 234
"123XST234xas" -> 123, 234
"123 234" -> 123, 234
"123~!@$%^&@234!" -> 123, 234

If this is a problem, my recommendation would be to change the
multiple_values_allowed attribute from a boolean to a regex/None
i.e. if multiple is None, then only one value is allowed.  Otherwise
multiple is used in a re.split() to separate the input.
"""

import sys
import os
import shutil
from tempfile import TemporaryFile

try:
    import cStringIO as StringIO
except ImportError:
    import StringIO

import re
import types
import locale

try:
    True, False, bool
except NameError:
    # Maintain compatibility with Python 2.2
    True, False = 1, 0
    def bool(val):
        return not not val

__all__ = ['OptionsClass',
           'HEADER_NAME', 'HEADER_VALUE',
           'INTEGER', 'REAL', 'BOOLEAN',
           'SERVER', 'PORT', 'EMAIL_ADDRESS',
           'PATH', 'VARIABLE_PATH', 'FILE', 'FILE_WITH_PATH',
           'IMAP_FOLDER', 'IMAP_ASTRING',
           'RESTORE', 'DO_NOT_RESTORE',
          ]

MultiContainerTypes = (types.TupleType, types.ListType)

class Option(object):
    def __init__(self, name, nice_name="", default=None,
                 help_text="", allowed=None, restore=True):
        self.name = name
        self.nice_name = nice_name
        self.default_value = default
        self.explanation_text = help_text
        self.allowed_values = allowed
        self.restore = restore
        self.delimiter = None
        # start with default value
        self.set(default)

    def display_name(self):
        '''A name for the option suitable for display to a user.'''
        return self.nice_name
    def default(self):
        '''The default value for the option.'''
        return self.default_value
    def doc(self):
        '''Documentation for the option.'''
        return self.explanation_text
    def valid_input(self):
        '''Valid values for the option.'''
        return self.allowed_values
    def no_restore(self):
        '''Do not restore this option when restoring to defaults.'''
        return not self.restore
    def set(self, val):
        '''Set option to value.'''
        self.value = val
    def get(self):
        '''Get option value.'''
        return self.value
    def multiple_values_allowed(self):
        '''Multiple values are allowed for this option.'''
        return type(self.value) in MultiContainerTypes
    
    def is_valid(self, value):
        '''Check if this is a valid value for this option.'''
# XXX This test is in the original code, but makes no sense....
# XXX self.allowed_values defaults to None, and if that is the
# XXX current value, then whatever is passed would be invalid
# XXX I agree this is a silly state to be in, but it is possible
# XXX I suppose that self.allowed_values should default to *any*
# XXX rather than None, but I'm not sure how to express that,
# XXX unless the regex r"." is correct.
        if self.allowed_values is None:
            return False

        if type(self.value) in MultiContainerTypes:
            return self.is_valid_multiple(value)
        else:
            return self.is_valid_single(value)

    def is_valid_multiple(self, value):
        '''Return True iff value is a valid value for this option.
        Use if multiple values are allowed.'''
        if type(value) in MultiContainerTypes:
            for val in value:
                if not self.is_valid_single(val):
                    return False
            return True
        return self.is_valid_single(value)

    def is_valid_single(self, value):
        '''Return True iff value is a valid value for this option.
        Use when multiple values are not allowed.'''
        if type(self.allowed_values) == types.TupleType:
            if value in self.allowed_values:
                return True
            else:
                return False
        else:
            # special handling for booleans, thanks to Python 2.2
            if self.is_boolean and (value == True or value == False):
                return True
            if type(value) != type(self.value) and \
               type(self.value) not in MultiContainerTypes:
                # This is very strict!  If the value is meant to be
                # a real number and an integer is passed in, it will fail.
                # (So pass 1. instead of 1, for example)
                return False
            avals = self._split_values(value)
            # in this case, allowed_values must be a regex, and
            # _split_values must match once and only once
            if len(avals) == 1:
                return True
            else:
                # either no match or too many matches
                return False
            
    def _split_values(self, value):
        # do the regex mojo here
        try:
            r = re.compile(self.allowed_values)
        except:
            print self.allowed_values
            raise
        s = str(value)
        i = 0
        vals = ()
        while True:
            m = r.search(s[i:])
            if m is None:
                break
            vals += (m.group(),)
            delimiter = s[i:i + m.start()]
            if self.delimiter is None and delimiter != "":
                self.delimiter = delimiter
            i += m.end()
        return vals

    def as_nice_string(self, section=None):
        '''Summarise the option in a user-readable format.'''
        if section is None:
            strval = ""
        else:
            strval = "[%s] " % (section)
        strval += "%s - \"%s\"\nDefault: %s\nDo not restore: %s\n" \
                 % (self.name, self.display_name(),
                    str(self.default()), str(self.no_restore()))
        strval += "Valid values: %s\nMultiple values allowed: %s\n" \
                  % (str(self.valid_input()),
                     str(self.multiple_values_allowed()))
        strval += "\"%s\"\n\n" % (str(self.doc()))
        return strval

    def write_config(self, file):
        '''Output value in configuration file format.'''
        file.write(self.name)
        file.write(': ')
        file.write(self.unconvert())
        file.write('\n')

    def convert(self, value):
        '''Convert value from a string to the appropriate type.'''
        svt = type(self.value)
        if svt == type(value):
            # already the correct type
            return value
        if self.is_boolean():
            if str(value) == "True" or value == 1:
                return True
            elif str(value) == "False" or value == 0:
                return False
            raise TypeError, self.name + " must be True or False"
        if self.multiple_values_allowed():
            # This will fall apart if the allowed_value is a tuple,
            # but not a homogenous one...
            if type(self.allowed_values) in types.StringTypes:
                vals = list(self._split_values(value))
            else:
                vals = value.split()
            if len(self.default_value) > 0:
                to_type = type(self.default_value[0])
            else:
                to_type = types.StringType
            for i in range(0, len(vals)):
                vals[i] = self._convert(vals[i], to_type)
            return tuple(vals)
        else:
            return self._convert(value, svt)
        raise TypeError, self.name + " has an invalid type."

    def _convert(self, value, to_type):
        '''Convert an int, float or string to the specified type.'''
        if to_type == type(value):
            # already the correct type
            return value
        if to_type == types.IntType:
            return locale.atoi(value)
        if to_type == types.FloatType:
            return locale.atof(value)
        if to_type in types.StringTypes:
            return str(value)
        raise TypeError, "Invalid type."

    def unconvert(self):
        '''Convert value from the appropriate type to a string.'''
        if type(self.value) in types.StringTypes:
            # nothing to do
            return self.value
        if self.is_boolean():
            # A wee bit extra for Python 2.2
            if self.value == True:
                return "True"
            else:
                return "False"
        if type(self.value) == types.TupleType:
            if len(self.value) == 0:
                return ""
            if len(self.value) == 1:
                v = self.value[0]
                if type(v) == types.FloatType:
                    return locale.str(self.value[0])
                return str(v)
            # We need to separate out the items
            strval = ""
            # We use a character that is invalid as the separator
            # so that it will reparse correctly.  We could try all
            # characters, but we make do with this set of commonly
            # used ones - note that the first one that works will
            # be used.  Perhaps a nicer solution than this would be
            # to specifiy a valid delimiter for all options that
            # can have multiple values.  Note that we have None at
            # the end so that this will crash and die if none of
            # the separators works <wink>.
            if self.delimiter is None:
                if type(self.allowed_values) == types.TupleType:
                    self.delimiter = ' '
                else:
                    v0 = self.value[0]
                    v1 = self.value[1]
                    for sep in [' ', ',', ':', ';', '/', '\\', None]:
                        # we know at this point that len(self.value) is at
                        # least two, because len==0 and len==1 were dealt
                        # with as special cases
                        test_str = str(v0) + sep + str(v1)
                        test_tuple = self._split_values(test_str)
                        if test_tuple[0] == str(v0) and \
                           test_tuple[1] == str(v1) and \
                           len(test_tuple) == 2:
                            break
                    # cache this so we don't always need to do the above
                    self.delimiter = sep
            for v in self.value:
                if type(v) == types.FloatType:
                    v = locale.str(v)
                else:
                    v = str(v)
                strval += v + self.delimiter
            strval = strval[:-len(self.delimiter)] # trailing seperator
        else:
            # Otherwise, we just hope str() will do the job
            strval = str(self.value)
        return strval

    def is_boolean(self):
        '''Return True iff the option is a boolean value.'''
        # This is necessary because of the Python 2.2 True=1, False=0
        # cheat.  The valid values are returned as 0 and 1, even if
        # they are actually False and True - but 0 and 1 are not
        # considered valid input (and 0 and 1 don't look as nice)
        # So, just for the 2.2 people, we have this helper function
        try:
            if type(self.allowed_values) == types.TupleType and \
               len(self.allowed_values) > 0 and \
               type(self.allowed_values[0]) == types.BooleanType:
                return True
            return False
        except AttributeError:
            # If the user has Python 2.2 and an option has valid values
            # of (0, 1) - i.e. integers, then this function will return
            # the wrong value.  I don't know what to do about that without
            # explicitly stating which options are boolean
            if self.allowed_values == (False, True):
                return True
            return False


class OptionsClass(object):
    def __init__(self):
        self.verbose = None
        self._options = {}
        self.conversion_table = {} # set by creator if they need it.
    #
    # Regular expressions for parsing section headers and options.
    # Lifted straight from ConfigParser
    #
    SECTCRE = re.compile(
        r'\['                                 # [
        r'(?P<header>[^]]+)'                  # very permissive!
        r'\]'                                 # ]
        )
    OPTCRE = re.compile(
        r'(?P<option>[^:=\s][^:=]*)'          # very permissive!
        r'\s*(?P<vi>[:=])\s*'                 # any number of space/tab,
                                              # followed by separator
                                              # (either : or =), followed
                                              # by any # space/tab
        r'(?P<value>.*)$'                     # everything up to EOL
        )

    def update_file(self, filename):
        '''Update the specified configuration file.'''
        sectname = None
        optname = None
        out = TemporaryFile()
        if os.path.exists(filename):
            f = file(filename, "r")
        else:
            # doesn't exist, so create it - all the changed options will
            # be added to it
            if self.verbose:
                print "Creating new configuration file", filename
            f = file(filename, "w")
            f.close()
            f = file(filename, "r")
        written = []
        vi = ": " # default; uses the one from the file where possible
        while True:
            line = f.readline()
            if not line:
                break
            # comment or blank line?
            if line.strip() == '' or line[0] in '#;':
                out.write(line)
                continue
            if line.split(None, 1)[0].lower() == 'rem' and line[0] in "rR":
                # no leading whitespace
                out.write(line)
                continue
            # continuation line?
            if line[0].isspace() and sectname is not None and optname:
                continue
            # a section header or option header?
            else:
                # is it a section header?
                mo = self.SECTCRE.match(line)
                if mo:
                    # Add any missing from the previous section
                    if sectname is not None:
                        self._add_missing(out, written, sectname, vi, False)
                    sectname = mo.group('header')
                    # So sections can't start with a continuation line
                    optname = None
                    if sectname in self.sections():
                        out.write(line)
                # an option line?
                else:
                    mo = self.OPTCRE.match(line)
                    if mo:
                        optname, vi, optval = mo.group('option', 'vi', 'value')
                        if vi in ('=', ':') and ';' in optval:
                            # ';' is a comment delimiter only if it follows
                            # a spacing character
                            pos = optval.find(';')
                            if pos != -1 and optval[pos-1].isspace():
                                optval = optval[:pos]
                        optval = optval.strip()
                        # allow empty values
                        if optval == '""':
                            optval = ''
                        optname = optname.rstrip().lower()
                        if self._options.has_key((sectname, optname)):
                            out.write(optname)
                            out.write(vi)
                            out.write(self.unconvert(sectname, optname))
                            out.write('\n')
                            written.append((sectname, optname))
        for sect in self.sections():
            self._add_missing(out, written, sect, vi)
        f.close()
        out.flush()
        if self.verbose:
            # save a backup of the old file
            shutil.copyfile(filename, filename + ".bak")
        # copy the new file across
        f = file(filename, "w")
        out.seek(0)
        shutil.copyfileobj(out, f)
        out.close()
        f.close()
    
    def _add_missing(self, out, written, sect, vi, label=True):
        # add any missing ones, where the value does not equal the default
        for opt in self.options_in_section(sect):
            if not (sect, opt) in written and \
               self.get(sect, opt) != self.default(sect, opt):
                if label:
                    out.write('[')
                    out.write(sect)
                    out.write("]\n")
                    label = False
                out.write(opt)
                out.write(vi)
                out.write(self.unconvert(sect, opt))
                out.write('\n')
                written.append((sect, opt))

    def load_defaults(self, defaults):
        '''Load default values (stored in this module).'''
        for section, opts in defaults.items():
            for opt in opts:
                # If first item of the tuple is a sub-class of Option, then
                # instantiate that (with the rest as args).  Otherwise,
                # assume standard Options class.
                klass = Option
                args = opt
                try:
                    if issubclass(opt[0], Option):
                        klass = opt[0]
                        args = opt[1:]
                except TypeError: # opt[0] not a class
                    pass
                        
                o = klass(*args)
                self._options[section, o.name] = o
                # A (really ugly) bit of backwards compatability
                # *** This will vanish soon, so do not make use of it in
                #     new code ***
                self._oldset(section, o.name, o.value)

    def _oldset(self, section, option, value):
        # A (really ugly) bit of backwards compatability
        # *** This will vanish soon, so do not make use of it in
        #     new code ***
        for (oldsect, oldopt), (newsect, newopt) in self.conversion_table.items():
            if (newsect, newopt) == (section, option):
                section = oldsect
                option = oldopt
        setattr(self, option, value)
        old_name = section[0:1].lower() + section[1:] + "_" + option
        setattr(self, old_name, value)
                
    def merge_files(self, file_list):
        for file in file_list:
            self.merge_file(file)

    def merge_file(self, filename):
        import ConfigParser
        c = ConfigParser.ConfigParser()
        c.read(filename)
        for sect in c.sections():
            for opt in c.options(sect):
                value = c.get(sect, opt)
                section = sect
                option = opt
                # backward compatibility guff, but only if needed.
                if self.conversion_table:
                    if opt[:len(sect) + 1].lower() == sect.lower() + '_':
                        opt = opt[len(sect)+1:]
                    if self.conversion_table.has_key((sect, opt)):
                        section, option = self.conversion_table[sect, opt]
                    else:
                        section = sect
                        option = opt
                else: # no b/w compat needed.
                    section = sect
                    option = opt
                # end of backward compatibility guff
                if not self._options.has_key((section, option)):
                    print "Invalid option %s in section %s in file %s" % \
                          (opt, sect, filename)
                else:
                    if self.multiple_values_allowed(section, option):
                        value = self.convert(section, option, value)
                    value = self.convert(section, option, value)
                    self.set(section, option, value)
                    # backward compatibility guff
                    if self.conversion_table:
                        self._oldset(sect, opt, value)
                    # end of backward compatibility guff

    # not strictly necessary, but convenient shortcuts to self._options
    def display_name(self, sect, opt):
        '''A name for the option suitable for display to a user.'''
        return self._options[sect, opt].display_name()
    def default(self, sect, opt):
        '''The default value for the option.'''
        return self._options[sect, opt].default()
    def doc(self, sect, opt):
        '''Documentation for the option.'''
        return self._options[sect, opt].doc()
    def valid_input(self, sect, opt):
        '''Valid values for the option.'''
        return self._options[sect, opt].valid_input()
    def no_restore(self, sect, opt):
        '''Do not restore this option when restoring to defaults.'''
        return self._options[sect, opt].no_restore()
    def is_valid(self, sect, opt, value):
        '''Check if this is a valid value for this option.'''
        return self._options[sect, opt].is_valid(value)
    def multiple_values_allowed(self, sect, opt):
        '''Multiple values are allowed for this option.'''
        return self._options[sect, opt].multiple_values_allowed()

    def is_boolean(self, sect, opt):
        '''The option is a boolean value. (Support for Python 2.2).'''
        return self._options[sect, opt].is_boolean()

    def convert(self, sect, opt, value):
        '''Convert value from a string to the appropriate type.'''
        return self._options[sect, opt].convert(value)

    def unconvert(self, sect, opt):
        '''Convert value from the appropriate type to a string.'''
        return self._options[sect, opt].unconvert()

    def get(self, sect, opt):
        '''Get an option.'''
        if self.conversion_table.has_key((sect, opt)):
            sect, opt = self.conversion_table[sect, opt]
        return self._options[sect, opt].get()

    def __getitem__(self, key):
        return self.get(key[0], key[1])

    def set(self, sect, opt, val=None):
        '''Set an option.'''
        if self.conversion_table.has_key((sect, opt)):
            sect, opt = self.conversion_table[sect, opt]
        if self.is_valid(sect, opt, val):
            self._options[sect, opt].set(val)
            # backwards compatibility stuff
            self._oldset(sect, opt, val)
        else:
            print "Attempted to set [%s] %s with invalid value %s (%s)" % \
                  (sect, opt, val, type(val))
        
    def __setitem__(self, key, value):
        self.set(key[0], key[1], value)

    def sections(self):
        '''Return an alphabetical list of all the sections.'''
        all = []
        for sect, opt in self._options.keys():
            if sect not in all:
                all.append(sect)
        all.sort()
        return all
    
    def options_in_section(self, section):
        '''Return an alphabetical list of all the options in this section.'''
        all = []
        for sect, opt in self._options.keys():
            if sect == section:
                all.append(opt)
        all.sort()
        return all

    def options(self, prepend_section_name=False):
        '''Return a alphabetical list of all the options, optionally
        prefixed with [section_name]'''
        all = []
        for sect, opt in self._options.keys():
            if prepend_section_name:
                all.append('[' + sect + ']' + opt)
            else:
                all.append(opt)
        all.sort()
        return all

    def display(self):
        '''Display options in a config file form.'''
        output = StringIO.StringIO()
        keys = self._options.keys()
        keys.sort()
        currentSection = None
        for sect, opt in keys:
            if sect != currentSection:
                if currentSection is not None:
                    output.write('\n')
                output.write('[')
                output.write(sect)
                output.write("]\n")
                currentSection = sect
            self._options[sect, opt].write_config(output)
        return output.getvalue()

    def display_full(self, section=None, option=None):
       '''Display options including all information.'''
       # Given that the Options class is no longer as nice looking
       # as it once was, this returns all the information, i.e.
       # the doc, default values, and so on
       output = StringIO.StringIO()

       # when section and option are both specified, this
       # is nothing more than a call to as_nice_string
       if section is not None and option is not None:
           output.write(self._options[section,
                                      option].as_nice_string(section))
           return output.getvalue()
       
       all = self._options.keys()
       all.sort()
       for sect, opt in all:
           if section is not None and sect != section:
               continue
           output.write(self._options[sect, opt].as_nice_string(sect))
       return output.getvalue()

# These are handy references to commonly used regex/tuples defining
# permitted values. Although the majority of options use one of these,
# you may use any regex or tuple you wish.
HEADER_NAME = r"[\w\.\-\*]+"
HEADER_VALUE = r"[\w\.\-\*]+"
INTEGER = r"[\d]+"              # actually, a *positive* integer
REAL = r"[\d]+[\.]?[\d]*"       # likewise, a *positive* real
BOOLEAN = (False, True)
SERVER = r"([\w\.\-]+(:[\d]+)?)"  # in the form server:port
PORT = r"[\d]+"
EMAIL_ADDRESS = r"[\w\-\.]+@[\w\-\.]+"
PATH = r"[\w\.\-~:\\/\*]+"
VARIABLE_PATH = PATH + r"%"
FILE = r"[\S]+"
FILE_WITH_PATH = PATH
# IMAP seems to allow any character at all in a folder name,
# but we want to use the comma as a delimiter for lists, so
# we don't allow this.  If anyone has folders with commas in the
# names, please let us know and we'll figure out something else.
# ImapUI.py prints out a warning if this is the case.
IMAP_FOLDER = r"[^,]+"

# IMAP's astring should also be valid in the form:
#   "{" number "}" CRLF *CHAR8
#   where number represents the number of CHAR8 octets
# but this is too complex for us at the moment.
IMAP_ASTRING = ""
for i in range(1, 128):
    if not chr(i) in ['"', '\\', '\n', '\r']:
        IMAP_ASTRING += chr(i)
IMAP_ASTRING = r"\"?\\?[" + re.escape(IMAP_ASTRING) + r"]+\"?"

# Similarly, each option must specify whether it should be reset to
# this value on a "reset to defaults" command.  Most should, but with some
# like a server name that defaults to "", this would be pointless.
# Again, for ease of reading, we define these here:
RESTORE = True
DO_NOT_RESTORE = False

Index: Options.py
===================================================================
RCS file: /cvsroot/spambayes/spambayes/spambayes/Options.py,v
retrieving revision 1.54
retrieving revision 1.55
diff -C2 -d -r1.54 -r1.55
*** Options.py	26 May 2003 00:00:53 -0000	1.54
--- Options.py	16 Jun 2003 03:46:13 -0000	1.55
***************
*** 1,8 ****
  """Options
  
- Classes:
-     Option - Holds information about an option
-     OptionsClass - A collection of options
- 
  Abstract:
  
--- 1,4 ----
***************
*** 14,26 ****
  the initial values are loaded from this.
  
! The Option class holds information about an option - the name of the
! option, a nice name (to display), documentation, default value,
! possible values (a tuple or a regex pattern), whether multiple values
! are allowed, and whether the option should be reset when restoring to
! defaults (options like server names should *not* be).
! 
! The OptionsClass class provides facility for a collection of Options.
! It is expected that manipulation of the options will be carried out
! via an instance of this class.
  
  To Do:
--- 10,16 ----
  the initial values are loaded from this.
  
! The Option class is defined in OptionsClass.py - this module
! is responsible only for instantiating and loading the globally
! shared instance.
  
  To Do:
***************
*** 28,90 ****
     many, many attributes to the options object) as soon as all the
     modules are changed over.
-  o Once the above is done, and we have waited a suitable time, stop
-    allowing invalid options in configuration files
-  o Find a regex expert to come up with *good* patterns for domains,
-    email addresses, and so forth.
-  o str(Option) should really call Option.unconvert since this is what
-    it does.  Try putting that in and running all the tests.
-  o [See also the __issues__ string.]
-  o Suggestions?
- 
- """
- 
- # This module is part of the spambayes project, which is Copyright 2002-3
- # The Python Software Foundation and is covered by the Python Software
- # Foundation license.
- 
- __credits__ = "All the Spambayes folk."
- # blame for the new format: Tony Meyer <ta-meyer at ihug.co.nz>
- 
- __issues__ = """Things that should be considered further and by
- other people:
- 
- We are very generous in checking validity when multiple values are
- allowed and the check is a regex (rather than a tuple).  Any sequence
- that does not match the regex may be used to delimit the values.
- For example, if the regex was simply r"[\d]*" then these would all
- be considered valid:
- "123a234" -> 123, 234
- "123abced234" -> 123, 234
- "123XST234xas" -> 123, 234
- "123 234" -> 123, 234
- "123~!@$%^&@234!" -> 123, 234
- 
- If this is a problem, my recommendation would be to change the
- multiple_values_allowed attribute from a boolean to a regex/None
- i.e. if multiple is None, then only one value is allowed.  Otherwise
- multiple is used in a re.split() to separate the input.
  """
- 
- import sys
  import os
- import shutil
- from tempfile import TemporaryFile
  
! try:
!     import cStringIO as StringIO
! except ImportError:
!     import StringIO
! 
! import re
! import types
! import locale
! 
! try:
!     True, False, bool
! except NameError:
!     # Maintain compatibility with Python 2.2
!     True, False = 1, 0
!     def bool(val):
!         return not not val
  
  # Backwards compatibility stuff - this will be removed at some point
--- 18,25 ----
     many, many attributes to the options object) as soon as all the
     modules are changed over.
  """
  import os
  
! __all__ = ['options']
  
  # Backwards compatibility stuff - this will be removed at some point
***************
*** 139,182 ****
  }
  
! # These are handy references to commonly used regex/tuples defining
! # permitted values. Although the majority of options use one of these,
! # you may use any regex or tuple you wish.
! HEADER_NAME = r"[\w\.\-\*]+"
! HEADER_VALUE = r"[\w\.\-\*]+"
! INTEGER = r"[\d]+"              # actually, a *positive* integer
! REAL = r"[\d]+[\.]?[\d]*"       # likewise, a *positive* real
! BOOLEAN = (False, True)
! SERVER = r"([\w\.\-]+(:[\d]+)?)"  # in the form server:port
! PORT = r"[\d]+"
! EMAIL_ADDRESS = r"[\w\-\.]+@[\w\-\.]+"
! PATH = r"[\w\.\-~:\\/\*]+"
! VARIABLE_PATH = PATH + r"%"
! FILE = r"[\S]+"
! FILE_WITH_PATH = PATH
! # IMAP seems to allow any character at all in a folder name,
! # but we want to use the comma as a delimiter for lists, so
! # we don't allow this.  If anyone has folders with commas in the
! # names, please let us know and we'll figure out something else.
! # ImapUI.py prints out a warning if this is the case.
! IMAP_FOLDER = r"[^,]+"
! 
! # IMAP's astring should also be valid in the form:
! #   "{" number "}" CRLF *CHAR8
! #   where number represents the number of CHAR8 octets
! # but this is too complex for us at the moment.
! IMAP_ASTRING = ""
! for i in range(1, 128):
!     if not chr(i) in ['"', '\\', '\n', '\r']:
!         IMAP_ASTRING += chr(i)
! IMAP_ASTRING = r"\"?\\?[" + re.escape(IMAP_ASTRING) + r"]+\"?"
! 
! # Similarly, each option must specify whether it should be reset to
! # this value on a "reset to defaults" command.  Most should, but with some
! # like a server name that defaults to "", this would be pointless.
! # Again, for ease of reading, we define these here:
! RESTORE = True
! DO_NOT_RESTORE = False
! 
! __all__ = ['options']
  
  # Format:
--- 74,79 ----
  }
  
! # Grab the stuff from the core options class.
! from OptionsClass import *
  
  # Format:
***************
*** 910,947 ****
    ),
  
-   "Outlook" : (
-     ("train_recovered_spam", "", True,
-      """""",
-      BOOLEAN, RESTORE),
- 
-     ("train_manual_spam", "", True,
-      """""",
-      BOOLEAN, RESTORE),
- 
-     ("spam_action", "", "Untouched",
-      """""",
-      ("Untouched", "Moved", "Copied"), RESTORE),
- 
-     ("unsure_action", "", "Untouched",
-      """""",
-      ("Untouched", "Moved", "Copied"), RESTORE),
- 
-     ("filter_enabled", "", False,
-      """""",
-      BOOLEAN, RESTORE),
- 
-     ("field_score_name", "", "Spam",
-      """""",
-      r"[\w]+", RESTORE),
- 
-     ("delete_as_spam_marks_as_read", "", False,
-      """""",
-      BOOLEAN, RESTORE),
- 
-     ("rescore", "", True,
-      """""",
-      BOOLEAN, RESTORE),
-   ),
-  
    "imap" : (
      ("server", "Server", (),
--- 807,810 ----
***************
*** 1025,1592 ****
  }
  
- class Option(object):
-     def __init__(self, name, nice_name="", default=None,
-                  help_text="", allowed=None, restore=True):
-         self.name = name
-         self.nice_name = nice_name
-         self.default_value = default
-         self.explanation_text = help_text
-         self.allowed_values = allowed
-         self.restore = restore
-         self.value = default
-         self.delimiter = None
- 
-     def display_name(self):
-         '''A name for the option suitable for display to a user.'''
-         return self.nice_name
-     def default(self):
-         '''The default value for the option.'''
-         return self.default_value
-     def doc(self):
-         '''Documentation for the option.'''
-         return self.explanation_text
-     def valid_input(self):
-         '''Valid values for the option.'''
-         return self.allowed_values
-     def no_restore(self):
-         '''Do not restore this option when restoring to defaults.'''
-         return not self.restore
-     def set(self, val):
-         '''Set option to value.'''
-         self.value = val
-     def get(self):
-         '''Get option value.'''
-         return self.value
-     def multiple_values_allowed(self):
-         '''Multiple values are allowed for this option.'''
-         return type(self.value) == types.TupleType
-     
-     def is_valid(self, value):
-         '''Check if this is a valid value for this option.'''
- # XXX This test is in the original code, but makes no sense....
- # XXX self.allowed_values defaults to None, and if that is the
- # XXX current value, then whatever is passed would be invalid
- # XXX I agree this is a silly state to be in, but it is possible
- # XXX I suppose that self.allowed_values should default to *any*
- # XXX rather than None, but I'm not sure how to express that,
- # XXX unless the regex r"." is correct.
-         if self.allowed_values is None:
-             return False
- 
-         if type(self.value) == types.TupleType:
-             return self.is_valid_multiple(value)
-         else:
-             return self.is_valid_single(value)
- 
-     def is_valid_multiple(self, value):
-         '''Return True iff value is a valid value for this option.
-         Use if multiple values are allowed.'''
-         if type(value) == types.TupleType:
-             for val in value:
-                 if not self.is_valid_single(val):
-                     return False
-             return True
-         return self.is_valid_single(value)
- 
-     def is_valid_single(self, value):
-         '''Return True iff value is a valid value for this option.
-         Use when multiple values are not allowed.'''
-         if type(self.allowed_values) == types.TupleType:
-             if value in self.allowed_values:
-                 return True
-             else:
-                 return False
-         else:
-             # special handling for booleans, thanks to Python 2.2
-             if self.is_boolean and (value == True or value == False):
-                 return True
-             if type(value) != type(self.value) and \
-                type(self.value) != types.TupleType:
-                 # This is very strict!  If the value is meant to be
-                 # a real number and an integer is passed in, it will fail.
-                 # (So pass 1. instead of 1, for example)
-                 return False
-             avals = self._split_values(value)
-             # in this case, allowed_values must be a regex, and
-             # _split_values must match once and only once
-             if len(avals) == 1:
-                 return True
-             else:
-                 # either no match or too many matches
-                 return False
-             
-     def _split_values(self, value):
-         # do the regex mojo here
-         try:
-             r = re.compile(self.allowed_values)
-         except:
-             print self.allowed_values
-             raise
-         s = str(value)
-         i = 0
-         vals = ()
-         while True:
-             m = r.search(s[i:])
-             if m is None:
-                 break
-             vals += (m.group(),)
-             delimiter = s[i:i + m.start()]
-             if self.delimiter is None and delimiter != "":
-                 self.delimiter = delimiter
-             i += m.end()
-         return vals
- 
-     def as_nice_string(self, section=None):
-         '''Summarise the option in a user-readable format.'''
-         if section is None:
-             strval = ""
-         else:
-             strval = "[%s] " % (section)
-         strval += "%s - \"%s\"\nDefault: %s\nDo not restore: %s\n" \
-                  % (self.name, self.display_name(),
-                     str(self.default()), str(self.no_restore()))
-         strval += "Valid values: %s\nMultiple values allowed: %s\n" \
-                   % (str(self.valid_input()),
-                      str(self.multiple_values_allowed()))
-         strval += "\"%s\"\n\n" % (str(self.doc()))
-         return strval
- 
-     def write_config(self, file):
-         '''Output value in configuration file format.'''
-         file.write(self.name)
-         file.write(': ')
-         file.write(self.unconvert())
-         file.write('\n')
- 
-     def convert(self, value):
-         '''Convert value from a string to the appropriate type.'''
-         svt = type(self.value)
-         if svt == type(value):
-             # already the correct type
-             return value
-         if self.is_boolean():
-             if str(value) == "True" or value == 1:
-                 return True
-             elif str(value) == "False" or value == 0:
-                 return False
-             raise TypeError, self.name + " must be True or False"
-         if self.multiple_values_allowed():
-             # This will fall apart if the allowed_value is a tuple,
-             # but not a homogenous one...
-             if type(self.allowed_values) in types.StringTypes:
-                 vals = list(self._split_values(value))
-             else:
-                 vals = value.split()
-             if len(self.default_value) > 0:
-                 to_type = type(self.default_value[0])
-             else:
-                 to_type = types.StringType
-             for i in range(0, len(vals)):
-                 vals[i] = self._convert(vals[i], to_type)
-             return tuple(vals)
-         else:
-             return self._convert(value, svt)
-         raise TypeError, self.name + " has an invalid type."
- 
-     def _convert(self, value, to_type):
-         '''Convert an int, float or string to the specified type.'''
-         if to_type == type(value):
-             # already the correct type
-             return value
-         if to_type == types.IntType:
-             return locale.atoi(value)
-         if to_type == types.FloatType:
-             return locale.atof(value)
-         if to_type in types.StringTypes:
-             return str(value)
-         raise TypeError, "Invalid type."
- 
-     def unconvert(self):
-         '''Convert value from the appropriate type to a string.'''
-         if type(self.value) in types.StringTypes:
-             # nothing to do
-             return self.value
-         if self.is_boolean():
-             # A wee bit extra for Python 2.2
-             if self.value == True:
-                 return "True"
-             else:
-                 return "False"
-         if type(self.value) == types.TupleType:
-             if len(self.value) == 0:
-                 return ""
-             if len(self.value) == 1:
-                 v = self.value[0]
-                 if type(v) == types.FloatType:
-                     return locale.str(self.value[0])
-                 return str(v)
-             # We need to separate out the items
-             strval = ""
-             # We use a character that is invalid as the separator
-             # so that it will reparse correctly.  We could try all
-             # characters, but we make do with this set of commonly
-             # used ones - note that the first one that works will
-             # be used.  Perhaps a nicer solution than this would be
-             # to specifiy a valid delimiter for all options that
-             # can have multiple values.  Note that we have None at
-             # the end so that this will crash and die if none of
-             # the separators works <wink>.
-             if self.delimiter is None:
-                 if type(self.allowed_values) == types.TupleType:
-                     self.delimiter = ' '
-                 else:
-                     v0 = self.value[0]
-                     v1 = self.value[1]
-                     for sep in [' ', ',', ':', ';', '/', '\\', None]:
-                         # we know at this point that len(self.value) is at
-                         # least two, because len==0 and len==1 were dealt
-                         # with as special cases
-                         test_str = str(v0) + sep + str(v1)
-                         test_tuple = self._split_values(test_str)
-                         if test_tuple[0] == str(v0) and \
-                            test_tuple[1] == str(v1) and \
-                            len(test_tuple) == 2:
-                             break
-                     # cache this so we don't always need to do the above
-                     self.delimiter = sep
-             for v in self.value:
-                 if type(v) == types.FloatType:
-                     v = locale.str(v)
-                 else:
-                     v = str(v)
-                 strval += v + self.delimiter
-             strval = strval[:-len(self.delimiter)] # trailing seperator
-         else:
-             # Otherwise, we just hope str() will do the job
-             strval = str(self.value)
-         return strval
- 
-     def is_boolean(self):
-         '''Return True iff the option is a boolean value.'''
-         # This is necessary because of the Python 2.2 True=1, False=0
-         # cheat.  The valid values are returned as 0 and 1, even if
-         # they are actually False and True - but 0 and 1 are not
-         # considered valid input (and 0 and 1 don't look as nice)
-         # So, just for the 2.2 people, we have this helper function
-         try:
-             if type(self.allowed_values) == types.TupleType and \
-                len(self.allowed_values) > 0 and \
-                type(self.allowed_values[0]) == types.BooleanType:
-                 return True
-             return False
-         except AttributeError:
-             # If the user has Python 2.2 and an option has valid values
-             # of (0, 1) - i.e. integers, then this function will return
-             # the wrong value.  I don't know what to do about that without
-             # explicitly stating which options are boolean
-             if self.allowed_values == (False, True):
-                 return True
-             return False
- 
- 
- class OptionsClass(object):
-     def __init__(self):
-         self._options = {}
- 
-     #
-     # Regular expressions for parsing section headers and options.
-     # Lifted straight from ConfigParser
-     #
-     SECTCRE = re.compile(
-         r'\['                                 # [
-         r'(?P<header>[^]]+)'                  # very permissive!
-         r'\]'                                 # ]
-         )
-     OPTCRE = re.compile(
-         r'(?P<option>[^:=\s][^:=]*)'          # very permissive!
-         r'\s*(?P<vi>[:=])\s*'                 # any number of space/tab,
-                                               # followed by separator
-                                               # (either : or =), followed
-                                               # by any # space/tab
-         r'(?P<value>.*)$'                     # everything up to EOL
-         )
- 
-     def update_file(self, filename):
-         '''Update the specified configuration file.'''
-         sectname = None
-         optname = None
-         out = TemporaryFile()
-         if os.path.exists(filename):
-             f = file(filename, "r")
-         else:
-             # doesn't exist, so create it - all the changed options will
-             # be added to it
-             if options["globals", "verbose"]:
-                 print "Creating new configuration file", filename
-             f = file(filename, "w")
-             f.close()
-             f = file(filename, "r")
-         written = []
-         vi = ": " # default; uses the one from the file where possible
-         while True:
-             line = f.readline()
-             if not line:
-                 break
-             # comment or blank line?
-             if line.strip() == '' or line[0] in '#;':
-                 out.write(line)
-                 continue
-             if line.split(None, 1)[0].lower() == 'rem' and line[0] in "rR":
-                 # no leading whitespace
-                 out.write(line)
-                 continue
-             # continuation line?
-             if line[0].isspace() and sectname is not None and optname:
-                 continue
-             # a section header or option header?
-             else:
-                 # is it a section header?
-                 mo = self.SECTCRE.match(line)
-                 if mo:
-                     # Add any missing from the previous section
-                     if sectname is not None:
-                         self._add_missing(out, written, sectname, vi, False)
-                     sectname = mo.group('header')
-                     # So sections can't start with a continuation line
-                     optname = None
-                     if sectname in self.sections():
-                         out.write(line)
-                 # an option line?
-                 else:
-                     mo = self.OPTCRE.match(line)
-                     if mo:
-                         optname, vi, optval = mo.group('option', 'vi', 'value')
-                         if vi in ('=', ':') and ';' in optval:
-                             # ';' is a comment delimiter only if it follows
-                             # a spacing character
-                             pos = optval.find(';')
-                             if pos != -1 and optval[pos-1].isspace():
-                                 optval = optval[:pos]
-                         optval = optval.strip()
-                         # allow empty values
-                         if optval == '""':
-                             optval = ''
-                         optname = optname.rstrip().lower()
-                         if self._options.has_key((sectname, optname)):
-                             out.write(optname)
-                             out.write(vi)
-                             out.write(self.unconvert(sectname, optname))
-                             out.write('\n')
-                             written.append((sectname, optname))
-         for sect in self.sections():
-             self._add_missing(out, written, sect, vi)
-         f.close()
-         out.flush()
-         if options["globals", "verbose"]:
-             # save a backup of the old file
-             shutil.copyfile(filename, filename + ".bak")
-         # copy the new file across
-         f = file(filename, "w")
-         out.seek(0)
-         shutil.copyfileobj(out, f)
-         out.close()
-         f.close()
-     
-     def _add_missing(self, out, written, sect, vi, label=True):
-         # add any missing ones, where the value does not equal the default
-         for opt in self.options_in_section(sect):
-             if not (sect, opt) in written and \
-                self.get(sect, opt) != self.default(sect, opt):
-                 if label:
-                     out.write('[')
-                     out.write(sect)
-                     out.write("]\n")
-                     label = False
-                 out.write(opt)
-                 out.write(vi)
-                 out.write(self.unconvert(sect, opt))
-                 out.write('\n')
-                 written.append((sect, opt))
- 
-     def load_defaults(self):
-         '''Load default values (stored in this module).'''
-         for section, opts in defaults.items():
-             for opt in opts:
-                 o = Option(opt[0], opt[1], opt[2], opt[3], opt[4], opt[5])
-                 self._options[section, opt[0]] = o
-                 # start with default value
-                 o.set(opt[2])
-                 # A (really ugly) bit of backwards compatability
-                 # *** This will vanish soon, so do not make use of it in
-                 #     new code ***
-                 self._oldset(section, opt[0], opt[2])
- 
-     def _oldset(self, section, option, value):
-         # A (really ugly) bit of backwards compatability
-         # *** This will vanish soon, so do not make use of it in
-         #     new code ***
-         for (oldsect, oldopt), (newsect, newopt) in conversion_table.items():
-             if (newsect, newopt) == (section, option):
-                 section = oldsect
-                 option = oldopt
-         setattr(options, option, value)
-         old_name = section[0:1].lower() + section[1:] + "_" + option
-         setattr(options, old_name, value)
-                 
-     def merge_files(self, file_list):
-         for file in file_list:
-             self.merge_file(file)
- 
-     def merge_file(self, filename):
-         import ConfigParser
-         c = ConfigParser.ConfigParser()
-         c.read(filename)
-         for sect in c.sections():
-             for opt in c.options(sect):
-                 value = c.get(sect, opt)
-                 # backward compatibility guff
-                 if opt[:len(sect) + 1].lower() == sect.lower() + '_':
-                     opt = opt[len(sect)+1:]
-                 if conversion_table.has_key((sect, opt)):
-                     section, option = conversion_table[sect, opt]
-                 else:
-                     section = sect
-                     option = opt
-                 # end of backward compatibility guff
-                 if not self._options.has_key((section, option)):
-                     print "Invalid option %s in section %s in file %s" % \
-                           (opt, sect, filename)
-                 else:
-                     if self.multiple_values_allowed(section, option):
-                         value = self.convert(section, option, value)
-                     value = self.convert(section, option, value)
-                     self.set(section, option, value)
-                     # backward compatibility guff
-                     self._oldset(sect, opt, value)
-                     # end of backward compatibility guff
- 
-     # not strictly necessary, but convenient shortcuts to self._options
-     def display_name(self, sect, opt):
-         '''A name for the option suitable for display to a user.'''
-         return self._options[sect, opt].display_name()
-     def default(self, sect, opt):
-         '''The default value for the option.'''
-         return self._options[sect, opt].default()
-     def doc(self, sect, opt):
-         '''Documentation for the option.'''
-         return self._options[sect, opt].doc()
-     def valid_input(self, sect, opt):
-         '''Valid values for the option.'''
-         return self._options[sect, opt].valid_input()
-     def no_restore(self, sect, opt):
-         '''Do not restore this option when restoring to defaults.'''
-         return self._options[sect, opt].no_restore()
-     def is_valid(self, sect, opt, value):
-         '''Check if this is a valid value for this option.'''
-         return self._options[sect, opt].is_valid(value)
-     def multiple_values_allowed(self, sect, opt):
-         '''Multiple values are allowed for this option.'''
-         return self._options[sect, opt].multiple_values_allowed()
- 
-     def is_boolean(self, sect, opt):
-         '''The option is a boolean value. (Support for Python 2.2).'''
-         return self._options[sect, opt].is_boolean()
- 
-     def convert(self, sect, opt, value):
-         '''Convert value from a string to the appropriate type.'''
-         return self._options[sect, opt].convert(value)
- 
-     def unconvert(self, sect, opt):
-         '''Convert value from the appropriate type to a string.'''
-         return self._options[sect, opt].unconvert()
- 
-     def get(self, sect, opt):
-         '''Get an option.'''
-         if conversion_table.has_key((sect, opt)):
-             sect, opt = conversion_table[sect, opt]
-         return self._options[sect, opt].get()
- 
-     def __getitem__(self, key):
-         return self.get(key[0], key[1])
- 
-     def set(self, sect, opt, val=None):
-         '''Set an option.'''
-         if conversion_table.has_key((sect, opt)):
-             sect, opt = conversion_table[sect, opt]
-         if self.is_valid(sect, opt, val):
-             self._options[sect, opt].set(val)
-             # backwards compatibility stuff
-             self._oldset(sect, opt, val)
-         else:
-             print "Attempted to set [%s] %s with invalid value %s (%s)" % \
-                   (sect, opt, val, type(val))
-         
-     def __setitem__(self, key, value):
-         self.set(key[0], key[1], value)
- 
-     def sections(self):
-         '''Return an alphabetical list of all the sections.'''
-         all = []
-         for sect, opt in self._options.keys():
-             if sect not in all:
-                 all.append(sect)
-         all.sort()
-         return all
-     
-     def options_in_section(self, section):
-         '''Return an alphabetical list of all the options in this section.'''
-         all = []
-         for sect, opt in self._options.keys():
-             if sect == section:
-                 all.append(opt)
-         all.sort()
-         return all
- 
-     def options(self, prepend_section_name=False):
-         '''Return a alphabetical list of all the options, optionally
-         prefixed with [section_name]'''
-         all = []
-         for sect, opt in self._options.keys():
-             if prepend_section_name:
-                 all.append('[' + sect + ']' + opt)
-             else:
-                 all.append(opt)
-         all.sort()
-         return all
- 
-     def display(self):
-         '''Display options in a config file form.'''
-         output = StringIO.StringIO()
-         keys = self._options.keys()
-         keys.sort()
-         currentSection = None
-         for sect, opt in keys:
-             if sect != currentSection:
-                 if currentSection is not None:
-                     output.write('\n')
-                 output.write('[')
-                 output.write(sect)
-                 output.write("]\n")
-                 currentSection = sect
-             self._options[sect, opt].write_config(output)
-         return output.getvalue()
- 
-     def display_full(self, section=None, option=None):
-        '''Display options including all information.'''
-        # Given that the Options class is no longer as nice looking
-        # as it once was, this returns all the information, i.e.
-        # the doc, default values, and so on
-        output = StringIO.StringIO()
- 
-        # when section and option are both specified, this
-        # is nothing more than a call to as_nice_string
-        if section is not None and option is not None:
-            output.write(self._options[section,
-                                       option].as_nice_string(section))
-            return output.getvalue()
-        
-        all = self._options.keys()
-        all.sort()
-        for sect, opt in all:
-            if section is not None and sect != section:
-                continue
-            output.write(self._options[sect, opt].as_nice_string(sect))
-        return output.getvalue()
- 
  
  # `optionsPathname` is the pathname of the last ini file in the list.
--- 888,891 ----
***************
*** 1597,1601 ****
  
  options = OptionsClass()
! options.load_defaults()
  
  alternate = None
--- 896,901 ----
  
  options = OptionsClass()
! options.conversion_table = conversion_table # Our b/w compat hacks
! options.load_defaults(defaults)
  
  alternate = None
***************
*** 1618,1619 ****
--- 918,921 ----
  if not optionsPathname:
      optionsPathname = os.path.abspath('bayescustomize.ini')
+ # Set verbosity of this options instance to an option value!
+ options.verbose = options["globals", "verbose"]





More information about the Spambayes-checkins mailing list