[Distutils] Behaviour of "install_data" changed

Rene Liebscher R.Liebscher@gmx.de
Mon, 26 Jun 2000 13:38:49 +0200


This is a multi-part message in MIME format.
--------------1B9072FF001699F74DA18394
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Greg Ward wrote:
> 
> I've just fixed the behaviour of the "install_data" command: the default
> directory for installating data files is now just the installation base,
> which is usually sys.prefix.  Thus, if you put
> 
>    data_files = ["my_data"]
> 
> in your setup script, you will wind up installing (eg.)
> /usr/local/my_data -- definitely the wrong thing on Unix, and probably
> wrong on Windows too.  (Except for applications that have their own
> prefix, but that's not really dealt with yet.)
> 
> This is the Right Thing now because of the change to "install_data" that
> lets you specify where to put data files; the above should be spelled
> 
>    data_files = [("share", ["my_data"])]
> 
> which will install /usr/local/share/my_data.
> 
> Of course, this still doesn't solve the problem of, "How does my
> application/module know where Distutils installed this data file if the
> user did some wild funky custom installation?".  Oh well, better than
> nothing.

Would you like to have a complete new concept, which could solve all
your
problems? Here it is.

I would like to introduce a class 'Data_Files' similar to the
'Extension'
class.
It should accept following parameters:
* A basis directory to which all other filenames will become relative.
  base_dir : 'install_data','install_lib','install_headers' or any other
	    path which is defined in 'install'
* A directory under this base dir like the 'share' dir in Greg's example
above.
  copy_to : a path name
* a files list as usual
  files : ['foo','bar', ...]
* a list of templates as in MANIFEST.in, which is used to manipulate the
files list
  template : a list or a ';' separeted string
* a flag which switches the preserving of paths on or off
  ( file 'foo/bar' would copied to 'base_dir/foo/bar', instead of
'base_dir/bar' )
  preserve_path : 0 or 1

How to use this?
Consider you had a distutils.cfg template which you want to install.
There are now different ways to so:

data_files = [
# The old way (the patch doesn't break old code.):
 ("lib/python1.5/site-packages/distutils",["distutils/distutils.cfg"]),
# this is not portable with win32

# the same using a different base dir
          Data_Files(
              base_dir='install_lib',
              files=["distutils/distutils.cfg"],
              copy_to='distutils'
          ),
# much better, but you still have to specify the target directory

# without target directory
           Data_Files(
               base_dir='install_lib',
               files=["distutils/distutils.cfg"],
               preserve_path=1,
           ),
# the file has already the rght path so we can use this

# and now the most sophisticated solution:
          Data_Files(
              base_dir='install_lib',
              template="recursive-include distutils *.cfg",
              preserve_path=1,
          ),

I think this can solve all our problems.

> Of course, this still doesn't solve the problem of, "How does my
> application/module know where Distutils installed this data file if the
> user did some wild funky custom installation?". 
You only have to create a config file with the path of the other files
and put this config file in your packages directory. (see the
example above)

Maybe you are not convinced, so here a more complex example. It is taken
from a distutils setup.py file for PyOpenGL. (One of the packages I use
to test distutils.) It installs some examples in its package directory,
these contain data files and some images.
Using the old way looks like this:
      # non python files of examples
       data_files = [
           ('lib/python1.5/site-packages/OpenGL/Demo/dek',[
               "OpenGL/Demo/dek/README",
               "OpenGL/Demo/dek/image.ppm"
               ]),
           ('lib/python1.5/site-packages/OpenGL/Demo/dek/OglSurface',[
               "OpenGL/Demo/dek/OglSurface/1crn.face",
               "OpenGL/Demo/dek/OglSurface/1crn.pdb",
               "OpenGL/Demo/dek/OglSurface/1crn.vert",
               "OpenGL/Demo/dek/OglSurface/1crn.xyzr",
               "OpenGL/Demo/dek/OglSurface/README",
               "OpenGL/Demo/dek/OglSurface/test.ppm",
               ]),
           ('lib/python1.5/site-packages/OpenGL/Demo/srenner',[
               "OpenGL/Demo/srenner/README",
               ]),
           ('lib/python1.5/site-packages/OpenGL/Demo/srenner/Images',
               glob (os.path.join
("OpenGL/Demo/srenner/Images","*.ppm"))
               ),
       ],
And the new way is much simpler (and more portable):
       # non python files of examples
       data_files = [
            Data_Files(
               base_dir='install_lib',
               # py-files shouldn't be installed with install_data
               template=["graft OpenGL","exclude *.py*"] ,
               preserve_path=1,
            )
       ],


What does the patch change in distutils?

First I created the class 'Data_Files', which is currently located
in install_data, maybe it should get its own file as Extension did.
There were also some changes in install_data to work with this new 
class.

Then I had to take the template proccesing out of the sdist command.
It is now a seperate class in sdist.py. This was necessary because 
I want to use it in install_data.

And finally the patch creates a distutils.cfg template and changes
distutils setup.py, so it would install this template.

Do you like it so, or is there anything to change? (Maybe some 
names could get better names, but this not a big problem.)


kind regards

Rene Liebscher
--------------1B9072FF001699F74DA18394
Content-Type: text/plain; charset=us-ascii; name="install_data.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline; filename="install_data.patch"

diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/install_data.py distutils/distutils/command/install_data.py
--- distutils.orig/distutils/command/install_data.py	Mon Jun 26 09:34:37 2000
+++ distutils/distutils/command/install_data.py	Mon Jun 26 12:51:52 2000
@@ -7,10 +7,59 @@
 
 __revision__ = "$Id: install_data.py,v 1.10 2000/06/24 17:36:24 gward Exp $"
 
-import os
-from types import StringType
+import os,sys,string
+from types import StringType,TupleType,ListType
 from distutils.core import Command
 from distutils.util import change_root
+from distutils.command.sdist import Template_Processor
+
+class Data_Files:
+    """ container for list of data files.
+        supports alternate base_dirs e.g. 'install_lib','install_header',...
+        supports a directory where to copy files
+        supports templates as in MANIFEST.in
+        supports preserving of paths in filenames eg. foo/xyz is copied to base_dir/foo/xyz 
+    """
+
+    def __init__(self,base_dir=None,files=None,copy_to=None,template=None,preserve_path=0):
+        self.base_dir = base_dir
+        self.files = files
+        self.copy_to = copy_to
+        self.template = template
+        self.preserve_path = preserve_path
+        self.finalized = 0
+
+    def warn (self, msg):
+        sys.stderr.write ("warning: %s: %s\n" %
+                          ("install_data", msg))
+        
+    def debug_print (self, msg):
+        """Print 'msg' to stdout if the global DEBUG (taken from the
+        DISTUTILS_DEBUG environment variable) flag is true.
+        """
+        from distutils.core import DEBUG
+        if DEBUG:
+            print msg
+
+        
+    def finalize(self):
+        """ complete the files list by processing the given template """
+        if self.finalized: 
+            return
+        if self.files == None:
+            self.files = []
+        if self.template != None:
+            if type(self.template) == StringType:
+                self.template = string.split(self.template,";")
+            template_processor = Template_Processor(self.files)
+            template_processor.set_warnings_function(self.warn)
+            template_processor.set_debug_print_function(self.debug_print)
+            for line in self.template:
+                template_processor.process_line(string.strip(line))
+        self.finalized = 1
+
+# end class Data_Files
+    
 
 class install_data (Command):
 
@@ -32,34 +81,86 @@
 
     def finalize_options (self):
         self.set_undefined_options('install',
-	                           ('install_data', 'install_dir'),
-				   ('root', 'root'),
-				  )
-
+                                  ('install_data', 'install_dir'),
+                                  ('root', 'root'),
+                                  )
+ 
+    def check_data(self,d):
+        """ check if data are in new format, if not create a suitable object.
+            returns finalized data object
+        """
+        if not isinstance(d, Data_Files):
+            self.warn(("old-style data files list found "
+                        "-- please convert to Data_Files instance"))
+            if type(d) is TupleType:
+                if len(d) != 2 or  not (type(d[1]) is ListType):
+                        raise DistutilsSetupError, \
+                          ("each element of 'data_files' option must be an "
+                            "Data File instance, a string or 2-tuple (string,[strings])")
+                d = Data_Files(copy_to=d[0],files=d[1])
+            else:
+                if not (type(d) is StringType):
+                        raise DistutilsSetupError, \
+                          ("each element of 'data_files' option must be an "
+                           "Data File instance, a string or 2-tuple (string,[strings])")
+                d = Data_Files(files=[d])
+        d.finalize()
+        return d
+ 
+ 
     def run (self):
-        self.mkpath(self.install_dir)
-        for f in self.data_files:
-            if type(f) == StringType:
-                # it's a simple file, so copy it
-                self.warn("setup script did not provide a directory for "
-                          "'%s' -- installing right in '%s'" %
-                          (f, self.install_dir))
-                out = self.copy_file(f, self.install_dir)
-                self.outfiles.append(out)
+        self.outfiles = []
+        install_cmd = self.get_finalized_command('install')
+ 
+        for d in self.data_files:
+            d = self.check_data(d)
+ 
+            install_dir = self.install_dir
+            # alternative base dir given => overwrite install_dir
+            if d.base_dir != None:
+                install_dir = getattr(install_cmd,d.base_dir)
+     
+            # copy to an other directory 
+            if d.copy_to != None:
+                if not os.path.isabs(d.copy_to):
+                    # relatiev path to install_dir
+                    dir = os.path.join(install_dir, d.copy_to)
+                elif install_cmd.root:
+                    # absolute path and alternative root set 
+                    dir = change_root(self.root,d.copy_to)
+                else:
+                    # absolute path
+                    dir = d.copy_to                  
             else:
-                # it's a tuple with path to install to and a list of files
-                dir = f[0]
-                if not os.path.isabs(dir):
-                    dir = os.path.join(self.install_dir, dir)
-                elif self.root:
-                    dir = change_root(self.root, dir)
-                self.mkpath(dir)
-                for data in f[1]:
-                    out = self.copy_file(data, dir)
-                    self.outfiles.append(out)
+                # simply copy to install_dir
+                dir = install_dir
+ 
+            dir=os.path.normpath(dir)
+            # create path
+            self.mkpath(dir)
+ 
+            # copy all files    
+            for f in d.files:
+                if d.copy_to == None:
+                    self.warn("setup script did not provide a directory for "
+                              "'%s' -- installing right in '%s'" %
+                              (f, install_dir))
+                if d.preserve_path:
+                    # preserve path in filename
+                    self.mkpath(os.path.dirname(os.path.join(dir,f)))
+                    out = self.copy_file(f, os.path.join(dir,f))
+                else:
+                    out = self.copy_file(f, dir)
+            self.outfiles.append(out)
+ 
+        return self.outfiles
 
     def get_inputs (self):
-        return self.data_files or []
-
+        inputs = []
+        for d in self.data_files:
+            d = self.check_data(d)
+            inputs.append(d.files)
+        return inputs
+  
     def get_outputs (self):
         return self.outfiles
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/sdist.py distutils/distutils/command/sdist.py
--- distutils.orig/distutils/command/sdist.py	Mon Jun 26 09:34:37 2000
+++ distutils/distutils/command/sdist.py	Mon Jun 26 12:39:08 2000
@@ -34,6 +34,260 @@
         "List of available source distribution formats:")
 
 
+class Template_Processor:
+
+    def __init__(self, files_list=[]):
+	self.files=files_list
+        self.all_files = findall ()
+	self.template_warn=self.dummy_function
+	self.debug_print=self.dummy_function
+
+    def set_warnings_function(self, warnings_function):
+	self.template_warn=warnings_function
+
+    def set_debug_print_function(self, debug_print_function):
+	self.debug_print=debug_print_function
+
+    def dummy_function(self,warning):
+	pass
+
+    def search_dir (self, dir, pattern=None):
+        """Recursively find files under 'dir' matching 'pattern' (a string
+        containing a Unix-style glob pattern).  If 'pattern' is None, find
+        all files under 'dir'.  Return the list of found filenames.
+        """
+        allfiles = findall (dir)
+        if pattern is None:
+            return allfiles
+
+        pattern_re = translate_pattern (pattern)
+        files = []
+        for file in allfiles:
+            if pattern_re.match (os.path.basename (file)):
+                files.append (file)
+
+        return files
+
+    # search_dir ()
+
+    def recursive_exclude_pattern (self, dir, pattern=None):
+        """Remove filenames from 'self.files' that are under 'dir' and
+        whose basenames match 'pattern'.
+        """
+        self.debug_print("recursive_exclude_pattern: dir=%s, pattern=%s" %
+                         (dir, pattern))
+        if pattern is None:
+            pattern_re = None
+        else:
+            pattern_re = translate_pattern (pattern)
+
+        for i in range (len (self.files)-1, -1, -1):
+            (cur_dir, cur_base) = os.path.split (self.files[i])
+            if (cur_dir == dir and
+                (pattern_re is None or pattern_re.match (cur_base))):
+                self.debug_print("removing %s" % self.files[i])
+                del self.files[i]
+
+
+    def process_line (self, line):
+        """Read and parse a line of a template file. Process all file
+        specifications (include and exclude) in the template and
+        update 'self.files' accordingly (filenames may be added to
+        or removed from 'self.files' based on the template).
+        """
+        assert self.files is not None and type (self.files) is ListType
+
+
+        words = string.split (line)
+        action = words[0]
+
+        # First, check that the right number of words are present
+        # for the given action (which is the first word)
+        if action in ('include','exclude',
+                      'global-include','global-exclude'):
+            if len (words) < 2:
+                self.template_warn \
+                    ("invalid manifest template line: " +
+                     "'%s' expects <pattern1> <pattern2> ..." %
+                     action)
+                return
+
+            pattern_list = map(convert_path, words[1:])
+
+        elif action in ('recursive-include','recursive-exclude'):
+            if len (words) < 3:
+                self.template_warn \
+                    ("invalid manifest template line: " +
+                     "'%s' expects <dir> <pattern1> <pattern2> ..." %
+                     action)
+                return
+
+            dir = convert_path(words[1])
+            pattern_list = map (convert_path, words[2:])
+
+        elif action in ('graft','prune'):
+            if len (words) != 2:
+                self.template_warn \
+                    ("invalid manifest template line: " +
+                     "'%s' expects a single <dir_pattern>" %
+                     action)
+                return
+
+            dir_pattern = convert_path (words[1])
+
+        else:
+            self.template_warn ("invalid manifest template line: " +
+                           "unknown action '%s'" % action)
+            return
+
+        # OK, now we know that the action is valid and we have the
+        # right number of words on the line for that action -- so we
+        # can proceed with minimal error-checking.  Also, we have
+        # defined either (pattern), (dir and pattern), or
+        # (dir_pattern) -- so we don't have to spend any time
+        # digging stuff up out of 'words'.
+
+        if action == 'include':
+            self.debug_print("include " + string.join(pattern_list))
+            for pattern in pattern_list:
+                files = self.select_pattern (self.all_files, pattern, anchor=1)
+                if not files:
+                    self.template_warn ("no files found matching '%s'" %
+                                   pattern)
+                else:
+                    self.files.extend (files)
+
+        elif action == 'exclude':
+            self.debug_print("exclude " + string.join(pattern_list))
+            for pattern in pattern_list:
+                num = self.exclude_pattern (self.files, pattern, anchor=1)
+                if num == 0:
+                    self.template_warn (
+                        "no previously-included files found matching '%s'"%
+                        pattern)
+
+        elif action == 'global-include':
+            self.debug_print("global-include " + string.join(pattern_list))
+            for pattern in pattern_list:
+                files = self.select_pattern (self.all_files, pattern, anchor=0)
+                if not files:
+                    self.template_warn (("no files found matching '%s' " +
+                                    "anywhere in distribution") %
+                                   pattern)
+                else:
+                    self.files.extend (files)
+
+        elif action == 'global-exclude':
+            self.debug_print("global-exclude " + string.join(pattern_list))
+            for pattern in pattern_list:
+                num = self.exclude_pattern (self.files, pattern, anchor=0)
+                if num == 0:
+                    self.template_warn \
+                        (("no previously-included files matching '%s' " +
+                          "found anywhere in distribution") %
+                         pattern)
+
+        elif action == 'recursive-include':
+            self.debug_print("recursive-include %s %s" %
+                             (dir, string.join(pattern_list)))
+            for pattern in pattern_list:
+                files = self.select_pattern (
+                    self.all_files, pattern, prefix=dir)
+                if not files:
+                    self.template_warn (("no files found matching '%s' " +
+                                        "under directory '%s'") %
+                                       (pattern, dir))
+                else:
+                    self.files.extend (files)
+
+        elif action == 'recursive-exclude':
+            self.debug_print("recursive-exclude %s %s" %
+                             (dir, string.join(pattern_list)))
+            for pattern in pattern_list:
+                num = self.exclude_pattern(
+                    self.files, pattern, prefix=dir)
+                if num == 0:
+                    self.template_warn \
+                        (("no previously-included files matching '%s' " +
+                          "found under directory '%s'") %
+                         (pattern, dir))
+
+        elif action == 'graft':
+            self.debug_print("graft " + dir_pattern)
+            files = self.select_pattern(
+                self.all_files, None, prefix=dir_pattern)
+            if not files:
+                self.template_warn ("no directories found matching '%s'" %
+                               dir_pattern)
+            else:
+                self.files.extend (files)
+
+        elif action == 'prune':
+            self.debug_print("prune " + dir_pattern)
+            num = self.exclude_pattern(
+                self.files, None, prefix=dir_pattern)
+            if num == 0:
+                self.template_warn \
+                    (("no previously-included directories found " +
+                      "matching '%s'") %
+                     dir_pattern)
+        else:
+            raise RuntimeError, \
+                  "this cannot happen: invalid action '%s'" % action
+
+    # process_line ()
+
+
+
+    def select_pattern (self, files, pattern, anchor=1, prefix=None):
+        """Select strings (presumably filenames) from 'files' that match
+        'pattern', a Unix-style wildcard (glob) pattern.  Patterns are not
+        quite the same as implemented by the 'fnmatch' module: '*' and '?'
+        match non-special characters, where "special" is platform-dependent:
+        slash on Unix, colon, slash, and backslash on DOS/Windows, and colon on
+        Mac OS.
+
+        If 'anchor' is true (the default), then the pattern match is more
+        stringent: "*.py" will match "foo.py" but not "foo/bar.py".  If
+        'anchor' is false, both of these will match.
+
+        If 'prefix' is supplied, then only filenames starting with 'prefix'
+        (itself a pattern) and ending with 'pattern', with anything in between
+        them, will match.  'anchor' is ignored in this case.
+
+        Return the list of matching strings, possibly empty.
+        """
+        matches = []
+        pattern_re = translate_pattern (pattern, anchor, prefix)
+        self.debug_print("select_pattern: applying regex r'%s'" %
+                         pattern_re.pattern)
+        for name in files:
+            if pattern_re.search (name):
+                matches.append (name)
+                self.debug_print(" adding " + name)
+
+        return matches
+
+    # select_pattern ()
+
+
+    def exclude_pattern (self, files, pattern, anchor=1, prefix=None):
+        """Remove strings (presumably filenames) from 'files' that match
+        'pattern'.  'pattern', 'anchor', 'and 'prefix' are the same
+        as for 'select_pattern()', above.  The list 'files' is modified
+        in place.
+        """
+        pattern_re = translate_pattern (pattern, anchor, prefix)
+        self.debug_print("exclude_pattern: applying regex r'%s'" %
+                         pattern_re.pattern)
+        for i in range (len(files)-1, -1, -1):
+            if pattern_re.search (files[i]):
+                self.debug_print(" removing " + files[i])
+                del files[i]
+
+    # exclude_pattern ()
+
+
 class sdist (Command):
 
     description = "create a source distribution (tarball, zip file, etc.)"
@@ -300,45 +554,6 @@
     # add_defaults ()
     
 
-    def search_dir (self, dir, pattern=None):
-        """Recursively find files under 'dir' matching 'pattern' (a string
-        containing a Unix-style glob pattern).  If 'pattern' is None, find
-        all files under 'dir'.  Return the list of found filenames.
-        """
-        allfiles = findall (dir)
-        if pattern is None:
-            return allfiles
-
-        pattern_re = translate_pattern (pattern)
-        files = []
-        for file in allfiles:
-            if pattern_re.match (os.path.basename (file)):
-                files.append (file)
-
-        return files
-
-    # search_dir ()
-
-
-    def recursive_exclude_pattern (self, dir, pattern=None):
-        """Remove filenames from 'self.files' that are under 'dir' and
-        whose basenames match 'pattern'.
-        """
-        self.debug_print("recursive_exclude_pattern: dir=%s, pattern=%s" %
-                         (dir, pattern))
-        if pattern is None:
-            pattern_re = None
-        else:
-            pattern_re = translate_pattern (pattern)
-
-        for i in range (len (self.files)-1, -1, -1):
-            (cur_dir, cur_base) = os.path.split (self.files[i])
-            if (cur_dir == dir and
-                (pattern_re is None or pattern_re.match (cur_base))):
-                self.debug_print("removing %s" % self.files[i])
-                del self.files[i]
-
-
     def read_template (self):
         """Read and parse the manifest template file named by
         'self.template' (usually "MANIFEST.in").  Process all file
@@ -357,151 +572,16 @@
                              rstrip_ws=1,
                              collapse_ws=1)
 
-        all_files = findall ()
 
-        while 1:
+	template_processor = Template_Processor(self.files)
+	template_processor.set_warnings_function(template.warn)
+	template_processor.set_debug_print_function(self.debug_print)
 
-            line = template.readline()
+	while 1:
+	    line = template.readline()
             if line is None:            # end of file
                 break
-
-            words = string.split (line)
-            action = words[0]
-
-            # First, check that the right number of words are present
-            # for the given action (which is the first word)
-            if action in ('include','exclude',
-                          'global-include','global-exclude'):
-                if len (words) < 2:
-                    template.warn \
-                        ("invalid manifest template line: " +
-                         "'%s' expects <pattern1> <pattern2> ..." %
-                         action)
-                    continue
-
-                pattern_list = map(convert_path, words[1:])
-
-            elif action in ('recursive-include','recursive-exclude'):
-                if len (words) < 3:
-                    template.warn \
-                        ("invalid manifest template line: " +
-                         "'%s' expects <dir> <pattern1> <pattern2> ..." %
-                         action)
-                    continue
-
-                dir = convert_path(words[1])
-                pattern_list = map (convert_path, words[2:])
-
-            elif action in ('graft','prune'):
-                if len (words) != 2:
-                    template.warn \
-                        ("invalid manifest template line: " +
-                         "'%s' expects a single <dir_pattern>" %
-                         action)
-                    continue
-
-                dir_pattern = convert_path (words[1])
-
-            else:
-                template.warn ("invalid manifest template line: " +
-                               "unknown action '%s'" % action)
-                continue
-
-            # OK, now we know that the action is valid and we have the
-            # right number of words on the line for that action -- so we
-            # can proceed with minimal error-checking.  Also, we have
-            # defined either (pattern), (dir and pattern), or
-            # (dir_pattern) -- so we don't have to spend any time
-            # digging stuff up out of 'words'.
-
-            if action == 'include':
-                self.debug_print("include " + string.join(pattern_list))
-                for pattern in pattern_list:
-                    files = self.select_pattern (all_files, pattern, anchor=1)
-                    if not files:
-                        template.warn ("no files found matching '%s'" %
-                                       pattern)
-                    else:
-                        self.files.extend (files)
-
-            elif action == 'exclude':
-                self.debug_print("exclude " + string.join(pattern_list))
-                for pattern in pattern_list:
-                    num = self.exclude_pattern (self.files, pattern, anchor=1)
-                    if num == 0:
-                        template.warn (
-                            "no previously-included files found matching '%s'"%
-                            pattern)
-
-            elif action == 'global-include':
-                self.debug_print("global-include " + string.join(pattern_list))
-                for pattern in pattern_list:
-                    files = self.select_pattern (all_files, pattern, anchor=0)
-                    if not files:
-                        template.warn (("no files found matching '%s' " +
-                                        "anywhere in distribution") %
-                                       pattern)
-                    else:
-                        self.files.extend (files)
-
-            elif action == 'global-exclude':
-                self.debug_print("global-exclude " + string.join(pattern_list))
-                for pattern in pattern_list:
-                    num = self.exclude_pattern (self.files, pattern, anchor=0)
-                    if num == 0:
-                        template.warn \
-                            (("no previously-included files matching '%s' " +
-                              "found anywhere in distribution") %
-                             pattern)
-
-            elif action == 'recursive-include':
-                self.debug_print("recursive-include %s %s" %
-                                 (dir, string.join(pattern_list)))
-                for pattern in pattern_list:
-                    files = self.select_pattern (
-                        all_files, pattern, prefix=dir)
-                    if not files:
-                        template.warn (("no files found matching '%s' " +
-                                        "under directory '%s'") %
-                                       (pattern, dir))
-                    else:
-                        self.files.extend (files)
-
-            elif action == 'recursive-exclude':
-                self.debug_print("recursive-exclude %s %s" %
-                                 (dir, string.join(pattern_list)))
-                for pattern in pattern_list:
-                    num = self.exclude_pattern(
-                        self.files, pattern, prefix=dir)
-                    if num == 0:
-                        template.warn \
-                            (("no previously-included files matching '%s' " +
-                              "found under directory '%s'") %
-                             (pattern, dir))
-
-            elif action == 'graft':
-                self.debug_print("graft " + dir_pattern)
-                files = self.select_pattern(
-                    all_files, None, prefix=dir_pattern)
-                if not files:
-                    template.warn ("no directories found matching '%s'" %
-                                   dir_pattern)
-                else:
-                    self.files.extend (files)
-
-            elif action == 'prune':
-                self.debug_print("prune " + dir_pattern)
-                num = self.exclude_pattern(
-                    self.files, None, prefix=dir_pattern)
-                if num == 0:
-                    template.warn \
-                        (("no previously-included directories found " +
-                          "matching '%s'") %
-                         dir_pattern)
-            else:
-                raise RuntimeError, \
-                      "this cannot happen: invalid action '%s'" % action
-
+	    template_processor.process_line(line)
         # while loop over lines of template file
 
     # read_template ()
@@ -516,57 +596,11 @@
         """
         build = self.get_finalized_command('build')
         base_dir = self.distribution.get_fullname()
-        self.exclude_pattern (self.files, None, prefix=build.build_base)
-        self.exclude_pattern (self.files, None, prefix=base_dir)
-
-
-    def select_pattern (self, files, pattern, anchor=1, prefix=None):
-        """Select strings (presumably filenames) from 'files' that match
-        'pattern', a Unix-style wildcard (glob) pattern.  Patterns are not
-        quite the same as implemented by the 'fnmatch' module: '*' and '?'
-        match non-special characters, where "special" is platform-dependent:
-        slash on Unix, colon, slash, and backslash on DOS/Windows, and colon on
-        Mac OS.
-
-        If 'anchor' is true (the default), then the pattern match is more
-        stringent: "*.py" will match "foo.py" but not "foo/bar.py".  If
-        'anchor' is false, both of these will match.
-
-        If 'prefix' is supplied, then only filenames starting with 'prefix'
-        (itself a pattern) and ending with 'pattern', with anything in between
-        them, will match.  'anchor' is ignored in this case.
-
-        Return the list of matching strings, possibly empty.
-        """
-        matches = []
-        pattern_re = translate_pattern (pattern, anchor, prefix)
-        self.debug_print("select_pattern: applying regex r'%s'" %
-                         pattern_re.pattern)
-        for name in files:
-            if pattern_re.search (name):
-                matches.append (name)
-                self.debug_print(" adding " + name)
-
-        return matches
-
-    # select_pattern ()
-
-
-    def exclude_pattern (self, files, pattern, anchor=1, prefix=None):
-        """Remove strings (presumably filenames) from 'files' that match
-        'pattern'.  'pattern', 'anchor', 'and 'prefix' are the same
-        as for 'select_pattern()', above.  The list 'files' is modified
-        in place.
-        """
-        pattern_re = translate_pattern (pattern, anchor, prefix)
-        self.debug_print("exclude_pattern: applying regex r'%s'" %
-                         pattern_re.pattern)
-        for i in range (len(files)-1, -1, -1):
-            if pattern_re.search (files[i]):
-                self.debug_print(" removing " + files[i])
-                del files[i]
-
-    # exclude_pattern ()
+	template_processor = Template_Processor(self.files)
+	# warnings are not of interest here
+	template_processor.set_debug_print_function(self.debug_print)
+        template_processor.process_line("prune " + build.build_base)
+        template_processor.process_line("prune " + base_dir)
 
 
     def write_manifest (self):
@@ -688,7 +722,8 @@
                 fullname = os.path.join (dir, name)
             else:
                 fullname = name
-            list.append (fullname)
+            if not os.path.isdir (fullname):
+		list.append (fullname)
             if os.path.isdir (fullname) and not os.path.islink(fullname):
                 push (fullname)
 
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/distutils.cfg distutils/distutils/distutils.cfg
--- distutils.orig/distutils/distutils.cfg	Thu Jan  1 01:00:00 1970
+++ distutils/distutils/distutils.cfg	Mon Jun 26 12:24:29 2000
@@ -0,0 +1,18 @@
+#
+# distutils.cfg
+#
+# the site-wide Distutils config file. 
+# This has the purpose of serving as an example of some of 
+# the things you can do with Distutils config files.  
+#
+# $Id: distutils.cfg $
+#
+
+[sdist]
+#formats=gztar,zip
+
+[build]
+#compiler=unix
+
+[bdist]
+#formats=gztar,rpm
diff -BurN --minimal --exclude=*.pyc distutils.orig/setup.py distutils/setup.py
--- distutils.orig/setup.py	Mon Jun 26 12:47:34 2000
+++ distutils/setup.py	Mon Jun 26 12:43:30 2000
@@ -9,6 +9,7 @@
 __revision__ = "$Id: setup.py,v 1.16 2000/06/02 02:23:42 gward Exp $"
 
 from distutils.core import setup
+from distutils.command.install_data import Data_Files
 
 setup (name = "Distutils",
        version = "0.9pre",
@@ -26,4 +27,23 @@
        # This implies all pure Python modules in ./distutils/ and
        # ./distutils/command/
        packages = ['distutils', 'distutils.command'],
+
+       # non python files      
+       data_files = [# ("lib/python1.5/site-packages/distutils",["distutils/distutils.cfg"]), # old style
+	    Data_Files(
+		base_dir='install_lib',
+		files=["distutils/distutils.cfg"],
+		preserve_path=1,
+	    ),       
+#	    Data_Files(
+#		base_dir='install_lib',
+#		files=["distutils/distutils.cfg"],
+#		copy_to='distutils'
+#	    ),       
+#	    Data_Files(
+#		base_dir='install_lib',
+#		template="recursive-include distutils *.cfg",
+#		preserve_path=1,
+#	    ),       
+       ],
       )

--------------1B9072FF001699F74DA18394--