Behaviour of "install_data" changed

I've just fixed the behaviour of the "install_data" command: the default directory for installating data files is now just the installation base, which is usually sys.prefix. Thus, if you put
data_files = ["my_data"]
in your setup script, you will wind up installing (eg.) /usr/local/my_data -- definitely the wrong thing on Unix, and probably wrong on Windows too. (Except for applications that have their own prefix, but that's not really dealt with yet.)
This is the Right Thing now because of the change to "install_data" that lets you specify where to put data files; the above should be spelled
data_files = [("share", ["my_data"])]
which will install /usr/local/share/my_data.
Of course, this still doesn't solve the problem of, "How does my application/module know where Distutils installed this data file if the user did some wild funky custom installation?". Oh well, better than nothing.
Greg

Greg Ward wrote:
I've just fixed the behaviour of the "install_data" command: the default directory for installating data files is now just the installation base, which is usually sys.prefix. Thus, if you put
data_files = ["my_data"]
in your setup script, you will wind up installing (eg.) /usr/local/my_data -- definitely the wrong thing on Unix, and probably wrong on Windows too. (Except for applications that have their own prefix, but that's not really dealt with yet.)
This is the Right Thing now because of the change to "install_data" that lets you specify where to put data files; the above should be spelled
data_files = [("share", ["my_data"])]
which will install /usr/local/share/my_data.
Of course, this still doesn't solve the problem of, "How does my application/module know where Distutils installed this data file if the user did some wild funky custom installation?". Oh well, better than nothing.
Would you like to have a complete new concept, which could solve all your problems? Here it is.
I would like to introduce a class 'Data_Files' similar to the 'Extension' class. It should accept following parameters: * A basis directory to which all other filenames will become relative. base_dir : 'install_data','install_lib','install_headers' or any other path which is defined in 'install' * A directory under this base dir like the 'share' dir in Greg's example above. copy_to : a path name * a files list as usual files : ['foo','bar', ...] * a list of templates as in MANIFEST.in, which is used to manipulate the files list template : a list or a ';' separeted string * a flag which switches the preserving of paths on or off ( file 'foo/bar' would copied to 'base_dir/foo/bar', instead of 'base_dir/bar' ) preserve_path : 0 or 1
How to use this? Consider you had a distutils.cfg template which you want to install. There are now different ways to so:
data_files = [ # The old way (the patch doesn't break old code.): ("lib/python1.5/site-packages/distutils",["distutils/distutils.cfg"]), # this is not portable with win32
# the same using a different base dir Data_Files( base_dir='install_lib', files=["distutils/distutils.cfg"], copy_to='distutils' ), # much better, but you still have to specify the target directory
# without target directory Data_Files( base_dir='install_lib', files=["distutils/distutils.cfg"], preserve_path=1, ), # the file has already the rght path so we can use this
# and now the most sophisticated solution: Data_Files( base_dir='install_lib', template="recursive-include distutils *.cfg", preserve_path=1, ),
I think this can solve all our problems.
Of course, this still doesn't solve the problem of, "How does my application/module know where Distutils installed this data file if the user did some wild funky custom installation?".
You only have to create a config file with the path of the other files and put this config file in your packages directory. (see the example above)
Maybe you are not convinced, so here a more complex example. It is taken from a distutils setup.py file for PyOpenGL. (One of the packages I use to test distutils.) It installs some examples in its package directory, these contain data files and some images. Using the old way looks like this: # non python files of examples data_files = [ ('lib/python1.5/site-packages/OpenGL/Demo/dek',[ "OpenGL/Demo/dek/README", "OpenGL/Demo/dek/image.ppm" ]), ('lib/python1.5/site-packages/OpenGL/Demo/dek/OglSurface',[ "OpenGL/Demo/dek/OglSurface/1crn.face", "OpenGL/Demo/dek/OglSurface/1crn.pdb", "OpenGL/Demo/dek/OglSurface/1crn.vert", "OpenGL/Demo/dek/OglSurface/1crn.xyzr", "OpenGL/Demo/dek/OglSurface/README", "OpenGL/Demo/dek/OglSurface/test.ppm", ]), ('lib/python1.5/site-packages/OpenGL/Demo/srenner',[ "OpenGL/Demo/srenner/README", ]), ('lib/python1.5/site-packages/OpenGL/Demo/srenner/Images', glob (os.path.join ("OpenGL/Demo/srenner/Images","*.ppm")) ), ], And the new way is much simpler (and more portable): # non python files of examples data_files = [ Data_Files( base_dir='install_lib', # py-files shouldn't be installed with install_data template=["graft OpenGL","exclude *.py*"] , preserve_path=1, ) ],
What does the patch change in distutils?
First I created the class 'Data_Files', which is currently located in install_data, maybe it should get its own file as Extension did. There were also some changes in install_data to work with this new class.
Then I had to take the template proccesing out of the sdist command. It is now a seperate class in sdist.py. This was necessary because I want to use it in install_data.
And finally the patch creates a distutils.cfg template and changes distutils setup.py, so it would install this template.
Do you like it so, or is there anything to change? (Maybe some names could get better names, but this not a big problem.)
kind regards
Rene Liebscher
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/install_data.py distutils/distutils/command/install_data.py --- distutils.orig/distutils/command/install_data.py Mon Jun 26 09:34:37 2000 +++ distutils/distutils/command/install_data.py Mon Jun 26 12:51:52 2000 @@ -7,10 +7,59 @@
__revision__ = "$Id: install_data.py,v 1.10 2000/06/24 17:36:24 gward Exp $"
-import os -from types import StringType +import os,sys,string +from types import StringType,TupleType,ListType from distutils.core import Command from distutils.util import change_root +from distutils.command.sdist import Template_Processor + +class Data_Files: + """ container for list of data files. + supports alternate base_dirs e.g. 'install_lib','install_header',... + supports a directory where to copy files + supports templates as in MANIFEST.in + supports preserving of paths in filenames eg. foo/xyz is copied to base_dir/foo/xyz + """ + + def __init__(self,base_dir=None,files=None,copy_to=None,template=None,preserve_path=0): + self.base_dir = base_dir + self.files = files + self.copy_to = copy_to + self.template = template + self.preserve_path = preserve_path + self.finalized = 0 + + def warn (self, msg): + sys.stderr.write ("warning: %s: %s\n" % + ("install_data", msg)) + + def debug_print (self, msg): + """Print 'msg' to stdout if the global DEBUG (taken from the + DISTUTILS_DEBUG environment variable) flag is true. + """ + from distutils.core import DEBUG + if DEBUG: + print msg + + + def finalize(self): + """ complete the files list by processing the given template """ + if self.finalized: + return + if self.files == None: + self.files = [] + if self.template != None: + if type(self.template) == StringType: + self.template = string.split(self.template,";") + template_processor = Template_Processor(self.files) + template_processor.set_warnings_function(self.warn) + template_processor.set_debug_print_function(self.debug_print) + for line in self.template: + template_processor.process_line(string.strip(line)) + self.finalized = 1 + +# end class Data_Files +
class install_data (Command):
@@ -32,34 +81,86 @@
def finalize_options (self): self.set_undefined_options('install', - ('install_data', 'install_dir'), - ('root', 'root'), - ) - + ('install_data', 'install_dir'), + ('root', 'root'), + ) + + def check_data(self,d): + """ check if data are in new format, if not create a suitable object. + returns finalized data object + """ + if not isinstance(d, Data_Files): + self.warn(("old-style data files list found " + "-- please convert to Data_Files instance")) + if type(d) is TupleType: + if len(d) != 2 or not (type(d[1]) is ListType): + raise DistutilsSetupError, \ + ("each element of 'data_files' option must be an " + "Data File instance, a string or 2-tuple (string,[strings])") + d = Data_Files(copy_to=d[0],files=d[1]) + else: + if not (type(d) is StringType): + raise DistutilsSetupError, \ + ("each element of 'data_files' option must be an " + "Data File instance, a string or 2-tuple (string,[strings])") + d = Data_Files(files=[d]) + d.finalize() + return d + + def run (self): - self.mkpath(self.install_dir) - for f in self.data_files: - if type(f) == StringType: - # it's a simple file, so copy it - self.warn("setup script did not provide a directory for " - "'%s' -- installing right in '%s'" % - (f, self.install_dir)) - out = self.copy_file(f, self.install_dir) - self.outfiles.append(out) + self.outfiles = [] + install_cmd = self.get_finalized_command('install') + + for d in self.data_files: + d = self.check_data(d) + + install_dir = self.install_dir + # alternative base dir given => overwrite install_dir + if d.base_dir != None: + install_dir = getattr(install_cmd,d.base_dir) + + # copy to an other directory + if d.copy_to != None: + if not os.path.isabs(d.copy_to): + # relatiev path to install_dir + dir = os.path.join(install_dir, d.copy_to) + elif install_cmd.root: + # absolute path and alternative root set + dir = change_root(self.root,d.copy_to) + else: + # absolute path + dir = d.copy_to else: - # it's a tuple with path to install to and a list of files - dir = f[0] - if not os.path.isabs(dir): - dir = os.path.join(self.install_dir, dir) - elif self.root: - dir = change_root(self.root, dir) - self.mkpath(dir) - for data in f[1]: - out = self.copy_file(data, dir) - self.outfiles.append(out) + # simply copy to install_dir + dir = install_dir + + dir=os.path.normpath(dir) + # create path + self.mkpath(dir) + + # copy all files + for f in d.files: + if d.copy_to == None: + self.warn("setup script did not provide a directory for " + "'%s' -- installing right in '%s'" % + (f, install_dir)) + if d.preserve_path: + # preserve path in filename + self.mkpath(os.path.dirname(os.path.join(dir,f))) + out = self.copy_file(f, os.path.join(dir,f)) + else: + out = self.copy_file(f, dir) + self.outfiles.append(out) + + return self.outfiles
def get_inputs (self): - return self.data_files or [] - + inputs = [] + for d in self.data_files: + d = self.check_data(d) + inputs.append(d.files) + return inputs + def get_outputs (self): return self.outfiles diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/command/sdist.py distutils/distutils/command/sdist.py --- distutils.orig/distutils/command/sdist.py Mon Jun 26 09:34:37 2000 +++ distutils/distutils/command/sdist.py Mon Jun 26 12:39:08 2000 @@ -34,6 +34,260 @@ "List of available source distribution formats:")
+class Template_Processor: + + def __init__(self, files_list=[]): + self.files=files_list + self.all_files = findall () + self.template_warn=self.dummy_function + self.debug_print=self.dummy_function + + def set_warnings_function(self, warnings_function): + self.template_warn=warnings_function + + def set_debug_print_function(self, debug_print_function): + self.debug_print=debug_print_function + + def dummy_function(self,warning): + pass + + def search_dir (self, dir, pattern=None): + """Recursively find files under 'dir' matching 'pattern' (a string + containing a Unix-style glob pattern). If 'pattern' is None, find + all files under 'dir'. Return the list of found filenames. + """ + allfiles = findall (dir) + if pattern is None: + return allfiles + + pattern_re = translate_pattern (pattern) + files = [] + for file in allfiles: + if pattern_re.match (os.path.basename (file)): + files.append (file) + + return files + + # search_dir () + + def recursive_exclude_pattern (self, dir, pattern=None): + """Remove filenames from 'self.files' that are under 'dir' and + whose basenames match 'pattern'. + """ + self.debug_print("recursive_exclude_pattern: dir=%s, pattern=%s" % + (dir, pattern)) + if pattern is None: + pattern_re = None + else: + pattern_re = translate_pattern (pattern) + + for i in range (len (self.files)-1, -1, -1): + (cur_dir, cur_base) = os.path.split (self.files[i]) + if (cur_dir == dir and + (pattern_re is None or pattern_re.match (cur_base))): + self.debug_print("removing %s" % self.files[i]) + del self.files[i] + + + def process_line (self, line): + """Read and parse a line of a template file. Process all file + specifications (include and exclude) in the template and + update 'self.files' accordingly (filenames may be added to + or removed from 'self.files' based on the template). + """ + assert self.files is not None and type (self.files) is ListType + + + words = string.split (line) + action = words[0] + + # First, check that the right number of words are present + # for the given action (which is the first word) + if action in ('include','exclude', + 'global-include','global-exclude'): + if len (words) < 2: + self.template_warn \ + ("invalid manifest template line: " + + "'%s' expects <pattern1> <pattern2> ..." % + action) + return + + pattern_list = map(convert_path, words[1:]) + + elif action in ('recursive-include','recursive-exclude'): + if len (words) < 3: + self.template_warn \ + ("invalid manifest template line: " + + "'%s' expects <dir> <pattern1> <pattern2> ..." % + action) + return + + dir = convert_path(words[1]) + pattern_list = map (convert_path, words[2:]) + + elif action in ('graft','prune'): + if len (words) != 2: + self.template_warn \ + ("invalid manifest template line: " + + "'%s' expects a single <dir_pattern>" % + action) + return + + dir_pattern = convert_path (words[1]) + + else: + self.template_warn ("invalid manifest template line: " + + "unknown action '%s'" % action) + return + + # OK, now we know that the action is valid and we have the + # right number of words on the line for that action -- so we + # can proceed with minimal error-checking. Also, we have + # defined either (pattern), (dir and pattern), or + # (dir_pattern) -- so we don't have to spend any time + # digging stuff up out of 'words'. + + if action == 'include': + self.debug_print("include " + string.join(pattern_list)) + for pattern in pattern_list: + files = self.select_pattern (self.all_files, pattern, anchor=1) + if not files: + self.template_warn ("no files found matching '%s'" % + pattern) + else: + self.files.extend (files) + + elif action == 'exclude': + self.debug_print("exclude " + string.join(pattern_list)) + for pattern in pattern_list: + num = self.exclude_pattern (self.files, pattern, anchor=1) + if num == 0: + self.template_warn ( + "no previously-included files found matching '%s'"% + pattern) + + elif action == 'global-include': + self.debug_print("global-include " + string.join(pattern_list)) + for pattern in pattern_list: + files = self.select_pattern (self.all_files, pattern, anchor=0) + if not files: + self.template_warn (("no files found matching '%s' " + + "anywhere in distribution") % + pattern) + else: + self.files.extend (files) + + elif action == 'global-exclude': + self.debug_print("global-exclude " + string.join(pattern_list)) + for pattern in pattern_list: + num = self.exclude_pattern (self.files, pattern, anchor=0) + if num == 0: + self.template_warn \ + (("no previously-included files matching '%s' " + + "found anywhere in distribution") % + pattern) + + elif action == 'recursive-include': + self.debug_print("recursive-include %s %s" % + (dir, string.join(pattern_list))) + for pattern in pattern_list: + files = self.select_pattern ( + self.all_files, pattern, prefix=dir) + if not files: + self.template_warn (("no files found matching '%s' " + + "under directory '%s'") % + (pattern, dir)) + else: + self.files.extend (files) + + elif action == 'recursive-exclude': + self.debug_print("recursive-exclude %s %s" % + (dir, string.join(pattern_list))) + for pattern in pattern_list: + num = self.exclude_pattern( + self.files, pattern, prefix=dir) + if num == 0: + self.template_warn \ + (("no previously-included files matching '%s' " + + "found under directory '%s'") % + (pattern, dir)) + + elif action == 'graft': + self.debug_print("graft " + dir_pattern) + files = self.select_pattern( + self.all_files, None, prefix=dir_pattern) + if not files: + self.template_warn ("no directories found matching '%s'" % + dir_pattern) + else: + self.files.extend (files) + + elif action == 'prune': + self.debug_print("prune " + dir_pattern) + num = self.exclude_pattern( + self.files, None, prefix=dir_pattern) + if num == 0: + self.template_warn \ + (("no previously-included directories found " + + "matching '%s'") % + dir_pattern) + else: + raise RuntimeError, \ + "this cannot happen: invalid action '%s'" % action + + # process_line () + + + + def select_pattern (self, files, pattern, anchor=1, prefix=None): + """Select strings (presumably filenames) from 'files' that match + 'pattern', a Unix-style wildcard (glob) pattern. Patterns are not + quite the same as implemented by the 'fnmatch' module: '*' and '?' + match non-special characters, where "special" is platform-dependent: + slash on Unix, colon, slash, and backslash on DOS/Windows, and colon on + Mac OS. + + If 'anchor' is true (the default), then the pattern match is more + stringent: "*.py" will match "foo.py" but not "foo/bar.py". If + 'anchor' is false, both of these will match. + + If 'prefix' is supplied, then only filenames starting with 'prefix' + (itself a pattern) and ending with 'pattern', with anything in between + them, will match. 'anchor' is ignored in this case. + + Return the list of matching strings, possibly empty. + """ + matches = [] + pattern_re = translate_pattern (pattern, anchor, prefix) + self.debug_print("select_pattern: applying regex r'%s'" % + pattern_re.pattern) + for name in files: + if pattern_re.search (name): + matches.append (name) + self.debug_print(" adding " + name) + + return matches + + # select_pattern () + + + def exclude_pattern (self, files, pattern, anchor=1, prefix=None): + """Remove strings (presumably filenames) from 'files' that match + 'pattern'. 'pattern', 'anchor', 'and 'prefix' are the same + as for 'select_pattern()', above. The list 'files' is modified + in place. + """ + pattern_re = translate_pattern (pattern, anchor, prefix) + self.debug_print("exclude_pattern: applying regex r'%s'" % + pattern_re.pattern) + for i in range (len(files)-1, -1, -1): + if pattern_re.search (files[i]): + self.debug_print(" removing " + files[i]) + del files[i] + + # exclude_pattern () + + class sdist (Command):
description = "create a source distribution (tarball, zip file, etc.)" @@ -300,45 +554,6 @@ # add_defaults ()
- def search_dir (self, dir, pattern=None): - """Recursively find files under 'dir' matching 'pattern' (a string - containing a Unix-style glob pattern). If 'pattern' is None, find - all files under 'dir'. Return the list of found filenames. - """ - allfiles = findall (dir) - if pattern is None: - return allfiles - - pattern_re = translate_pattern (pattern) - files = [] - for file in allfiles: - if pattern_re.match (os.path.basename (file)): - files.append (file) - - return files - - # search_dir () - - - def recursive_exclude_pattern (self, dir, pattern=None): - """Remove filenames from 'self.files' that are under 'dir' and - whose basenames match 'pattern'. - """ - self.debug_print("recursive_exclude_pattern: dir=%s, pattern=%s" % - (dir, pattern)) - if pattern is None: - pattern_re = None - else: - pattern_re = translate_pattern (pattern) - - for i in range (len (self.files)-1, -1, -1): - (cur_dir, cur_base) = os.path.split (self.files[i]) - if (cur_dir == dir and - (pattern_re is None or pattern_re.match (cur_base))): - self.debug_print("removing %s" % self.files[i]) - del self.files[i] - - def read_template (self): """Read and parse the manifest template file named by 'self.template' (usually "MANIFEST.in"). Process all file @@ -357,151 +572,16 @@ rstrip_ws=1, collapse_ws=1)
- all_files = findall ()
- while 1: + template_processor = Template_Processor(self.files) + template_processor.set_warnings_function(template.warn) + template_processor.set_debug_print_function(self.debug_print)
- line = template.readline() + while 1: + line = template.readline() if line is None: # end of file break - - words = string.split (line) - action = words[0] - - # First, check that the right number of words are present - # for the given action (which is the first word) - if action in ('include','exclude', - 'global-include','global-exclude'): - if len (words) < 2: - template.warn \ - ("invalid manifest template line: " + - "'%s' expects <pattern1> <pattern2> ..." % - action) - continue - - pattern_list = map(convert_path, words[1:]) - - elif action in ('recursive-include','recursive-exclude'): - if len (words) < 3: - template.warn \ - ("invalid manifest template line: " + - "'%s' expects <dir> <pattern1> <pattern2> ..." % - action) - continue - - dir = convert_path(words[1]) - pattern_list = map (convert_path, words[2:]) - - elif action in ('graft','prune'): - if len (words) != 2: - template.warn \ - ("invalid manifest template line: " + - "'%s' expects a single <dir_pattern>" % - action) - continue - - dir_pattern = convert_path (words[1]) - - else: - template.warn ("invalid manifest template line: " + - "unknown action '%s'" % action) - continue - - # OK, now we know that the action is valid and we have the - # right number of words on the line for that action -- so we - # can proceed with minimal error-checking. Also, we have - # defined either (pattern), (dir and pattern), or - # (dir_pattern) -- so we don't have to spend any time - # digging stuff up out of 'words'. - - if action == 'include': - self.debug_print("include " + string.join(pattern_list)) - for pattern in pattern_list: - files = self.select_pattern (all_files, pattern, anchor=1) - if not files: - template.warn ("no files found matching '%s'" % - pattern) - else: - self.files.extend (files) - - elif action == 'exclude': - self.debug_print("exclude " + string.join(pattern_list)) - for pattern in pattern_list: - num = self.exclude_pattern (self.files, pattern, anchor=1) - if num == 0: - template.warn ( - "no previously-included files found matching '%s'"% - pattern) - - elif action == 'global-include': - self.debug_print("global-include " + string.join(pattern_list)) - for pattern in pattern_list: - files = self.select_pattern (all_files, pattern, anchor=0) - if not files: - template.warn (("no files found matching '%s' " + - "anywhere in distribution") % - pattern) - else: - self.files.extend (files) - - elif action == 'global-exclude': - self.debug_print("global-exclude " + string.join(pattern_list)) - for pattern in pattern_list: - num = self.exclude_pattern (self.files, pattern, anchor=0) - if num == 0: - template.warn \ - (("no previously-included files matching '%s' " + - "found anywhere in distribution") % - pattern) - - elif action == 'recursive-include': - self.debug_print("recursive-include %s %s" % - (dir, string.join(pattern_list))) - for pattern in pattern_list: - files = self.select_pattern ( - all_files, pattern, prefix=dir) - if not files: - template.warn (("no files found matching '%s' " + - "under directory '%s'") % - (pattern, dir)) - else: - self.files.extend (files) - - elif action == 'recursive-exclude': - self.debug_print("recursive-exclude %s %s" % - (dir, string.join(pattern_list))) - for pattern in pattern_list: - num = self.exclude_pattern( - self.files, pattern, prefix=dir) - if num == 0: - template.warn \ - (("no previously-included files matching '%s' " + - "found under directory '%s'") % - (pattern, dir)) - - elif action == 'graft': - self.debug_print("graft " + dir_pattern) - files = self.select_pattern( - all_files, None, prefix=dir_pattern) - if not files: - template.warn ("no directories found matching '%s'" % - dir_pattern) - else: - self.files.extend (files) - - elif action == 'prune': - self.debug_print("prune " + dir_pattern) - num = self.exclude_pattern( - self.files, None, prefix=dir_pattern) - if num == 0: - template.warn \ - (("no previously-included directories found " + - "matching '%s'") % - dir_pattern) - else: - raise RuntimeError, \ - "this cannot happen: invalid action '%s'" % action - + template_processor.process_line(line) # while loop over lines of template file
# read_template () @@ -516,57 +596,11 @@ """ build = self.get_finalized_command('build') base_dir = self.distribution.get_fullname() - self.exclude_pattern (self.files, None, prefix=build.build_base) - self.exclude_pattern (self.files, None, prefix=base_dir) - - - def select_pattern (self, files, pattern, anchor=1, prefix=None): - """Select strings (presumably filenames) from 'files' that match - 'pattern', a Unix-style wildcard (glob) pattern. Patterns are not - quite the same as implemented by the 'fnmatch' module: '*' and '?' - match non-special characters, where "special" is platform-dependent: - slash on Unix, colon, slash, and backslash on DOS/Windows, and colon on - Mac OS. - - If 'anchor' is true (the default), then the pattern match is more - stringent: "*.py" will match "foo.py" but not "foo/bar.py". If - 'anchor' is false, both of these will match. - - If 'prefix' is supplied, then only filenames starting with 'prefix' - (itself a pattern) and ending with 'pattern', with anything in between - them, will match. 'anchor' is ignored in this case. - - Return the list of matching strings, possibly empty. - """ - matches = [] - pattern_re = translate_pattern (pattern, anchor, prefix) - self.debug_print("select_pattern: applying regex r'%s'" % - pattern_re.pattern) - for name in files: - if pattern_re.search (name): - matches.append (name) - self.debug_print(" adding " + name) - - return matches - - # select_pattern () - - - def exclude_pattern (self, files, pattern, anchor=1, prefix=None): - """Remove strings (presumably filenames) from 'files' that match - 'pattern'. 'pattern', 'anchor', 'and 'prefix' are the same - as for 'select_pattern()', above. The list 'files' is modified - in place. - """ - pattern_re = translate_pattern (pattern, anchor, prefix) - self.debug_print("exclude_pattern: applying regex r'%s'" % - pattern_re.pattern) - for i in range (len(files)-1, -1, -1): - if pattern_re.search (files[i]): - self.debug_print(" removing " + files[i]) - del files[i] - - # exclude_pattern () + template_processor = Template_Processor(self.files) + # warnings are not of interest here + template_processor.set_debug_print_function(self.debug_print) + template_processor.process_line("prune " + build.build_base) + template_processor.process_line("prune " + base_dir)
def write_manifest (self): @@ -688,7 +722,8 @@ fullname = os.path.join (dir, name) else: fullname = name - list.append (fullname) + if not os.path.isdir (fullname): + list.append (fullname) if os.path.isdir (fullname) and not os.path.islink(fullname): push (fullname)
diff -BurN --minimal --exclude=*.pyc distutils.orig/distutils/distutils.cfg distutils/distutils/distutils.cfg --- distutils.orig/distutils/distutils.cfg Thu Jan 1 01:00:00 1970 +++ distutils/distutils/distutils.cfg Mon Jun 26 12:24:29 2000 @@ -0,0 +1,18 @@ +# +# distutils.cfg +# +# the site-wide Distutils config file. +# This has the purpose of serving as an example of some of +# the things you can do with Distutils config files. +# +# $Id: distutils.cfg $ +# + +[sdist] +#formats=gztar,zip + +[build] +#compiler=unix + +[bdist] +#formats=gztar,rpm diff -BurN --minimal --exclude=*.pyc distutils.orig/setup.py distutils/setup.py --- distutils.orig/setup.py Mon Jun 26 12:47:34 2000 +++ distutils/setup.py Mon Jun 26 12:43:30 2000 @@ -9,6 +9,7 @@ __revision__ = "$Id: setup.py,v 1.16 2000/06/02 02:23:42 gward Exp $"
from distutils.core import setup +from distutils.command.install_data import Data_Files
setup (name = "Distutils", version = "0.9pre", @@ -26,4 +27,23 @@ # This implies all pure Python modules in ./distutils/ and # ./distutils/command/ packages = ['distutils', 'distutils.command'], + + # non python files + data_files = [# ("lib/python1.5/site-packages/distutils",["distutils/distutils.cfg"]), # old style + Data_Files( + base_dir='install_lib', + files=["distutils/distutils.cfg"], + preserve_path=1, + ), +# Data_Files( +# base_dir='install_lib', +# files=["distutils/distutils.cfg"], +# copy_to='distutils' +# ), +# Data_Files( +# base_dir='install_lib', +# template="recursive-include distutils *.cfg", +# preserve_path=1, +# ), + ], )

On 26 June 2000, Rene Liebscher said:
Would you like to have a complete new concept, which could solve all your problems? Here it is.
Aiieeee! "Complete" or "complicated"?!?
I would like to introduce a class 'Data_Files' similar to the 'Extension' class.
Conceptual problem here: Python extensions are fairly complicated beasts, with lots of knobs to twiddle. Data files are not: they have a source filename, a destination directory, and that's about it.
I kinda like the ability to specify an "abstract base directory" ie. one of the keys to the installation scheme dictionary. But that's not compelling enough.
I think the use of the manifest template machinery is complete overkill. What's wrong with os.glob()? Even in a really complex case, where the manifest machinery would be useful, I think it would be better just to expose it as one of the "utility classes" that Distutils makes available to authors of setup scripts, rather than working it intimately into the specification of data files, which ought to be fairly simple.
I think this can solve all our problems.
Except for code bloat. >smirk<
Sorry, that was low. Oh well, sometimes I just have to be mean, and tonight is one of those times.
Final excuse: it's getting to close to Distutils 0.9/Python 1.6b1 to do major renovations. Bug fixes only, please.
Greg

I've just fixed the behaviour of the "install_data" command: the default directory for installating data files is now just the installation base, which is usually sys.prefix. Thus, if you put
data_files = ["my_data"]
in your setup script, you will wind up installing (eg.) /usr/local/my_data -- definitely the wrong thing on Unix, and probably wrong on Windows too. (Except for applications that have their own prefix, but that's not really dealt with yet.)
This is the Right Thing now because of the change to "install_data" that lets you specify where to put data files; the above should be spelled
data_files = [("share", ["my_data"])]
which will install /usr/local/share/my_data.
Of course, this still doesn't solve the problem of, "How does my application/module know where Distutils installed this data file if the user did some wild funky custom installation?". Oh well, better than nothing.
Greg
It works for me.
Context: I'm currently writing a setup-script for Marks win32 extensions. Mark has put a lot of "data files" inside the source code tree: html docs, cfg files, ...
If this is installed into the destination directories, the python-scripts (or PythonWin.exe) will know how to find these files.
Thomas
participants (3)
-
Greg Ward
-
Rene Liebscher
-
Thomas Heller