Greg Ward wrote:

[Berthold Höllmann mentions...]

...

There is support for installing data files, but they get to data directories. When writing setup.py for biggles, I had the problem, that a config file had to go to the package directory. All I did was adding

class my_install_data (install_data): def finalize_options (self): self.set_undefined_options('install', ('install_lib', 'install_dir'), ('root', 'root'), ('force', 'force'), )

and

cmdclass = {'install_data': my_install_data}, data_files = [('biggles', ["src/config.ini"])]

to the setup command line. This got everything to the right place.

[Rene Liebscher replies...]

...

This is probably the simplest possible solution. My approach goes further, it handles also MANIFEST.in-like templates and some other special things, and it was written primary to replace distutils' install_data as whole. (Greg, do you remember my mail of 06/26/2000? http://www.python.org/pipermail/distutils-sig/2000-June/001671.html )

As it turns out, the Berthold's "my_install_data" solution is really a bare minimum -- the "install_data" command should work that way, or have an option to work that way. I wrote a setup script for IDLE that does basically the same as Berthold's, and if the Distutils installed their own config file, the Distutils setup script would also need this -- d'oh! I don't think I should have to extend the Distutils in order to install the Distutils -- droit de seigneur, in a sense. ;-)

Rene, I originally thought your "DataFiles" concept was overkill, but I originally thought having a class to describe extensions was overkill too. If nothing else, introducing a simple class may be the only sane way to maintain backwards compatibility with the current "install_data class. Here's a somewhat simpler conception of the "DataFiles" class...

I changed my my_install_data.py a little bit too.

DataFiles -- represents a collection of files to be installed in a common location, the "base directory".

It is called in my class copy_to, but it is only a name.

Files in the collection are fragmentary paths that are concatenated to the base directory at install time to create a complete installation path.

This is at my class, the files list parameter.

The base directory may be a literal path, or it may use any of the configuration variables allowed in installation directories -- dist_name dist_version dist_fullname py_version py_version_short sys_prefix prefix sys_exec_prefix exec_prefix

as well as the config vars corresponding to the installation directory options to the "install" command (most of which are just one of the entries in an installation scheme): install_base install_platbase install_root install_purelib install_platlib install_headers install_scripts install_data (?maybe obsolete?)

Can you read my thoughts, I was putting this in my class when I got your email.

Thus, a base directory of "$install_base" would expand to /usr/lib/python2.0/site-packages on a typical Linux installation, or C:\Python on a Windows installation. Or you could use "$install_base/mypkg" to install to (eg.) C:\Python\mypkg -- presumably the directory where modules from your distribution get installed. (As usual, Unix paths would be translated to native paths by the Distutils, so that setup scripts port across operating systems.)

(There probably ought to be a special name for "the directory where the highest-level package from this module distribution is installed", which would expand to eg. /usr/lib/python1.5/site-packages/distutils.)

This gets rid of dependence on the manifest machinery -- since that's now been factored out into a separate class (FileList, in distutils.filelist), developers can use it if they want. Or they can use the standard 'glob' module if that's good enough, or they can list files manually if *that's* good enough. Supporting an additional template parameter is about ten lines of code.

For PyOpenGL it looks then like this: ------------------------------------------------------------- data_files = [ Data_Files( copy_to = '$install_lib/OpenGL/Demo', strip_dirs = 2, template=[ # take the whole tree 'graft py/Demo', # python files are already installed 'global-exclude *.py*', 'global-exclude Cvs/*', 'global-exclude CVS/*' ], ) ], ------------------------------------------------------------- For an example how it looks if you have to specify the files even by using glob(), look at the setup file of 4Suite ftp://ftp.fourthought.com/pub/4Suite/4Suite-0.9.1.tar.gz (glob doesn't work recursiv through subdirectories.) To show you what I mean: the code (about 20% of 4Suite's data files section) -------------------------------------- ('4XSLT/demo',glob.glob('Xslt/demo/*.*')), ('4XSLT',['Xslt/README', 'Xslt/ChangeLog', 'Xslt/TODO', ]), ('4XSLT',glob.glob('Xslt/docs/*.*')), ('4XSLT/test_suite',['Xslt/test_suite/README']), ('4XSLT/test_suite',glob.glob('Xslt/test_suite/*.*')), ('4XSLT/test_suite/borrowed',glob.glob('Xslt/test_suite/borrowed/README')), ('4XSLT/test_suite/borrowed',glob.glob('Xslt/test_suite/borrowed/*.*')), ('4XSLT/test_suite/profile_data',glob.glob('Xslt/test_suite/profile_data/*.*')), ('4XSLT/test_suite/graph_trav',glob.glob('Xslt/test_suite/borrowed/graph_trav/README')) , ('4XSLT/test_suite/graph_trav',glob.glob('Xslt/test_suite/borrowed/graph_trav/*.*')), --------------------------------------- could be replaced by something like this --------------------------------------- Data_Files( copy_to = '$install_data/4XSLT', strip_dirs = 1, # this is a number template=[ # take the whole tree 'graft Xslt/demo', 'graft Xslt/doc', 'graft Xslt/test_suite', 'include Xslt/README', 'include Xslt/ChangeLog', 'include Xslt/TODO', ], ) --------------------------------------- I think the second is easier. Kind regards Rene Liebscher """my_install_data.py Provides a more sophisticated facility to install data files than distutils' install_data does. You can specify your files as a template like in MANIFEST.in and you have more control over the copy process. """ # created 2000/08/01, Rene Liebscher <R.Liebscher@gmx.de> ########################################################################### # import some modules we need import os,sys,string from types import StringType,TupleType,ListType from distutils.util import change_root,subst_vars from distutils.filelist import FileList from distutils.command.install_data import install_data ########################################################################### # a container class for our more sophisticated install mechanism class Data_Files: """ container for list of data files. supports a directory where to copy files supports variable substitution e.g. '$install_lib','$install_header',... supports templates as in MANIFEST.in supports preserving of paths in filenames eg. foo/xyz is copied to base_dir/foo/xyz supports stripping of leading dirs of source paths eg. foo/bar1/xyz, foo/bar2/abc can be copied to bar1/xyz, bar2/abc """ def __init__(self,files=None,copy_to=None,template=None,preserve_path=1,strip_dirs=0): self.files = files self.copy_to = copy_to self.template = template self.preserve_path = preserve_path self.strip_dirs = strip_dirs self.finalized = 0 def warn (self, msg): sys.stderr.write ("warning: %s: %s\n" % ("install_data", msg)) def debug_print (self, msg): """Print 'msg' to stdout if the global DEBUG (taken from the DISTUTILS_DEBUG environment variable) flag is true. """ from distutils.core import DEBUG if DEBUG: print msg def finalize(self): """ complete the files list by processing the given template """ if self.finalized: return if self.files == None: self.files = [] if self.template != None: if type(self.template) == StringType: self.template = string.split(self.template,";") filelist = FileList(self.warn,self.debug_print) for line in self.template: filelist.process_template_line(string.strip(line)) filelist.sort() filelist.remove_duplicates() self.files.extend(filelist.files) self.finalized = 1 # end class Data_Files ########################################################################### # a more sophisticated install routine than distutils install_data class my_install_data (install_data): def check_data(self,d): """ check if data are in new format, if not create a suitable object. returns finalized data object """ if not isinstance(d, Data_Files): self.warn(("old-style data files list found " "-- please convert to Data_Files instance")) if type(d) is TupleType: if len(d) != 2 or not (type(d[1]) is ListType): raise DistutilsSetupError, \ ("each element of 'data_files' option must be an " "Data File instance, a string or 2-tuple (string,[strings])") d = Data_Files(copy_to=d[0],files=d[1]) else: if not (type(d) is StringType): raise DistutilsSetupError, \ ("each element of 'data_files' option must be an " "Data File instance, a string or 2-tuple (string,[strings])") d = Data_Files(files=[d],preserve_path=0) d.finalize() return d def run(self): self.outfiles = [] install_cmd = self.get_finalized_command('install') config_vars = install_cmd.config_vars.copy() config_vars.update({ "install_purelib": install_cmd.install_purelib, "install_platlib": install_cmd.install_platlib, "install_lib": install_cmd.install_lib, "install_headers": install_cmd.install_headers, "install_scripts": install_cmd.install_scripts, "install_data": install_cmd.install_data, }) for d in self.data_files: d = self.check_data(d) # copy to an other directory if d.copy_to != None: if not os.path.isabs(d.copy_to): # relative path to install_dir dir = os.path.join(self.install_dir, d.copy_to) elif install_cmd.root: # absolute path and alternative root set dir = change_root(self.root,d.copy_to) else: # absolute path dir = d.copy_to else: # simply copy to install_dir dir = install_dir # warn if necceassary self.warn("setup script did not provide a directory to copy files to " " -- installing right in '%s'" % self.install_dir) # replace variables ($install_lib, ...) in dir dir = os.path.normpath(subst_vars(dir,config_vars)) # create path self.mkpath(dir) # copy all files for src in d.files: if d.strip_dirs > 0: dst = string.join(string.split(os.path.normcase(src),os.sep)[d.strip_dirs:],os.sep) else: dst = src if d.preserve_path: # preserve path in filename self.mkpath(os.path.dirname(os.path.join(dir,dst))) out = self.copy_file(src, os.path.join(dir,dst)) else: out = self.copy_file(src, dir) if type(out) is TupleType: out = out[0] self.outfiles.append(out) return self.outfiles def get_inputs (self): inputs = [] for d in self.data_files: d = self.check_data(d) inputs.append(d.files) return inputs def get_outputs (self): return self.outfiles ###########################################################################