[Distutils] extra files into the package dir
Rene Liebscher
R.Liebscher@gmx.de
Wed Nov 8 09:55:02 2000
This is a multi-part message in MIME format.
--------------7004C26B0F2FA02762F9DA02
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Greg Ward wrote:
>
> [Berthold Höllmann mentions...]
> > There is support for installing data files, but they get to data
> > directories. When writing setup.py for biggles, I had the problem,
> > that a config file had to go to the package directory. All I did was adding
> >
> > class my_install_data (install_data):
> > def finalize_options (self):
> > self.set_undefined_options('install',
> > ('install_lib', 'install_dir'),
> > ('root', 'root'),
> > ('force', 'force'),
> > )
> >
> > and
> >
> > cmdclass = {'install_data': my_install_data},
> > data_files = [('biggles', ["src/config.ini"])]
> >
> > to the setup command line. This got everything to the right place.
>
> [Rene Liebscher replies...]
> > This is probably the simplest possible solution. My approach goes
> > further, it handles also MANIFEST.in-like templates and some other
> > special things, and it was written primary to replace distutils'
> > install_data as whole.
> > (Greg, do you remember my mail of 06/26/2000?
> > http://www.python.org/pipermail/distutils-sig/2000-June/001671.html )
>
> As it turns out, the Berthold's "my_install_data" solution is really a
> bare minimum -- the "install_data" command should work that way, or have
> an option to work that way. I wrote a setup script for IDLE that does
> basically the same as Berthold's, and if the Distutils installed their
> own config file, the Distutils setup script would also need this --
> d'oh! I don't think I should have to extend the Distutils in order to
> install the Distutils -- droit de seigneur, in a sense. ;-)
>
> Rene, I originally thought your "DataFiles" concept was overkill, but I
> originally thought having a class to describe extensions was overkill
> too. If nothing else, introducing a simple class may be the only sane
> way to maintain backwards compatibility with the current "install_data
> class. Here's a somewhat simpler conception of the "DataFiles" class...
I changed my my_install_data.py a little bit too.
>
> DataFiles --
> represents a collection of files to be installed in a common
> location, the "base directory".
It is called in my class copy_to, but it is only a name.
>
> Files in the collection are fragmentary paths that are concatenated
> to the base directory at install time to create a complete
> installation path.
This is at my class, the files list parameter.
>
> The base directory may be a literal path, or it may use any of the
> configuration variables allowed in installation directories --
> dist_name
> dist_version
> dist_fullname
> py_version
> py_version_short
> sys_prefix
> prefix
> sys_exec_prefix
> exec_prefix
>
> as well as the config vars corresponding to the installation directory
> options to the "install" command (most of which are just one of the
> entries in an installation scheme):
> install_base
> install_platbase
> install_root
> install_purelib
> install_platlib
> install_headers
> install_scripts
> install_data (?maybe obsolete?)
Can you read my thoughts, I was putting this in my class when I got your
email.
> Thus, a base directory of "$install_base" would expand to
> /usr/lib/python2.0/site-packages on a typical Linux installation,
> or C:\Python on a Windows installation. Or you could use
> "$install_base/mypkg" to install to (eg.) C:\Python\mypkg --
> presumably the directory where modules from your distribution
> get installed. (As usual, Unix paths would be translated to
> native paths by the Distutils, so that setup scripts port across
> operating systems.)
>
> (There probably ought to be a special name for "the directory where
> the highest-level package from this module distribution is installed",
> which would expand to eg. /usr/lib/python1.5/site-packages/distutils.)
>
> This gets rid of dependence on the manifest machinery -- since that's
> now been factored out into a separate class (FileList, in
> distutils.filelist), developers can use it if they want. Or they can
> use the standard 'glob' module if that's good enough, or they can list
> files manually if *that's* good enough.
Supporting an additional template parameter is about ten lines of code.
For PyOpenGL it looks then like this:
-------------------------------------------------------------
data_files = [
Data_Files(
copy_to = '$install_lib/OpenGL/Demo',
strip_dirs = 2,
template=[
# take the whole tree
'graft py/Demo',
# python files are already installed
'global-exclude *.py*',
'global-exclude Cvs/*',
'global-exclude CVS/*'
],
)
],
-------------------------------------------------------------
For an example how it looks if you have to specify the files
even by using glob(), look at the setup file of 4Suite
ftp://ftp.fourthought.com/pub/4Suite/4Suite-0.9.1.tar.gz
(glob doesn't work recursiv through subdirectories.)
To show you what I mean:
the code (about 20% of 4Suite's data files section)
--------------------------------------
('4XSLT/demo',glob.glob('Xslt/demo/*.*')),
('4XSLT',['Xslt/README',
'Xslt/ChangeLog',
'Xslt/TODO',
]),
('4XSLT',glob.glob('Xslt/docs/*.*')),
('4XSLT/test_suite',['Xslt/test_suite/README']),
('4XSLT/test_suite',glob.glob('Xslt/test_suite/*.*')),
('4XSLT/test_suite/borrowed',glob.glob('Xslt/test_suite/borrowed/README')),
('4XSLT/test_suite/borrowed',glob.glob('Xslt/test_suite/borrowed/*.*')),
('4XSLT/test_suite/profile_data',glob.glob('Xslt/test_suite/profile_data/*.*')),
('4XSLT/test_suite/graph_trav',glob.glob('Xslt/test_suite/borrowed/graph_trav/README'))
,
('4XSLT/test_suite/graph_trav',glob.glob('Xslt/test_suite/borrowed/graph_trav/*.*')),
---------------------------------------
could be replaced by something like this
---------------------------------------
Data_Files(
copy_to = '$install_data/4XSLT',
strip_dirs = 1, # this is a number
template=[
# take the whole tree
'graft Xslt/demo',
'graft Xslt/doc',
'graft Xslt/test_suite',
'include Xslt/README',
'include Xslt/ChangeLog',
'include Xslt/TODO',
],
)
---------------------------------------
I think the second is easier.
Kind regards
Rene Liebscher
--------------7004C26B0F2FA02762F9DA02
Content-Type: text/plain; charset=us-ascii;
name="my_install_data.py"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="my_install_data.py"
"""my_install_data.py
Provides a more sophisticated facility to install data files
than distutils' install_data does.
You can specify your files as a template like in MANIFEST.in
and you have more control over the copy process.
"""
# created 2000/08/01, Rene Liebscher <R.Liebscher@gmx.de>
###########################################################################
# import some modules we need
import os,sys,string
from types import StringType,TupleType,ListType
from distutils.util import change_root,subst_vars
from distutils.filelist import FileList
from distutils.command.install_data import install_data
###########################################################################
# a container class for our more sophisticated install mechanism
class Data_Files:
""" container for list of data files.
supports a directory where to copy files
supports variable substitution e.g. '$install_lib','$install_header',...
supports templates as in MANIFEST.in
supports preserving of paths in filenames
eg. foo/xyz is copied to base_dir/foo/xyz
supports stripping of leading dirs of source paths
eg. foo/bar1/xyz, foo/bar2/abc can be copied to bar1/xyz, bar2/abc
"""
def __init__(self,files=None,copy_to=None,template=None,preserve_path=1,strip_dirs=0):
self.files = files
self.copy_to = copy_to
self.template = template
self.preserve_path = preserve_path
self.strip_dirs = strip_dirs
self.finalized = 0
def warn (self, msg):
sys.stderr.write ("warning: %s: %s\n" %
("install_data", msg))
def debug_print (self, msg):
"""Print 'msg' to stdout if the global DEBUG (taken from the
DISTUTILS_DEBUG environment variable) flag is true.
"""
from distutils.core import DEBUG
if DEBUG:
print msg
def finalize(self):
""" complete the files list by processing the given template """
if self.finalized:
return
if self.files == None:
self.files = []
if self.template != None:
if type(self.template) == StringType:
self.template = string.split(self.template,";")
filelist = FileList(self.warn,self.debug_print)
for line in self.template:
filelist.process_template_line(string.strip(line))
filelist.sort()
filelist.remove_duplicates()
self.files.extend(filelist.files)
self.finalized = 1
# end class Data_Files
###########################################################################
# a more sophisticated install routine than distutils install_data
class my_install_data (install_data):
def check_data(self,d):
""" check if data are in new format, if not create a suitable object.
returns finalized data object
"""
if not isinstance(d, Data_Files):
self.warn(("old-style data files list found "
"-- please convert to Data_Files instance"))
if type(d) is TupleType:
if len(d) != 2 or not (type(d[1]) is ListType):
raise DistutilsSetupError, \
("each element of 'data_files' option must be an "
"Data File instance, a string or 2-tuple (string,[strings])")
d = Data_Files(copy_to=d[0],files=d[1])
else:
if not (type(d) is StringType):
raise DistutilsSetupError, \
("each element of 'data_files' option must be an "
"Data File instance, a string or 2-tuple (string,[strings])")
d = Data_Files(files=[d],preserve_path=0)
d.finalize()
return d
def run(self):
self.outfiles = []
install_cmd = self.get_finalized_command('install')
config_vars = install_cmd.config_vars.copy()
config_vars.update({
"install_purelib": install_cmd.install_purelib,
"install_platlib": install_cmd.install_platlib,
"install_lib": install_cmd.install_lib,
"install_headers": install_cmd.install_headers,
"install_scripts": install_cmd.install_scripts,
"install_data": install_cmd.install_data,
})
for d in self.data_files:
d = self.check_data(d)
# copy to an other directory
if d.copy_to != None:
if not os.path.isabs(d.copy_to):
# relative path to install_dir
dir = os.path.join(self.install_dir, d.copy_to)
elif install_cmd.root:
# absolute path and alternative root set
dir = change_root(self.root,d.copy_to)
else:
# absolute path
dir = d.copy_to
else:
# simply copy to install_dir
dir = install_dir
# warn if necceassary
self.warn("setup script did not provide a directory to copy files to "
" -- installing right in '%s'" % self.install_dir)
# replace variables ($install_lib, ...) in dir
dir = os.path.normpath(subst_vars(dir,config_vars))
# create path
self.mkpath(dir)
# copy all files
for src in d.files:
if d.strip_dirs > 0:
dst = string.join(string.split(os.path.normcase(src),os.sep)[d.strip_dirs:],os.sep)
else:
dst = src
if d.preserve_path:
# preserve path in filename
self.mkpath(os.path.dirname(os.path.join(dir,dst)))
out = self.copy_file(src, os.path.join(dir,dst))
else:
out = self.copy_file(src, dir)
if type(out) is TupleType:
out = out[0]
self.outfiles.append(out)
return self.outfiles
def get_inputs (self):
inputs = []
for d in self.data_files:
d = self.check_data(d)
inputs.append(d.files)
return inputs
def get_outputs (self):
return self.outfiles
###########################################################################
--------------7004C26B0F2FA02762F9DA02--