I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead. 1) Is there a way for me to force it to copy the files into the temporary directory instead of using hard links? 2) Given that it is very hard to predict whether a particular usage of hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy. /Bo Bai
Bo Nygaard Bai wrote:
I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead.
1) Is there a way for me to force it to copy the files into the temporary directory instead of using hard links?
2) Given that it is very hard to predict whether a particular usage of hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy.
How do you test whether a hard link works ? I suppose this would have to be made based on the path since you have mixed setups where some parts of the file system tree allow hard links where others don't. I don't find hard links all that useful. Disk space is cheap and they often cause problems (e.g. in our case with symlinks in the source tree). Here's a quick hack: import os delattr(os, link) This is more elegant, but requires extra work: class mx_sdist(sdist): """ Build a source distribution. This version does not use hard links when preparing the source archive - hard links don't match well with symlinks which we use in our source repositories. """ def make_release_tree (self, base_dir, files): # Prepare release dir self.mkpath(base_dir) self.distribution.metadata.write_pkg_info(base_dir) if not files: log.warn('no files found in release !') return # Create dir structure log.info('preparing release tree in %s...' % base_dir) create_tree(base_dir, files, dry_run=self.dry_run) for file in files: if not os.path.isfile(file): log.warn('%s is not a regular file -- skipping' % file) else: dest = os.path.join(base_dir, file) self.copy_file(file, dest, link=None) and then register this command class as replacement for 'sdist'. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
M.-A. Lemburg wrote:
Bo Nygaard Bai wrote:
I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead.
1) Is there a way for me to force it to copy the files into the temporary directory instead of using hard links?
2) Given that it is very hard to predict whether a particular usage of hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy.
How do you test whether a hard link works ? I suppose this would have to be made based on the path since you have mixed setups where some parts of the file system tree allow hard links where others don't.
I do not think it is possible (in the real world) to predict when you can safely use a hard link. My suggestion was that *if* hard links are to be used at all, the only reasonable way to implement it would be to silently catch the exception when creation of a link failed, and then try a copy instead.
I don't find hard links all that useful. Disk space is cheap and they often cause problems (e.g. in our case with symlinks in the source tree).
I agree! In fact I would be quite happy with an implementation that always did a copy, since copying is *much* more robust, and I doubt that anybody has a python module that is big enough for the speed difference to really matter.
[... snip ... ]
Thanks for the plugin! I must admit that I used the much less elegant solution of copying my entire code base into /tmp to do the build there. End result: Lots of copying to save a little copying ;-) Would the maintainers consider not to use hard links? It seems I am not the only one having problems because of them. /Bo Bai
participants (2)
-
Bo Nygaard Bai -
M.-A. Lemburg