
I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead.
1) Is there a way for me to force it to copy the files into the temporary directory instead of using hard links?
2) Given that it is very hard to predict whether a particular usage of hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy.
/Bo Bai

Bo Nygaard Bai wrote:
I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead.
Is there a way for me to force it to copy the files into the temporary directory instead of using hard links?
Given that it is very hard to predict whether a particular usage of hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy.
How do you test whether a hard link works ? I suppose this would have to be made based on the path since you have mixed setups where some parts of the file system tree allow hard links where others don't.
I don't find hard links all that useful. Disk space is cheap and they often cause problems (e.g. in our case with symlinks in the source tree).
Here's a quick hack:
import os delattr(os, link)
This is more elegant, but requires extra work:
class mx_sdist(sdist):
""" Build a source distribution.
This version does not use hard links when preparing the source archive - hard links don't match well with symlinks which we use in our source repositories.
""" def make_release_tree (self, base_dir, files):
# Prepare release dir self.mkpath(base_dir) self.distribution.metadata.write_pkg_info(base_dir) if not files: log.warn('no files found in release !') return
# Create dir structure log.info('preparing release tree in %s...' % base_dir) create_tree(base_dir, files, dry_run=self.dry_run) for file in files: if not os.path.isfile(file): log.warn('%s is not a regular file -- skipping' % file) else: dest = os.path.join(base_dir, file) self.copy_file(file, dest, link=None)
and then register this command class as replacement for 'sdist'.

M.-A. Lemburg wrote:
Bo Nygaard Bai wrote:
I was trying to use the distutils sdist target on an AFS mounted filesystem on Linux. It fails because it can't make hard links between directories on AFS (you can inside a directory). I have not been able to find any options to make it copy the files instead.
- Is there a way for me to force it to copy the files into the
temporary directory instead of using hard links?
- Given that it is very hard to predict whether a particular usage of
hard links is possible on any given OS/filesystem/mount combination. It seems to me that the best strategy would be to try to make the hard links, and if this fails, silently fall back to making a copy.
How do you test whether a hard link works ? I suppose this would have to be made based on the path since you have mixed setups where some parts of the file system tree allow hard links where others don't.
I do not think it is possible (in the real world) to predict when you can safely use a hard link. My suggestion was that *if* hard links are to be used at all, the only reasonable way to implement it would be to silently catch the exception when creation of a link failed, and then try a copy instead.
I don't find hard links all that useful. Disk space is cheap and they often cause problems (e.g. in our case with symlinks in the source tree).
I agree! In fact I would be quite happy with an implementation that always did a copy, since copying is *much* more robust, and I doubt that anybody has a python module that is big enough for the speed difference to really matter.
[... snip ... ]
Thanks for the plugin!
I must admit that I used the much less elegant solution of copying my entire code base into /tmp to do the build there.
End result: Lots of copying to save a little copying ;-)
Would the maintainers consider not to use hard links?
It seems I am not the only one having problems because of them.
/Bo Bai
participants (2)
-
Bo Nygaard Bai
-
M.-A. Lemburg