Including symlinked data files in an egg
I have a package with a source that looks like this: setup.py MANIFEST.in Prototype/libraries/jquery.js Prototype/style/main.css my/ my/package/ my/package/templates/libraries -> ../../../Prototype/libraries my/package/templates/style -> ../../../Prototype/style what I would like to accomplish is that when a bdist is made either Prototype/ is installed somewhere and the symlinks point to it, or the symlinks are replaced with a copy of the data they point to. For a package maintained in subverion this works: when the sdist is created the symlink is replaced with a copy of the thing it points to, which is then included in the bdist. For a package maintained in git this does not work: the symlink always appears to be ignored. Is there a way to accomplish this? Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
At 10:44 PM 12/14/2010 +0100, Wichert Akkerman wrote:
I have a package with a source that looks like this:
setup.py MANIFEST.in Prototype/libraries/jquery.js Prototype/style/main.css my/ my/package/ my/package/templates/libraries -> ../../../Prototype/libraries my/package/templates/style -> ../../../Prototype/style
what I would like to accomplish is that when a bdist is made either Prototype/ is installed somewhere and the symlinks point to it, or the symlinks are replaced with a copy of the data they point to.
For a package maintained in subverion this works: when the sdist is created the symlink is replaced with a copy of the thing it points to, which is then included in the bdist. For a package maintained in git this does not work: the symlink always appears to be ignored. Is there a way to accomplish this?
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
Wichert.
-- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On 12/15/10 16:15 , P.J. Eby wrote:
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
I am not using a git (or other) plugin. I tried listing things in MANIFEST.in, for example using: graft my/package/templates/ graft my/package/templates/libraries unfortunately that had no effect: libraries was not included, not as symlink nor as a copy of the symlink target. Wichert.
At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
On 12/15/10 16:15 , P.J. Eby wrote:
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
I am not using a git (or other) plugin. I tried listing things in MANIFEST.in, for example using:
graft my/package/templates/ graft my/package/templates/libraries
unfortunately that had no effect: libraries was not included, not as symlink nor as a copy of the symlink target.
I don't know whether MANIFEST.in is supposed to support symlinks or not, but the fact that it's working with subversion is strongly indicative that setuptools' file finder is picking up stuff that MANIFEST.in is not. I don't know *why* MANIFEST.in wouldn't pick up symlinks, though. Perhaps you could try boiling this down to a simpler test case (e.g., subversion and git versions of the same minimal package containing only a symlink referred to by MANIFEST.in) so we can narrow it down some more?
Wichert.
On 12/16/10 00:52 , P.J. Eby wrote:
At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
On 12/15/10 16:15 , P.J. Eby wrote:
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
I am not using a git (or other) plugin. I tried listing things in MANIFEST.in, for example using:
graft my/package/templates/ graft my/package/templates/libraries
unfortunately that had no effect: libraries was not included, not as symlink nor as a copy of the symlink target.
I don't know whether MANIFEST.in is supposed to support symlinks or not, but the fact that it's working with subversion is strongly indicative that setuptools' file finder is picking up stuff that MANIFEST.in is not.
I don't know *why* MANIFEST.in wouldn't pick up symlinks, though. Perhaps you could try boiling this down to a simpler test case (e.g., subversion and git versions of the same minimal package containing only a symlink referred to by MANIFEST.in) so we can narrow it down some more?
I've finally gotten around to doing this. I have attached test.tar.gz with a minimal package that demonstrates this. Wichert.
At 10:56 AM 4/20/2011 +0200, Wichert Akkerman wrote:
On 12/16/10 00:52 , P.J. Eby wrote:
At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
On 12/15/10 16:15 , P.J. Eby wrote:
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
I am not using a git (or other) plugin. I tried listing things in MANIFEST.in, for example using:
graft my/package/templates/ graft my/package/templates/libraries
unfortunately that had no effect: libraries was not included, not as symlink nor as a copy of the symlink target.
I don't know whether MANIFEST.in is supposed to support symlinks or not, but the fact that it's working with subversion is strongly indicative that setuptools' file finder is picking up stuff that MANIFEST.in is not.
I don't know *why* MANIFEST.in wouldn't pick up symlinks, though. Perhaps you could try boiling this down to a simpler test case (e.g., subversion and git versions of the same minimal package containing only a symlink referred to by MANIFEST.in) so we can narrow it down some more?
I've finally gotten around to doing this. I have attached test.tar.gz with a minimal package that demonstrates this.
Ah. It appears MANIFEST.in simply doesn't support symlinks. (Distutils has explicit code to filter them out, in fact.) This is a distutils issue, which setuptools can't do anything about.
On 4/20/11 18:04 , P.J. Eby wrote:
At 10:56 AM 4/20/2011 +0200, Wichert Akkerman wrote:
On 12/16/10 00:52 , P.J. Eby wrote:
At 04:18 PM 12/15/2010 +0100, Wichert Akkerman wrote:
On 12/15/10 16:15 , P.J. Eby wrote:
Most likely, the problem is with the file finder being used. Are you using a git plugin for setuptools, or simply listing everything in MANIFEST.in? If the former, it may be that the git plugin you're using simply doesn't support symlinks. If you're not using a git plugin, perhaps there is something amiss in the MANIFEST.in, and setuptools' built-in support for subversion is hiding the problem by picking up the symlinks.
I am not using a git (or other) plugin. I tried listing things in MANIFEST.in, for example using:
graft my/package/templates/ graft my/package/templates/libraries
unfortunately that had no effect: libraries was not included, not as symlink nor as a copy of the symlink target.
I don't know whether MANIFEST.in is supposed to support symlinks or not, but the fact that it's working with subversion is strongly indicative that setuptools' file finder is picking up stuff that MANIFEST.in is not.
I don't know *why* MANIFEST.in wouldn't pick up symlinks, though. Perhaps you could try boiling this down to a simpler test case (e.g., subversion and git versions of the same minimal package containing only a symlink referred to by MANIFEST.in) so we can narrow it down some more?
I've finally gotten around to doing this. I have attached test.tar.gz with a minimal package that demonstrates this.
Ah. It appears MANIFEST.in simply doesn't support symlinks. (Distutils has explicit code to filter them out, in fact.) This is a distutils issue, which setuptools can't do anything about.
Is there another way to get this working? I'ld hate to have to resort to monkeypatching distutils. Wichert.
At 08:12 PM 4/20/2011 +0200, Wichert Akkerman wrote:
Is there another way to get this working? I'ld hate to have to resort to monkeypatching distutils.
Well, you could have the symlinks go the other way around, I suppose. ;-) Alternately, you could write a "file finder" plugin for setuptools, or see whether one of the git plugins for setuptools fixes the problem. (Note: the plugin would only be needed when building from a direct git checkout - building from an "sdist" (source .tar.gz or .zip) will not need it.)
On 4/20/11 23:38 , P.J. Eby wrote:
At 08:12 PM 4/20/2011 +0200, Wichert Akkerman wrote:
Is there another way to get this working? I'ld hate to have to resort to monkeypatching distutils.
Well, you could have the symlinks go the other way around, I suppose. ;-)
In theory I could, but I doubt the designers who work with this structure will be able to deal with symlinks.
Alternately, you could write a "file finder" plugin for setuptools, or see whether one of the git plugins for setuptools fixes the problem.
setuptools-git did not support this, but luckily that was easily fixed (see https://github.com/wichert/setuptools-git/commit/8528d58975c595b64eea9e7e672... ). Wichert.
Hi, Le 20/04/2011 10:56, Wichert Akkerman a écrit :
I've finally gotten around to doing this. I have attached test.tar.gz with a minimal package that demonstrates this.
Distutils is frozen except for bug fixes (i.e. discrepancies between the documentation and the behavior), but distutils2 is open to feature requests. Can you open such a request on bugs.python.org, with a rationale (use cases)? (Note that if you include files such as your test, one or more files or diffs is preferred to binary archives.) Thanks in advance. Regards
participants (3)
-
P.J. Eby -
Wichert Akkerman -
Éric Araujo