Best practices on distributing documentation and examples with eggs?
Over on the enthought-dev mailing list we're having a bit of a discussion on what the best way to distribute documenation and examples is for projects that we distribute binary eggs for. The general consensus is that it would be very nice indeed if there was a way to generate a tarball, or platform install, of just documentation and examples so that people wouldn't need to download a full, presumably significantly larger, source tarball. Another option would be that eggs included the documentation and examples and that, during the installation of the egg, those docs and examples were relocated to a common location (outside of the zip) to make access by users more convenient. This latter idea is similar to how Ruby Gems deal with docs. I don't claim to be a distutils or setuptools guru, so it shouldn't be too surprising that I can't seem to find anything about a setuptools or distutils command to do either of these. Am I missing something? If not, does it seem like something that might be worthy of putting into setuptools? -- Dave
At 03:02 AM 7/26/2007 -0500, Dave Peterson wrote:
Over on the enthought-dev mailing list we're having a bit of a discussion on what the best way to distribute documenation and examples is for projects that we distribute binary eggs for. The general consensus is that it would be very nice indeed if there was a way to generate a tarball, or platform install, of just documentation and examples so that people wouldn't need to download a full, presumably significantly larger, source tarball. Another option would be that eggs included the documentation and examples and that, during the installation of the egg, those docs and examples were relocated to a common location (outside of the zip) to make access by users more convenient. This latter idea is similar to how Ruby Gems deal with docs.
I don't claim to be a distutils or setuptools guru, so it shouldn't be too surprising that I can't seem to find anything about a setuptools or distutils command to do either of these. Am I missing something?
There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of "standard" tools in the future.
If not, does it seem like something that might be worthy of putting into setuptools?
It might be worth establishing convention(s) for, yes. Tools could then be developed around them, including perhaps changes to easy_install.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 26, 2007, at 11:52 AM, Phillip J. Eby wrote:
There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of "standard" tools in the future.
I always thought .egg-info was just an artifact of the build. Recommending that things get put there means it should get version controlled. But I've found that occasionally I have to blow away .egg-info files to get 'setup.py develop' to work properly (i.e. like a 'make clean' step). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRqjuInEjvBPtnXfVAQJLrAP/S/q2EFM7gnokBTsdgoEjXXn1LXtEWwkP OJaPu4a0jymVy7+R0o9eGXSorOaAO0vN/ay7LGXvkllWTv0pOLa6aQ01cQlEuuhk GLdw+eLVOKmTWnhv4448YhgD/r74ywBWJ9rYD+Ygg93bNHbyTZU/zaoCc2SLpzYq CrVpeK4ArBM= =KfaK -----END PGP SIGNATURE-----
At 02:55 PM 7/26/2007 -0400, Barry Warsaw wrote:
On Jul 26, 2007, at 11:52 AM, Phillip J. Eby wrote:
There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of "standard" tools in the future.
I always thought .egg-info was just an artifact of the build.
No - it's also a place for metadata that doesn't live under a specific package directory.
Recommending that things get put there means it should get version controlled.
Yep.
But I've found that occasionally I have to blow away .egg-info files to get 'setup.py develop' to work properly (i.e. like a 'make clean' step).
Really? I'd consider that to be a bug. Could you perhaps be more specific? Note that if you are using a revision control system, and setuptools deletes a file from there, it might get restored by your revision control, and that could potentially be a problem unless you also manually delete the file.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 26, 2007, at 3:44 PM, Phillip J. Eby wrote:
At 02:55 PM 7/26/2007 -0400, Barry Warsaw wrote:
On Jul 26, 2007, at 11:52 AM, Phillip J. Eby wrote:
There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of "standard" tools in the future.
I always thought .egg-info was just an artifact of the build.
No - it's also a place for metadata that doesn't live under a specific package directory.
Recommending that things get put there means it should get version controlled.
Yep.
But I've found that occasionally I have to blow away .egg-info files to get 'setup.py develop' to work properly (i.e. like a 'make clean' step).
Really? I'd consider that to be a bug. Could you perhaps be more specific?
Now that I know that, I'll investigate further next time I see something 'weird'. I'll keep you posted.
Note that if you are using a revision control system, and setuptools deletes a file from there, it might get restored by your revision control, and that could potentially be a problem unless you also manually delete the file.
When does setuptools delete a file there? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRqj/BXEjvBPtnXfVAQICXQP9F4Iv3ew0BufrhfZt1WYUCnqNF08DiXCK xSvX2KrCc3vV6Z67kFNHJsdAZnJzQR8N0x2n82TXo/q0/36Jcpyt8rp7dIdvD93I mDZtyyk2cFxuRW78xOtYw3lpUL75e1YbVMzSyhjwaXMdnKNYTfqRyfHkaR/dBz0H hKbHNuroUWg= =/4VX -----END PGP SIGNATURE-----
At 04:07 PM 7/26/2007 -0400, Barry Warsaw wrote:
On Jul 26, 2007, at 3:44 PM, Phillip J. Eby wrote:
Note that if you are using a revision control system, and setuptools deletes a file from there, it might get restored by your revision control, and that could potentially be a problem unless you also manually delete the file.
When does setuptools delete a file there?
Whenever a setuptools "egg_info.writers" plugin does. :) More usefully, most files generated by setup() arguments (e.g. entry_points) will be deleted the next time you run egg_info, if you removed that argument from setup().
Phillip J. Eby wrote:
At 02:55 PM 7/26/2007 -0400, Barry Warsaw wrote:
On Jul 26, 2007, at 11:52 AM, Phillip J. Eby wrote:
There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of "standard" tools in the future. I always thought .egg-info was just an artifact of the build.
No - it's also a place for metadata that doesn't live under a specific package directory.
Recommending that things get put there means it should get version controlled.
Yep.
But isn't .egg-info completely recreatable through setup.py? Given that all it really does is present information from setup.py in a different manner, I've always been putting it on my svn:ignore list and only version-controlled the canonical source -- setup.py. That's becaue I can see and have seen the information get out of synch easily, e.g. if somebody changes setup.py but forgets to run setup.py egg_info. -- http://worldcookery.com -- Professional Zope documentation and training
At 08:48 AM 7/31/2007 +0200, Philipp von Weitershausen wrote:
But isn't .egg-info completely recreatable through setup.py?
Not if you're putting other data there, no. Chandler, for example, has an i18n framework that stores message catalogs and other localized resources there. Of course, it's certainly possible to use egg_info.writers plugins that would copy those things there, but it's a lot easier for development if you don't, as it would require a build step every time you changed something.
On Jul 31, 2007, at 12:31 PM, Phillip J. Eby wrote:
At 08:48 AM 7/31/2007 +0200, Philipp von Weitershausen wrote:
But isn't .egg-info completely recreatable through setup.py?
Not if you're putting other data there, no. Chandler, for example, has an i18n framework that stores message catalogs and other localized resources there.
Um. Why? That isn't egg meta data is it? Couldn't that just go in good old package data? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
At 03:37 PM 8/1/2007 -0400, Jim Fulton wrote:
On Jul 31, 2007, at 12:31 PM, Phillip J. Eby wrote:
At 08:48 AM 7/31/2007 +0200, Philipp von Weitershausen wrote:
But isn't .egg-info completely recreatable through setup.py?
Not if you're putting other data there, no. Chandler, for example, has an i18n framework that stores message catalogs and other localized resources there.
Um. Why?
That isn't egg meta data is it?
Egg metadata is data that belongs to an egg as a whole, rather than to any specific package within the egg.
Couldn't that just go in good old package data?
Not if the egg doesn't contain any packages, and it's possible to ship a set of localizations as an egg, without it containing any actual code.
participants (5)
-
Barry Warsaw -
Dave Peterson -
Jim Fulton -
Philipp von Weitershausen -
Phillip J. Eby