I'm trying to figure out how I should deal with the .egg-info directory, and PKG-INFO. Right now PKG-INFO is being generated from setup.py. So I don't want to put it in my source control repository, since it is derivative. But I'm maintaining by hand some other files in the .egg-info directory, usually custom package metadata that we consume internally, so I need those files (and .egg-info) in the repository, and hence in the checkout. However, without PKG-INFO lots of things break, as pkg_resources gives a ValueError during some operations (e.g.: ValueError: ("Missing 'Version:' header and/or PKG-INFO file", ProjectName [unknown version] (/usr/home/ianb/co/IscapeLib/tests/appsetup/output/Proj-05))) -- this happens anytime I iterate over entry points, including when setuptools iterates over entry points, which means lots of setup.py commands are broken. They are broken simply because the incomplete .egg-info directory is on the path, even though I don't need direct access to it. I can run setup.py egg_info (creating PKG-INFO) for some reason (that I have not yet determined), but not my own custom setup.py extensions. This includes stuff like "python setup.py egg_info my_custom_command", so I have to write PKG-INFO in a completely separate command. If the .egg-info directory doesn't exist then pkg_resources will be happy, because it won't try to read the package as an egg at all. But it does, and so it's just awkward. Frankly I'd be 100% okay putting PKG-INFO in source control if it was the authoritative source, and I don't particularly care to use setup.py for this data. But that's a cop-out, I should really figure this out as it is. So... what do I do here? Should pkg_resources really just ignore packages without a PKG-INFO file? At least during scan? If I have to live with running setup.py egg_info until 0.6a7, that's fine. BTW, here's the full traceback I get: Traceback (most recent call last): File "setup.py", line 18, in ? package_data={ File "/usr/local/lib/python2.4/distutils/core.py", line 135, in setup ok = dist.parse_command_line() File "/usr/home/ianb/co/setuptools/setuptools/dist.py", line 227, in parse_command_line result = _Distribution.parse_command_line(self) File "/usr/local/lib/python2.4/distutils/dist.py", line 432, in parse_command_line args = self._parse_command_opts(parser, args) File "/usr/home/ianb/co/setuptools/setuptools/dist.py", line 549, in _parse_command_opts nargs = _Distribution._parse_command_opts(self, parser, args) File "/usr/local/lib/python2.4/distutils/dist.py", line 490, in _parse_command_opts cmd_class = self.get_command_class(command) File "/usr/home/ianb/co/setuptools/setuptools/dist.py", line 353, in get_command_class ep.require(installer=self.fetch_build_egg) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1637, in require working_set.resolve(self.dist.requires(self.extras),env,installer)) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 480, in resolve env = Environment(self.entries) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 558, in __init__ self.scan(search_path) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 588, in scan self.add(dist) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 610, in add if dist not in dists: File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1760, in __cmp__ def __cmp__(self, other): return cmp(self.hashcmp, other) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1760, in __cmp__ def __cmp__(self, other): return cmp(self.hashcmp, other) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1755, in <lambda> lambda self: ( File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1782, in parsed_version self._parsed_version = pv = parse_version(self.version) File "/usr/home/ianb/co/setuptools/pkg_resources.py", line 1797, in version raise ValueError( ValueError: ("Missing 'Version:' header and/or PKG-INFO file", ProjectName [unknown version] (/usr/home/ianb/co/IscapeLib/tests/appsetup/output/Proj-05)) -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 04:30 PM 10/25/2005 -0500, Ian Bicking wrote:
I'm trying to figure out how I should deal with the .egg-info directory, and PKG-INFO.
Right now PKG-INFO is being generated from setup.py. So I don't want to put it in my source control repository, since it is derivative. But I'm maintaining by hand some other files in the .egg-info directory, usually custom package metadata that we consume internally, so I need those files (and .egg-info) in the repository, and hence in the checkout.
However, without PKG-INFO lots of things break, as pkg_resources gives a ValueError during some operations (e.g.: ValueError: ("Missing 'Version:' header and/or PKG-INFO file", ProjectName [unknown version] (/usr/home/ianb/co/IscapeLib/tests/appsetup/output/Proj-05))) -- this happens anytime I iterate over entry points, including when setuptools iterates over entry points, which means lots of setup.py commands are broken. They are broken simply because the incomplete .egg-info directory is on the path, even though I don't need direct access to it. I can run setup.py egg_info (creating PKG-INFO) for some reason (that I have not yet determined), but not my own custom setup.py extensions. This includes stuff like "python setup.py egg_info my_custom_command", so I have to write PKG-INFO in a completely separate command.
'setup.py develop' should create the info and put the directory on the path for you. Likewise, so does 'setup.py test'. Which other commands are giving you problems?
So... what do I do here? Should pkg_resources really just ignore packages without a PKG-INFO file? At least during scan? If I have to live with running setup.py egg_info until 0.6a7, that's fine.
How would pkg_resources then determine what version your package is, if it's under development? It has to know the version in order to be able to detect duplicates during scanning, and also to order available packages by version number.
Sorry, I somehow missed this response... Phillip J. Eby wrote:
At 04:30 PM 10/25/2005 -0500, Ian Bicking wrote:
I'm trying to figure out how I should deal with the .egg-info directory, and PKG-INFO.
Right now PKG-INFO is being generated from setup.py. So I don't want to put it in my source control repository, since it is derivative. But I'm maintaining by hand some other files in the .egg-info directory, usually custom package metadata that we consume internally, so I need those files (and .egg-info) in the repository, and hence in the checkout.
However, without PKG-INFO lots of things break, as pkg_resources gives a ValueError during some operations (e.g.: ValueError: ("Missing 'Version:' header and/or PKG-INFO file", ProjectName [unknown version] (/usr/home/ianb/co/IscapeLib/tests/appsetup/output/Proj-05))) -- this happens anytime I iterate over entry points, including when setuptools iterates over entry points, which means lots of setup.py commands are broken. They are broken simply because the incomplete .egg-info directory is on the path, even though I don't need direct access to it. I can run setup.py egg_info (creating PKG-INFO) for some reason (that I have not yet determined), but not my own custom setup.py extensions. This includes stuff like "python setup.py egg_info my_custom_command", so I have to write PKG-INFO in a completely separate command.
'setup.py develop' should create the info and put the directory on the path for you. Likewise, so does 'setup.py test'. Which other commands are giving you problems?
Well, like I said, any custom command. These just inherit from setuptools.Command, implementing the standard initialize_options/finalize_options/run methods. I even tried having my custom command recursively call "egg_info" (with get_finalized_command), but because the error is signaled before I get as far as .run() (where I put the egg_info call) it doesn't help. Maybe I could use another method that is invoked earlier. Though now I can't reproduce it; maybe this was fixed in 0.6a7? -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 01:09 PM 11/4/2005 -0600, Ian Bicking wrote:
Well, like I said, any custom command. These just inherit from setuptools.Command, implementing the standard initialize_options/finalize_options/run methods. I even tried having my custom command recursively call "egg_info" (with get_finalized_command), but because the error is signaled before I get as far as .run() (where I put the egg_info call) it doesn't help. Maybe I could use another method that is invoked earlier.
Though now I can't reproduce it; maybe this was fixed in 0.6a7?
Dunno. As I mentioned on the Turbogears list, the problem isn't going to happen if your source code is in a subdirectory of the one containing setup.py. Perhaps you've changed layouts? In any case, as I mentioned on the Turbogears list, I'm going to try to put in some code to detect whether there is a "broken egg" in the working set and monkeypatch its 'version' attribute to work around this bootstrapping problem. That should make it more robust and eliminate the need to include any of the .egg-info files in Subversion, at least for most normal projects. (Setuptools is an oddity in that it defines entry points that it then uses while building itself! Its .egg-info/entry_points.txt file is therefore not optional. But the need for this sort of self-bootstrapping setup process should be extremely rare.)
Phillip J. Eby wrote:
At 01:09 PM 11/4/2005 -0600, Ian Bicking wrote:
Well, like I said, any custom command. These just inherit from setuptools.Command, implementing the standard initialize_options/finalize_options/run methods. I even tried having my custom command recursively call "egg_info" (with get_finalized_command), but because the error is signaled before I get as far as .run() (where I put the egg_info call) it doesn't help. Maybe I could use another method that is invoked earlier.
Though now I can't reproduce it; maybe this was fixed in 0.6a7?
Dunno. As I mentioned on the Turbogears list, the problem isn't going to happen if your source code is in a subdirectory of the one containing setup.py. Perhaps you've changed layouts?
In any case, as I mentioned on the Turbogears list, I'm going to try to put in some code to detect whether there is a "broken egg" in the working set and monkeypatch its 'version' attribute to work around this bootstrapping problem. That should make it more robust and eliminate the need to include any of the .egg-info files in Subversion, at least for most normal projects. (Setuptools is an oddity in that it defines entry points that it then uses while building itself! Its .egg-info/entry_points.txt file is therefore not optional. But the need for this sort of self-bootstrapping setup process should be extremely rare.)
FYI, I'm already putting my own custom files in the .egg-info/ directory. And sqlobject-admin now looks for a sqlobject.txt file in the egg metadata, which it uses to find the SQLObject subclasses in the project (and a couple other things). These could all possibly be moved to entry points, but treating them as simple configuration files has so far seemed easier to manage. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 02:19 PM 11/4/2005 -0600, Ian Bicking wrote:
FYI, I'm already putting my own custom files in the .egg-info/ directory. And sqlobject-admin now looks for a sqlobject.txt file in the egg metadata, which it uses to find the SQLObject subclasses in the project (and a couple other things). These could all possibly be moved to entry points, but treating them as simple configuration files has so far seemed easier to manage.
Not everything has to be an entry point. But if it's configuration that's easier to express in Python code, or if you prefer to put everything in the setup script (as I usually do), then you can define an "egg_info writer" entry point and a "setup keyword" entry point in a parent project (e.g. SQLObject itself), and then projects using SQLObject can do setup_requires="SQLObject" and then define the other metadata using keyword arguments. Most of the existing .egg-info files are written in exactly this way, including install_requires and entry_points.
participants (2)
-
Ian Bicking
-
Phillip J. Eby