How to get list of package requirements from PyPI without downloading egg?
Hello, I want the list of package requirements, which I hoped would be real simple, but have not been able to find a good solution. I could download the egg and look in the EGG_INFO directory, but is there any way to have setuptools develop the list for me? And hopefully not have to download the egg. If the above idea doesn't go anywhere - is it possible that the package requirements could be added to CheeseShop's XMLRPC? Kindest regards, Tim Cera
At 12:39 AM 10/26/2006 -0400, Tim Cera wrote:
Hello,
I want the list of package requirements, which I hoped would be real simple, but have not been able to find a good solution.
I could download the egg and look in the EGG_INFO directory, but is there any way to have setuptools develop the list for me?
Use the pkg_resources module, specifically the requires() method of Distribution objects: http://peak.telecommunity.com/DevCenter/PkgResources#distribution-methods Here are some ways to obtain Distribution objects: http://peak.telecommunity.com/DevCenter/PkgResources#getting-or-creating-dis...
And hopefully not have to download the egg.
No, sorry, you have to have the egg to extract its requirements.
If the above idea doesn't go anywhere - is it possible that the package requirements could be added to CheeseShop's XMLRPC?
Requirements can be Python version and platform-specific, so I'm afraid this can't be reasonably captured in a Cheeseshop entry at the present time.
On Wednesday 25 October 2006 11:51 pm, Phillip J. Eby wrote:
At 12:39 AM 10/26/2006 -0400, Tim Cera wrote:
If the above idea doesn't go anywhere - is it possible that the package requirements could be added to CheeseShop's XMLRPC?
Requirements can be Python version and platform-specific, so I'm afraid this can't be reasonably captured in a Cheeseshop entry at the present time.
Except Cheeseshop already supports the PEP 314 metadeta fields Requires, Provides and Obsoletes. They are available to all packages built with Python 2.5 distutils. That said, the community still needs to agree on what the names of the requirements are. The Cheeseshop restrictions match package names and that is what 4Suite's PackageManager uses for its lookup routines. It solves the Python version problem by defining the PEP 345 metadata field Requires-Python. -- Jeremy Kloth http://4suite.org/
At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote:
On Wednesday 25 October 2006 11:51 pm, Phillip J. Eby wrote:
At 12:39 AM 10/26/2006 -0400, Tim Cera wrote:
If the above idea doesn't go anywhere - is it possible that the package requirements could be added to CheeseShop's XMLRPC?
Requirements can be Python version and platform-specific, so I'm afraid this can't be reasonably captured in a Cheeseshop entry at the present time.
Except Cheeseshop already supports the PEP 314 metadeta fields Requires, Provides and Obsoletes. They are available to all packages built with Python 2.5 distutils.
Indeed it does... but that information is both unused and useless at the moment.
That said, the community still needs to agree on what the names of the requirements are.
Yep - like I said, it's both unused and useless. :)
It solves the Python version problem by defining the PEP 345 metadata field Requires-Python.
No, it doesn't, actually. My statement was that the requirements *themselves* can be version and platform specific. PEP 314 only allows for one set of requirements for a package, not one set of requirements per version and platform. In any case, saying that we have a solution and that we just have to decide what it means, reminds me of a passage from the Douglas Adams novel, "Dirk Gently's Holistic Detective Agency", wherein Dirk makes a cryptic scrawl on a piece of paper and declares that it is the solution to the mystery! They need only determine what language he wrote it in, and translate it back to English... :) Personally, I think PEP 314 should be rejected, as this issue is only one of its flaws. I'm surprised to see it is now marked "final", given the near total lack of discussion on it, except for me periodically posting about its flaws (since 2005, no less). None of the new fields except Download-URL are actually useful. PEP 314 delenda est.
On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote:
At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote:
Except Cheeseshop already supports the PEP 314 metadeta fields Requires, Provides and Obsoletes. They are available to all packages built with Python 2.5 distutils.
Indeed it does... but that information is both unused and useless at the moment.
Could it be that it is unused because Pyhon 2.5 was just released? That is when these fields made it into Distutils.
That said, the community still needs to agree on what the names of the requirements are.
Yep - like I said, it's both unused and useless. :)
It solves the Python version problem by defining the PEP 345 metadata field Requires-Python.
No, it doesn't, actually. My statement was that the requirements *themselves* can be version and platform specific. PEP 314 only allows for one set of requirements for a package, not one set of requirements per version and platform.
So how do you specify an exact platform and Python version with setuptools' install_requires? They are, IMO, used at the same level.
Personally, I think PEP 314 should be rejected, as this issue is only one of its flaws. I'm surprised to see it is now marked "final", given the near total lack of discussion on it, except for me periodically posting about its flaws (since 2005, no less). None of the new fields except Download-URL are actually useful.
It is a little late now. Those fields are part of Python 2.5 so they are going to be around for at least 3 more years (if the 2.3 to 2.5 timeline is any indicator). So how about doing something to actually *improve* the fields besides just complaining? Going forward with (probable) modifications to the fields' allowed values, it should be fairly easy to expand the allowed values as the need arises since the current values are so restrictive.
PEP 314 delenda est.
Now there is a postitive attitude... -- Jeremy Kloth http://4suite.org/
At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote:
At 12:05 AM 10/26/2006 -0600, Jeremy Kloth wrote:
Except Cheeseshop already supports the PEP 314 metadeta fields Requires, Provides and Obsoletes. They are available to all packages built with Python 2.5 distutils.
Indeed it does... but that information is both unused and useless at the moment.
Could it be that it is unused because Pyhon 2.5 was just released? That is when these fields made it into Distutils.
That could be why it's unused; but not why it's useless. No agreement on semantics simply means that the odds are astronomically against anyone putting in data that can actually be used by someone else. You would *first* need to create a tool that could do something useful with the information, and second, you'd need it to be something that actually *tested* the usefulness -- meaning you'd need a distutils extension, or else you'd have to wait until 2.6 to add new features to distutils itself. (If the data can't be tested by the person putting it in their setup() script, then it will be wrong, and therefore useless to others for any sort of automated dependency resolution.)
So how do you specify an exact platform and Python version with setuptools' install_requires? They are, IMO, used at the same level.
install_requires is set by a script, so the script can set it based on the Python version and platform the script is running on. Thus, eggs built for different platforms or Python versions can have different requirements.
Personally, I think PEP 314 should be rejected, as this issue is only one of its flaws. I'm surprised to see it is now marked "final", given the near total lack of discussion on it, except for me periodically posting about its flaws (since 2005, no less). None of the new fields except Download-URL are actually useful.
It is a little late now. Those fields are part of Python 2.5 so they are going to be around for at least 3 more years (if the 2.3 to 2.5 timeline is any indicator). So how about doing something to actually *improve* the fields besides just complaining?
If there were a *way* to improve them, I'd have suggested it a long time ago.
Going forward with (probable) modifications to the fields' allowed values, it should be fairly easy to expand the allowed values as the need arises since the current values are so restrictive.
Until somebody has some idea of 1) what the semantics are, and 2) what the use cases for those semantics are, the fields aren't of any value whatsoever. This isn't something that's going to be improved by fiddling with their syntax, so long as the semantics remain undefined.
PEP 314 delenda est.
Now there is a postitive attitude...
Well, I'm pretty positive that it delenda est. :) In truth, however, it's just a lame duck that won't fly, doomed to irrelevance in the bigger scheme of things. It would take a heroic effort to make anything useful out of it, and there's no incentive for anybody to make that effort. So, from my POV, the only useful forward action is to put it out of its misery, since the only other thing that's ever going to happen with it is talk, and maybe a few people putting some useless and incompatible things in there. You need some kind of community that wants to specify dependencies, but *doesn't* want anybody to be able to automatically install them! Or else, that develops its own set of tools for automatic installation... that only works with packages whose users followed that new tool's conventions. In effect, you need a community large enough to create its own mini-PyPI... and that isn't already using setuptools dependencies to do the same thing. Or, you need to have it use the metadata in a style similar to setuptools, in which case you're limited by PEP 314's crazy syntax restrictions, that aren't compatible with existing PyPI distribution names. About the only thing that could possibly outweigh all that is a BDFL proclamation in favor of some particular semantics... which nobody has proposed or agrees upon as yet.
On Thursday 26 October 2006 9:21 am, Phillip J. Eby wrote:
At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote:
No agreement on semantics simply means that the odds are astronomically against anyone putting in data that can actually be used by someone else.
As you like to point out so often, I do not actually recall anyone has ever brought up the semantics of the fields except yourself whom obviously is 110% against them to begin with.
You would *first* need to create a tool that could do something useful with the information,
Done. 4Suite's PackageManager.
and second, you'd need it to be something that actually *tested* the usefulness -- meaning you'd need a distutils extension,
Done. See above.
or else you'd have to wait until 2.6 to add new features to distutils itself.
Hmm, interesting idea. It so happens that I've been using/extending distutils since its introduction and am *very* comfortable with its code base. I believe that basically an offer to take maintenanceship of it was extended on python-dev, this may be something that I will look into. There wouldn't need to be wait if distutils itself starting making separate releases again.
install_requires is set by a script, so the script can set it based on the Python version and platform the script is running on. Thus, eggs built for different platforms or Python versions can have different requirements.
And you're telling me that the `requires` argument isn't set by a script? A platform/version specific dependency issue can only come into play if a particular distribution is begin scanned for dependencies out-of-band, which is what the OP was talking about, but that is not what is at issue here.
Until somebody has some idea of 1) what the semantics are, and 2) what the use cases for those semantics are, the fields aren't of any value whatsoever. This isn't something that's going to be improved by fiddling with their syntax, so long as the semantics remain undefined.
Aparently you haven't used a recent Linux distribution? Both apt or rpm have the semantics clearly defined and that is the first thing that came to my mind when I came across those fields. Are you sure that you are not just against things that are not setuptools? -- Jeremy Kloth http://4suite.org/
At 10:20 AM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 9:21 am, Phillip J. Eby wrote:
At 12:53 AM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 12:21 am, Phillip J. Eby wrote:
No agreement on semantics simply means that the odds are astronomically against anyone putting in data that can actually be used by someone else.
As you like to point out so often, I do not actually recall anyone has ever brought up the semantics of the fields except yourself whom obviously is 110% against them to begin with.
You would *first* need to create a tool that could do something useful with the information,
Done. 4Suite's PackageManager.
and second, you'd need it to be something that actually *tested* the usefulness -- meaning you'd need a distutils extension,
Done. See above.
Sounds great, then. What's the problem? (Btw, do you have a documentation link? Google doesn't turn up anything except API docs that say nothing about the PEP 314 stuff.)
install_requires is set by a script, so the script can set it based on the Python version and platform the script is running on. Thus, eggs built for different platforms or Python versions can have different requirements.
And you're telling me that the `requires` argument isn't set by a script?
No, I'm saying that the Cheeseshop only holds one set of "requires" values.
A platform/version specific dependency issue can only come into play if a particular distribution is begin scanned for dependencies out-of-band, which is what the OP was talking about, but that is not what is at issue here.
I was assuming we were talking about the OP's request, so that explains the disconnect.
Until somebody has some idea of 1) what the semantics are, and 2) what the use cases for those semantics are, the fields aren't of any value whatsoever. This isn't something that's going to be improved by fiddling with their syntax, so long as the semantics remain undefined.
Aparently you haven't used a recent Linux distribution? Both apt or rpm have the semantics clearly defined and that is the first thing that came to my mind when I came across those fields.
Well, the PEP explicitly says there is *no* semantics defined, so I chose to believe the PEP author. :) (Yes, I'm familiar with RPM-style requirements.)
Are you sure that you are not just against things that are not setuptools?
Quite sure. PEP 314 was the first place I looked for an opportunity to implement setuptools' requirements. However, I found it to be inadequate for my goals, which included backward-compatibility with existing distutils-based packages, and support for simple, statically-published package indexes and links. In essence, the "provides" portion of the PEP makes it impossible (AFAICT) to provide these features. In my mind, supporting discovery of non-setuptools packages and allowing distributed package publishing was more important than mimicking RPM. That's why I'm saying that getting PEP 314-based stuff out into the wild requires either ignoring PEP 314's syntax restrictions (and the provides/obsoletes fields along with them), or a break with both backward compatibility and distributed publishing. So, my opinion isn't really a matter of objecting to a "competing" system (which would be silly). It's more a prediction that PEP 314 isn't really going to go anywhere, because it imposes the cost of adoption on the wrong people, without providing them any countervailing benefits. So it doesn't take any great leap of intuition to see that it will go nowhere unless something changes. That having been said, if I'm wrong and it *does* succeed in going somewhere (e.g. the BDFL blesses some set of semantics), then I'll certainly look at what needs to be done to add support for it to setuptools. But at this point, it just looks to me like one of the perennial "we should do something about X" ideas that doesn't go anywhere because there isn't enough critical mass for adoption or anybody who knows how to solve the technical problems.
On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote:
(Btw, do you have a documentation link? Google doesn't turn up anything except API docs that say nothing about the PEP 314 stuff.)
http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Dist or (as always, "Read the source, Luke"): http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup The entire body of Ft.Lib.DistExt contains many things that we as developers found quite useful. Even Guido himself has shown interest in some of our extensions. It is just recently that I'm of the mind to unleash DistExt from its 4Suite-XML binds. -- Jeremy Kloth http://4suite.org/
At 01:36 PM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote:
(Btw, do you have a documentation link? Google doesn't turn up anything except API docs that say nothing about the PEP 314 stuff.)
http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Dist
or (as always, "Read the source, Luke"):
http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup
I'm still lost. What are you *doing* with the requires, provides, and obsoletes fields, other than storing or outputting them? There's nothing in the doc or the source linked above that actually *does* anything with the data that I can see. How do you use this to find and install dependencies, for example?
On Thursday 26 October 2006 1:45 pm, Phillip J. Eby wrote:
At 01:36 PM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 1:14 pm, Phillip J. Eby wrote:
(Btw, do you have a documentation link? Google doesn't turn up anything except API docs that say nothing about the PEP 314 stuff.)
http://skew.org/~mike/4suite-docs/html/modules/Ft.Lib.DistExt.Dist.html#Di st
or (as always, "Read the source, Luke"):
http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/Dist.py?view=markup
I'm still lost. What are you *doing* with the requires, provides, and obsoletes fields, other than storing or outputting them?
Well, you asked where they were mentioned, not used ;)
There's nothing in the doc or the source linked above that actually *does* anything with the data that I can see. How do you use this to find and install dependencies, for example?
The consumer of those fields is actually the PackageManager class: http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view=m... -- Jeremy Kloth http://4suite.org/
At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote:
The consumer of those fields is actually the PackageManager class:
http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view=m...
Yeah, I'm still not getting what it *does* with this information. What's it *for*?
On Thursday 26 October 2006 2:41 pm, Phillip J. Eby wrote:
At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote:
The consumer of those fields is actually the PackageManager class:
http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view =markup
Yeah, I'm still not getting what it *does* with this information. What's it *for*?
Since there is no current way to get the requires/provides/obsoletes from PyPI RPC, they are used to determine the sub-package installation order and to verify that the requirements are installed if not already provided by another sub-package. PackageManager doesn't do any downloading as it is pointless until those fields become available through RPC. And no, I have no desire to do screen scraping to get their values. -- Jeremy Kloth http://4suite.org/
At 03:04 PM 10/26/2006 -0600, Jeremy Kloth wrote:
On Thursday 26 October 2006 2:41 pm, Phillip J. Eby wrote:
At 01:58 PM 10/26/2006 -0600, Jeremy Kloth wrote:
The consumer of those fields is actually the PackageManager class:
http://cvs.4suite.org/viewcvs/4Suite/Ft/Lib/DistExt/PackageManager.py?view =markup
Yeah, I'm still not getting what it *does* with this information. What's it *for*?
Since there is no current way to get the requires/provides/obsoletes from PyPI RPC, they are used to determine the sub-package installation order and to verify that the requirements are installed if not already provided by another sub-package.
What's a sub-package?
PackageManager doesn't do any downloading as it is pointless until those fields become available through RPC.
Okay, I thought you said this was a tool that did something useful with the PEP 314 data. So I'm confused now. This sub-package thing, if I'm getting the gist correctly, only works with stuff you're actually *bundling*, so what difference does PyPI make?
participants (3)
-
Jeremy Kloth -
Phillip J. Eby -
Tim Cera