Re: [Distutils] distlib updated with "locators" API
Daniel Holth <dholth <at> gmail.com> writes:
When you are using setuptools, setup.py can be used to generate declarative metadata with the command "python setup.py egg_info". You can add --egg-base <dir> to put the static metadata in a different directory. Setuptools even makes it easy to add new metadata files to the .egg-info folder by using a plugin to write each file. The majority of the sdists on pypi contain pre-built versions of this static metadata in .egg-info folders.
Can it be correctly said that all of the metadata passed to setup() is in .egg-info? How far can we rely on the accuracy of the content of .egg-info dirs in sdists on PyPI? I have found hundreds of projects where even basic metadata is missing or incorrect - for example, the version.
Unfortunately setup.py is used to build not one, but many different dists (the most common difference of course being different requirements) depending on any number of factors, so pip rebuilds the static metadata before using it. Metadata 1.2+ of course allow us to put the most common "reasons for different dependencies" conditions in the static metadata itself.
Unfortunately, that doesn't help with nearly all the stuff currently on PyPI. Regards, Vinay Sajip
On Mon, Oct 15, 2012 at 2:41 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Daniel Holth <dholth <at> gmail.com> writes:
When you are using setuptools, setup.py can be used to generate declarative metadata with the command "python setup.py egg_info". You can add --egg-base <dir> to put the static metadata in a different directory. Setuptools even makes it easy to add new metadata files to the .egg-info folder by using a plugin to write each file. The majority of the sdists on pypi contain pre-built versions of this static metadata in .egg-info folders.
Can it be correctly said that all of the metadata passed to setup() is in .egg-info? How far can we rely on the accuracy of the content of .egg-info dirs in sdists on PyPI? I have found hundreds of projects where even basic metadata is missing or incorrect - for example, the version.
You can't trust the bundled .egg-info folder, that is why pip rebuilds it during dependency resolution. You can trust the fresh one that you've just built. The .egg-info contains the three things you need during install: Name, Version, requirements. It's always been a mystery to me that we don't just edit PKG-INFO directly.
Unfortunately setup.py is used to build not one, but many different dists (the most common difference of course being different requirements) depending on any number of factors, so pip rebuilds the static metadata before using it. Metadata 1.2+ of course allow us to put the most common "reasons for different dependencies" conditions in the static metadata itself.
Unfortunately, that doesn't help with nearly all the stuff currently on PyPI.
Nope, it's a ~25k-package hard problem. Or ~1.5k if you discard the simple ones.
How could I have forgotten about draft PEP 390: http://www.python.org/dev/peps/pep-0390 , was looking for it in vain in the draft distutils2 docs. I don't like putting my Extensions in setup.cfg but putting at least some of the [metadata] in there is very practical. Simply write a setup.cfg for the ~1.5k packages that have conditional requirements, create a patch, and have the installer check and apply http://example.org/package/patch during the build phase. PEP 390 syntax: [metadata] name = Distribute version = 0.6.4 [metadata:sys_platform == 'win32'] requires = pywin32, bar > 1.0 obsoletes = pywin31 Current bdist_wheel syntax: [metadata] provides-extra = tool signatures faster-signatures requires-dist = distribute >= 0.6.28 markerlib argparse; python_version < '2.7' keyring; extra == 'signatures' dirspec; sys.platform != 'win32' and extra == 'signatures' ed25519ll; extra == 'faster-signatures'
On 16.10.2012, at 20:52, Éric Araujo <merwok@netwok.org> wrote:
Le 16/10/2012 14:45, Daniel Holth a écrit :
How could I have forgotten about draft PEP 390: This is obsolete and should not be implemented.
Hi Éric, Excuse my questions in case that information is somewhere noted but could you elaborate why it's obsolete? What's the replacement? Jannis
On 10/16/12 8:52 PM, Éric Araujo wrote:
Le 16/10/2012 14:45, Daniel Holth a écrit :
How could I have forgotten about draft PEP 390: This is obsolete and should not be implemented. you probably mean:
the code has evolved and the PEP should reflect the latest changes
Regards _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Le 18/10/2012 13:52, Tarek Ziadé a écrit :
you probably mean:
the code has evolved and the PEP should reflect the latest changes
No, I really mean that this PEP was not developed to completion like the other free ones, was not used to guide the development of distutils2, and is no more than a historical document. You told me to ignore it two years ago. :) Cheers
i dont recall this at all . its important to finish it because the syntax and the markers are a basis for several things cheers "Éric Araujo" <merwok@netwok.org> a écrit :
Le 18/10/2012 13:52, Tarek Ziadé a écrit :
you probably mean:
the code has evolved and the PEP should reflect the latest changes
No, I really mean that this PEP was not developed to completion like the other free ones, was not used to guide the development of distutils2, and is no more than a historical document. You told me to ignore it two years ago. :)
Cheers _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.
On 10/18/12 8:08 PM, Éric Araujo wrote:
Le 18/10/2012 13:52, Tarek Ziadé a écrit :
you probably mean:
the code has evolved and the PEP should reflect the latest changes No, I really mean that this PEP was not developed to completion like the other free ones, was not used to guide the development of distutils2, and is no more than a historical document. You told me to ignore it two years ago. :)
Cheers _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig Also this one looks fresher: http://alexis.notmyidea.org/distutils2/setupcfg.html
and has the part about resources that's super important: http://alexis.notmyidea.org/distutils2/setupcfg.html#resources
Zitat von Tarek Ziadé <tarek@ziade.org>:
On 10/16/12 8:52 PM, Éric Araujo wrote:
Le 16/10/2012 14:45, Daniel Holth a écrit :
How could I have forgotten about draft PEP 390: This is obsolete and should not be implemented. you probably mean:
the code has evolved and the PEP should reflect the latest changes
He might also have meant: - "I suggest that this PEP be rejected", or - "I'd like to ask the PEP author to withdraw the PEP" Which of these is most appropriate, I really don't know, but I agree that "is obsolete and should not be implemented" is insufficient: some action must be taken. Regards, Martin
participants (7)
-
Daniel Holth -
Jannis Leidel -
martin＠v.loewis.de -
Tarek Ziade -
Tarek Ziadé -
Vinay Sajip -
Éric Araujo