Hello Let's write PEPs ! I started to write a new PEP (well a wiki page in fact...) that describes a new package called "pypi" that would be dedicated to package registering and uploading mechanisms. It would also provide enhancements like a proper password hash, or deepers metadata controls http://wiki.python.org/moin/A_new_pypi_module Any opinions about this PEP ? I tried to include all the problems people are having with register and upload. I linked it here in the page where I am trying to summarize the current state of distutils http://wiki.python.org/moin/Distribute (work in progress) Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
Tarek Ziadé wrote:
I started to write a new PEP (well a wiki page in fact...) that describes a new package called "pypi" that would be dedicated to package registering and uploading mechanisms. It would also provide enhancements like a proper password hash, or deepers metadata controls
http://wiki.python.org/moin/A_new_pypi_module
Any opinions about this PEP ? I tried to include all the problems people are having with register and upload.
I think that catalog-sig would be interested in this. That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
+1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible. There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install. Gaël
Gael Varoquaux wrote:
On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
+1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible.
There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install.
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking
Gael Varoquaux wrote:
On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
+1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible.
There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install.
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Yes, so having them at PyPI would be a good idea indeed, I am adding that to that small PEP
-- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking
Gael Varoquaux wrote:
On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
+1. I want to be able do list all the packages an easy_install run will download without running it. Something like the "-s" option of apt-get. In addition, I want this information to be available programmatically (ie with a good api, not something that expects to be called from the command line) to be able to use it to build dependency graphs, generate conflicts list, or simply tell me that I have requested something that is impossible.
There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install.
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
is a "simple catalog "db storage for metadata like /usr/ports/ on freebsd a bad idea ? the idea is to not download all packages to get the metadata, but just query the catalog/db Cheers, Jean-Philippe
-- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
...yes, and this is why PyPI should change! Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Does the DOAP output for a package not contain enough metadata? --amk
A.M. Kuchling wrote:
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Does the DOAP output for a package not contain enough metadata?
No. It probably could hold that information, but currently PyPI doesn't keep any record of requirements, and so the DOAP file it generates doesn't indicate requirements either. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Does the DOAP output for a package not contain enough metadata?
Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.)
On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby
At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Does the DOAP output for a package not contain enough metadata?
Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.)
Not to mention the DOAP vocabulary lacks a way to describe dependency information. This is planned but it has to be well thought out because of all the variations Philip mentions. The good news is much of this dependency info is already in existence in Linux distributions. Take a Gentoo ebuild, for example. It has separate run-time, build-time and test dependency info, dependencies based on enabled features, and dependencies based on the version of Python used. Ebuilds also have metadata mapping the PyPI name to the Gentoo package name, so it'll be easy enough to create a database with all this info. I'm working on this now at http://doapspace.org/ where you can find DOAP for Python packages with a bit more metadata than the DOAP supplied on PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME )
On Tue, Sep 30, 2008 at 8:21 PM, Rob Cakebread
On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby
wrote: At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Does the DOAP output for a package not contain enough metadata?
Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.)
Not to mention the DOAP vocabulary lacks a way to describe dependency information. This is planned but it has to be well thought out because of all the variations Philip mentions.
out of curiosity, : - can a RDF-based database can possibly handle such a graph ? - would it make sense for PyPI to query the doap server to get those dependency infos ? -
The good news is much of this dependency info is already in existence in Linux distributions. Take a Gentoo ebuild, for example. It has separate run-time, build-time and test dependency info, dependencies based on enabled features, and dependencies based on the version of Python used.
Ebuilds also have metadata mapping the PyPI name to the Gentoo package name, so it'll be easy enough to create a database with all this info.
I'm working on this now at http://doapspace.org/ where you can find DOAP for Python packages with a bit more metadata than the DOAP supplied on PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME ) _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/
Phillip J. Eby wrote:
Nope. And it can't possibly do so, unless it contains dependency data for every possible variation of the package. For example, a package might dynamically declare dependency on ctypes, depending on whether you're installing it for Python 2.4 or Python 2.5. (Dependencies can also be platform-specific and build-option-specific, as well as Python-version-specific.)
Ug. No-one actually does this do they? man, setup.py both sucks and blows at the same time :-( Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
There is nothing that I hate more than easy_install failing after having half-installed a package because of a missing dependency. This is one of the reasons I am never too happy when I have to run easy_install.
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
Yes, I have seen that. I was very happy to witness the release of this tool. Thank you. Gaël
Ian Bicking wrote:
FWIW, pyinstall can collect all the packages before installing any of them. You do have to download all packages, though, as that's the only way to get the metadata.
This may be something to make sure is on the requirements list for a metatdata standard: Make sure there is a defined way of getting just the metadata from a repository without having to download the whole package. -- Greg
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
Contributions are welcome. The source code of PyPI is available publically, and I'm willing to accept patches. I won't have time to work on this in the next 12 months myself. Regards, Martin
Martin v. Löwis wrote:
That said, I didn't see any indication of what I consider to be a critical failure in PyPI: No dependency metadata prior to downloading the package.
Contributions are welcome. The source code of PyPI is available publically,
Where?
and I'm willing to accept patches. I won't have time to work on this in the next 12 months myself.
These two don't seem to go hand in hand and don't really seem to fit my experiene of the catalog-sig :-( Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
Contributions are welcome. The source code of PyPI is available publically,
Where?
https://svn.python.org/packages/trunk/
and I'm willing to accept patches. I won't have time to work on this in the next 12 months myself.
These two don't seem to go hand in hand and don't really seem to fit my experiene of the catalog-sig :-(
Saying what? that I do have time in the next twelve months? That's nice to hear. Regards, Martin
participants (10)
-
"Martin v. Löwis"
-
A.M. Kuchling
-
Chris Withers
-
Gael Varoquaux
-
Greg Ewing
-
Ian Bicking
-
Jean-Philippe CAMGUILHEM
-
Phillip J. Eby
-
Rob Cakebread
-
Tarek Ziadé