From Gude.Roland at dlr.de Wed Sep 3 11:26:17 2008
From: Gude.Roland at dlr.de (Gude.Roland at dlr.de)
Date: Wed, 3 Sep 2008 11:26:17 +0200
Subject: [Catalog-sig] High-availability for PyPI, mirroring infrastructures?
Message-ID: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de>
Hello Everybody,
I am new to this list and found the discussion about PyPI Mirrors in the
archieves.
I am working at the German Aerospace Center where I have just started
development of a PyPI mirroring solution which addresses a few issues in
QA and Continous Integration we encountered here.
For Java/maven based projects we use Sonatype-Nexus
(http://nexus.sonatype.org/)
Which solves a lot of problems there but does not help with python.
It should be able to proxy/mirror any number of package indexes (storing
all the files that have been downloaded once.)
It should be able to host multiple indexes which are available under
different urls and which may or may not restrict access to certain
users.
It should be possible to combine several mirrors and/or hosted
repositories into groups which appear to be a single repository.
It should integrate with LDAP/Activedirectory (at some time)
Anyway... the project is in early planning and will be available under
Apache License 2.0
The project has been registered at Sourceforge just yesterday
(https://sourceforge.net/projects/hatchery/)
There is nothing there yet, but hopefully that will change soon.
If you have specific features you would wish for in a mirroring/hosting
solution
Please tell me.
Discussion and participation is always welcome.
Kind regards,
Roland Gude
From lists at zopyx.com Wed Sep 3 11:51:49 2008
From: lists at zopyx.com (Andreas Jung)
Date: Wed, 03 Sep 2008 11:51:49 +0200
Subject: [Catalog-sig] High-availability for PyPI,
mirroring infrastructures?
In-Reply-To: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de>
References: <97A3CC5C4C8069449795A84ED88D631BD7318E@exbe01.intra.dlr.de>
Message-ID: <C86847C7182BD2D0FF294DD9@[192.168.0.24]>
--On 3. September 2008 11:26:17 +0200 Gude.Roland at dlr.de wrote:
>
> Anyway... the project is in early planning and will be available under
> Apache License 2.0
> The project has been registered at Sourceforge just yesterday
> (https://sourceforge.net/projects/hatchery/)
>
How is it different from out project
<http://pypi.python.org/pypi/z3c.pypimirror/>
?
Especially: where is the code?
-aj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080903/b0383d68/attachment.pgp>
From ben at groovie.org Sun Sep 7 08:39:55 2008
From: ben at groovie.org (Ben Bangert)
Date: Sat, 6 Sep 2008 23:39:55 -0700
Subject: [Catalog-sig] Hosting documentation on PyPI
In-Reply-To: <489A017B.6050708@colorstudy.com>
References: <48995F4B.6060409@v.loewis.de>
<4899CE06.3060206@colorstudy.com> <4899D508.7030001@v.loewis.de>
<4899DB3B.5000100@colorstudy.com> <4899DC03.3010203@colorstudy.com>
<4899E3FF.2000600@v.loewis.de> <489A017B.6050708@colorstudy.com>
Message-ID: <8DFA3184-B138-453E-9DC3-BCCDDCC42252@groovie.org>
On Aug 6, 2008, at 12:54 PM, Ian Bicking wrote:
> Also, I'd like the "current" version to be bookmarkable. If every
> URL is bound to a version then you can't do that.
Actually, the only way you can bookmark is if it includes the version.
What's the point of bookmarking URL's that may likely disappear in
version 2.0, 3.0, etc? Unless your bookmark is tied to the version,
there's no saying if its forever... as the feature/module it documents
may be removed, moved, merged elsewhere, etc.
- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2507 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080906/cb8f581a/attachment.bin>
From skip at pobox.com Sun Sep 14 05:42:14 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 13 Sep 2008 22:42:14 -0500
Subject: [Catalog-sig] Access to uploaded documentation?
Message-ID: <18636.34838.200049.343500@montanaro-dyndns-org.local>
I noticed that you can upload documentation for packages in PyPI, so I took
advantage of that for the lockfile module:
http://pypi.python.org/pypi/lockfile
http://packages.python.org/lockfile/
I don't see any links to the latter URL at the former though. How are users
supposed to know documentation is available online? I don't see a link on
the lockfile page.
Thx,
Skip
From lists at zopyx.com Sun Sep 14 08:20:31 2008
From: lists at zopyx.com (Andreas Jung)
Date: Sun, 14 Sep 2008 08:20:31 +0200
Subject: [Catalog-sig] Access to uploaded documentation?
In-Reply-To: <18636.34838.200049.343500@montanaro-dyndns-org.local>
References: <18636.34838.200049.343500@montanaro-dyndns-org.local>
Message-ID: <2D134BEF6316B04D0E9654B3@[192.168.0.24]>
--On 13. September 2008 22:42:14 -0500 skip at pobox.com wrote:
> I noticed that you can upload documentation for packages in PyPI, so I
> took advantage of that for the lockfile module:
>
> http://pypi.python.org/pypi/lockfile
> http://packages.python.org/lockfile/
>
> I don't see any links to the latter URL at the former though. How are
> users supposed to know documentation is available online? I don't see a
> link on the lockfile page.
We have the 'long_description' field of the metadata for providing
documentation. See the various zope.* packages as an example.
-aj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080914/9fe56422/attachment.pgp>
From gary at modernsongs.com Sun Sep 14 19:17:56 2008
From: gary at modernsongs.com (Gary Poster)
Date: Sun, 14 Sep 2008 13:17:56 -0400
Subject: [Catalog-sig] Trouble changing my email address
Message-ID: <5A2A7077-281B-4727-9DA1-8F51879521F7@modernsongs.com>
Hi.
I need to change my email address on PyPI but I'm having problems.
I log in, click on "Your Details" on the top right, and try to change
my email address.
If I try to just change my email address, I get this error message:
-----
Error processing form
password is required
-----
If I then enter my password (in addition to my email change), I get
this error message:
-----
User registration
user "gary" already exists
------
Could someone help?
Thank you
Gary
From flo at chaoflow.net Mon Sep 15 14:38:07 2008
From: flo at chaoflow.net (Florian Friesdorf)
Date: Mon, 15 Sep 2008 14:38:07 +0200
Subject: [Catalog-sig] APGL classifier is missing
Message-ID: <20080915123807.GA30791@marvin>
Hi,
the classifier:
License :: OSI Approved :: GNU Affero General Public License (AGPL)
is missing.
florian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080915/fbb8400c/attachment.pgp>
From chris at simplistix.co.uk Tue Sep 30 17:01:01 2008
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Sep 2008 16:01:01 +0100
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
Message-ID: <48E23F2D.3000802@simplistix.co.uk>
Tarek Ziad? wrote:
> I started to write a new PEP (well a wiki page in fact...) that
> describes a new package called "pypi" that would be dedicated to package
> registering and uploading mechanisms.
> It would also provide enhancements like a proper password hash, or
> deepers metadata controls
>
> http://wiki.python.org/moin/A_new_pypi_module
>
> Any opinions about this PEP ? I tried to include all the problems people
> are having with register and upload.
I think that catalog-sig would be interested in this.
That said, I didn't see any indication of what I consider to be a
critical failure in PyPI: No dependency metadata prior to downloading
the package.
cheers,
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
From ianb at colorstudy.com Tue Sep 30 17:41:11 2008
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 30 Sep 2008 10:41:11 -0500
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <20080930153822.GJ26878@phare.normalesup.org>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
Message-ID: <48E24897.8010800@colorstudy.com>
Gael Varoquaux wrote:
> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
>> That said, I didn't see any indication of what I consider to be a critical
>> failure in PyPI: No dependency metadata prior to downloading the package.
>
> +1. I want to be able do list all the packages an easy_install run will
> download without running it. Something like the "-s" option of apt-get.
> In addition, I want this information to be available programmatically (ie
> with a good api, not something that expects to be called from the command
> line) to be able to use it to build dependency graphs, generate conflicts
> list, or simply tell me that I have requested something that is
> impossible.
>
> There is nothing that I hate more than easy_install failing after having
> half-installed a package because of a missing dependency. This is one of
> the reasons I am never too happy when I have to run easy_install.
FWIW, pyinstall can collect all the packages before installing any of
them. You do have to download all packages, though, as that's the only
way to get the metadata.
--
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
From ziade.tarek at gmail.com Tue Sep 30 17:43:51 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Sep 2008 17:43:51 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E24897.8010800@colorstudy.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
Message-ID: <94bdd2610809300843r516851aasdca03f7ce60878f9@mail.gmail.com>
On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> Gael Varoquaux wrote:
>>
>> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
>>>
>>> That said, I didn't see any indication of what I consider to be a
>>> critical failure in PyPI: No dependency metadata prior to downloading the
>>> package.
>>
>> +1. I want to be able do list all the packages an easy_install run will
>> download without running it. Something like the "-s" option of apt-get.
>> In addition, I want this information to be available programmatically (ie
>> with a good api, not something that expects to be called from the command
>> line) to be able to use it to build dependency graphs, generate conflicts
>> list, or simply tell me that I have requested something that is
>> impossible.
>>
>> There is nothing that I hate more than easy_install failing after having
>> half-installed a package because of a missing dependency. This is one of
>> the reasons I am never too happy when I have to run easy_install.
>
> FWIW, pyinstall can collect all the packages before installing any of them.
> You do have to download all packages, though, as that's the only way to get
> the metadata.
Yes, so having them at PyPI would be a good idea indeed,
I am adding that to that small PEP
>
>
> --
> Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
--
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
From jp.camguilhem at gmail.com Tue Sep 30 17:53:22 2008
From: jp.camguilhem at gmail.com (Jean-Philippe CAMGUILHEM)
Date: Tue, 30 Sep 2008 17:53:22 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E24897.8010800@colorstudy.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
Message-ID: <c74379c90809300853t4e5b472keb8731f252a5daa2@mail.gmail.com>
On Tue, Sep 30, 2008 at 5:41 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> Gael Varoquaux wrote:
>
>> On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
>>
>>> That said, I didn't see any indication of what I consider to be a
>>> critical failure in PyPI: No dependency metadata prior to downloading the
>>> package.
>>>
>>
>> +1. I want to be able do list all the packages an easy_install run will
>> download without running it. Something like the "-s" option of apt-get.
>> In addition, I want this information to be available programmatically (ie
>> with a good api, not something that expects to be called from the command
>> line) to be able to use it to build dependency graphs, generate conflicts
>> list, or simply tell me that I have requested something that is
>> impossible.
>>
>> There is nothing that I hate more than easy_install failing after having
>> half-installed a package because of a missing dependency. This is one of
>> the reasons I am never too happy when I have to run easy_install.
>>
>
> FWIW, pyinstall can collect all the packages before installing any of them.
> You do have to download all packages, though, as that's the only way to get
> the metadata.
is a "simple catalog "db storage for metadata like /usr/ports/ on freebsd a
bad idea ?
the idea is to not download all packages to get the metadata, but just query
the catalog/db
Cheers,
Jean-Philippe
>
>
>
> --
> Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
>
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20080930/9fd6e692/attachment.htm>
From chris at simplistix.co.uk Tue Sep 30 18:13:18 2008
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Sep 2008 17:13:18 +0100
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E24897.8010800@colorstudy.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
Message-ID: <48E2501E.90202@simplistix.co.uk>
Ian Bicking wrote:
> FWIW, pyinstall can collect all the packages before installing any of
> them. You do have to download all packages, though, as that's the only
> way to get the metadata.
...yes, and this is why PyPI should change!
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
From amk at amk.ca Tue Sep 30 18:25:59 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 30 Sep 2008 12:25:59 -0400
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E24897.8010800@colorstudy.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
Message-ID: <20080930162559.GA11804@amk-desktop.matrixgroup.net>
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
> FWIW, pyinstall can collect all the packages before installing any of
> them. You do have to download all packages, though, as that's the only
> way to get the metadata.
Does the DOAP output for a package not contain enough metadata?
--amk
From gael.varoquaux at normalesup.org Tue Sep 30 17:38:22 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 30 Sep 2008 17:38:22 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E23F2D.3000802@simplistix.co.uk>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
Message-ID: <20080930153822.GJ26878@phare.normalesup.org>
On Tue, Sep 30, 2008 at 04:01:01PM +0100, Chris Withers wrote:
> That said, I didn't see any indication of what I consider to be a critical
> failure in PyPI: No dependency metadata prior to downloading the package.
+1. I want to be able do list all the packages an easy_install run will
download without running it. Something like the "-s" option of apt-get.
In addition, I want this information to be available programmatically (ie
with a good api, not something that expects to be called from the command
line) to be able to use it to build dependency graphs, generate conflicts
list, or simply tell me that I have requested something that is
impossible.
There is nothing that I hate more than easy_install failing after having
half-installed a package because of a missing dependency. This is one of
the reasons I am never too happy when I have to run easy_install.
Ga?l
From ianb at colorstudy.com Tue Sep 30 18:37:01 2008
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 30 Sep 2008 11:37:01 -0500
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com> <48E23F2D.3000802@simplistix.co.uk> <20080930153822.GJ26878@phare.normalesup.org> <48E24897.8010800@colorstudy.com>
<20080930162559.GA11804@amk-desktop.matrixgroup.net>
Message-ID: <48E255AD.7020408@colorstudy.com>
A.M. Kuchling wrote:
> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
>> FWIW, pyinstall can collect all the packages before installing any of
>> them. You do have to download all packages, though, as that's the only
>> way to get the metadata.
>
> Does the DOAP output for a package not contain enough metadata?
No. It probably could hold that information, but currently PyPI doesn't
keep any record of requirements, and so the DOAP file it generates
doesn't indicate requirements either.
--
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
From ziade.tarek at gmail.com Tue Sep 30 19:13:03 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Sep 2008 19:13:03 +0200
Subject: [Catalog-sig] [Distutils] "just use debian"
In-Reply-To: <48E255B9.50806@colorstudy.com>
References: <20080917171851.GA27261@logilab.fr>
<94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com>
<1222784868.4166.88.camel@shizuru>
<94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com>
<48E24765.1020908@simplistix.co.uk>
<a26746990809300851l45bc06d2h1428541ff47fd74@mail.gmail.com>
<48E24BDA.2050104@simplistix.co.uk>
<94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com>
<48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com>
Message-ID: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com>
On Tue, Sep 30, 2008 at 6:37 PM, Ian Bicking <ianb at colorstudy.com> wrote:
> Chris Withers wrote:
>>
>> Tarek Ziad? wrote:
>>>>
>>>> Tarek Ziade wrote:
>>>>>
>>>>> For KGS I agree that this is a big work, but there's the need to work
>>>>> at a
>>>>> higher level that in your package
>>>>
>>>> Why? You really need to explain to me why the dependency information in
>>>> each
>>>> of the packages isn't enough?
>>>
>>> Because you can keep up with the dependencies changes, removed, or
>>> introduced
>>> by a package you depend on.
>>
>> Why can this not be expressed in the dependency information in the
>> package?
>
> I tried this briefly for a while when Setuptools first came out, and I found
> it completely unmaintainable.
>
> Say I have a package that represents an application. We'll call it FooBlog.
> I release version 1.0. It uses the Turplango web framework (1.5 at the
> time of release) and the Storchalmy ORM (0.4), and Turplango uses HardJSON
> (1.2.1).
>
> I want my version 1.0 to keep working. So, I figure I'll add the
> dependencies:
>
> Turplango==1.5
> Storchalmy==0.4
>
> Then HardJSON 2.0 is released, and Turplango only required HardJSON>=1.2, so
> new installations start installing HardJSON 2.0. But my app happens not to
> be compatible with that library, and so it's broken. OK... so, I could add
> HardJSON==1.2.1 in my requirements.
>
> But then a small bug fix, HardJSON 1.2.2 comes out, that fixes a security
> bug. Turplango releases version 1.5.1 that requires HardJSON>=1.2.2. I now
> have have to update FooBlog to require both Turplango==1.5.1 and
> HardJSON==1.2.2.
>
> Later on, I decide that Turplango 1.6 fixes some important bugs, and I want
> to try it with my app. I can install Turplango 1.6, but I can't start my
> app because I'll get a version conflict. So to even experiment with a new
> version of the app, I have to check out FooBlog, update setup.py, reinstall
> (setup.py develop) the package, and then I can start using it. But if I've
> made other hard requirements of packages like HardJSON, I'll have to update
> all those too.
>
> So... that's the kind of thing I encountered with just a couple
> dependencies, but in practice it was much worse because there were a lot
> more than 3 libraries involved. I now think it is best to only use version
> requirements to express known conflicts. For future versions of packages
> you can't really know if they will cause conflicts until they are released.
Exactly, you can't control everything from your package unless you
work in an isolated environement like virtualenv or zc.buildout
provides, so I can't see any solution unless someone is taking care of
it at a higher level :(
maybe PyPI though, can automate this, when a package is uploaded, by
browsing all dependency and
finding relevant conflict ? PyPI "knows" all the packages out there.
At least display those conflicts somehow ? or warn about them.
(I am pushing this to catalog-sig as well, sorry for the cross-post. I
do thing though, these mailing lists should merge)
>
>
> --
> Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
>
--
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
From ianb at colorstudy.com Tue Sep 30 19:25:17 2008
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 30 Sep 2008 12:25:17 -0500
Subject: [Catalog-sig] [Distutils] "just use debian"
In-Reply-To: <94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com>
References: <20080917171851.GA27261@logilab.fr>
<94bdd2610809300649q6bc9cdf3t84d5221d956bfc9d@mail.gmail.com>
<1222784868.4166.88.camel@shizuru>
<94bdd2610809300820g36a3f1bbpc2a04e03b43ee9c9@mail.gmail.com>
<48E24765.1020908@simplistix.co.uk>
<a26746990809300851l45bc06d2h1428541ff47fd74@mail.gmail.com>
<48E24BDA.2050104@simplistix.co.uk>
<94bdd2610809300901o3fa62601q985784f01a8ad774@mail.gmail.com>
<48E24FF7.9000300@simplistix.co.uk> <48E255B9.50806@colorstudy.com>
<94bdd2610809301013m5d07865eu7eabc4cb3b62d354@mail.gmail.com>
Message-ID: <48E260FD.9010309@colorstudy.com>
Tarek Ziad? wrote:
>> So... that's the kind of thing I encountered with just a couple
>> dependencies, but in practice it was much worse because there were a lot
>> more than 3 libraries involved. I now think it is best to only use version
>> requirements to express known conflicts. For future versions of packages
>> you can't really know if they will cause conflicts until they are released.
>
> Exactly, you can't control everything from your package unless you
> work in an isolated environement like virtualenv or zc.buildout
> provides, so I can't see any solution unless someone is taking care of
> it at a higher level :(
>
> maybe PyPI though, can automate this, when a package is uploaded, by
> browsing all dependency and
> finding relevant conflict ? PyPI "knows" all the packages out there.
>
> At least display those conflicts somehow ? or warn about them.
Yes, keeping this version information separate from packages would help,
I think. If you find out more information about a conflict it shouldn't
require a new release -- new releases take a while to do, and have
cascading effects. This kind of metadata isn't so much about the
package, as about how the package relates to other packages. If we
could somewhat safely have collaborative conflict information that would
be nice, though there's different kinds of conflicts so it might be
infeasible. It's all too common for a person to just poke around with
version stuff until something works, but in a way that is only accurate
for the context of their application, and if they submit that
information upstream they could easily break other people's setups
unnecessarily.
--
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org
From gael.varoquaux at normalesup.org Tue Sep 30 19:38:26 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 30 Sep 2008 19:38:26 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E24897.8010800@colorstudy.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
Message-ID: <20080930173826.GL26878@phare.normalesup.org>
On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
>> There is nothing that I hate more than easy_install failing after having
>> half-installed a package because of a missing dependency. This is one of
>> the reasons I am never too happy when I have to run easy_install.
>
> FWIW, pyinstall can collect all the packages before installing any of them.
> You do have to download all packages, though, as that's the only way to
> get the metadata.
Yes, I have seen that. I was very happy to witness the release of this
tool. Thank you.
Ga?l
From pje at telecommunity.com Tue Sep 30 19:51:30 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 30 Sep 2008 13:51:30 -0400
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <20080930162559.GA11804@amk-desktop.matrixgroup.net>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
<20080930162559.GA11804@amk-desktop.matrixgroup.net>
Message-ID: <20080930175201.106863A409C@sparrow.telecommunity.com>
At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
>On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
> > FWIW, pyinstall can collect all the packages before installing any of
> > them. You do have to download all packages, though, as that's the only
> > way to get the metadata.
>
>Does the DOAP output for a package not contain enough metadata?
Nope. And it can't possibly do so, unless it contains dependency
data for every possible variation of the package. For example, a
package might dynamically declare dependency on ctypes, depending on
whether you're installing it for Python 2.4 or Python
2.5. (Dependencies can also be platform-specific and
build-option-specific, as well as Python-version-specific.)
From cakebread at gmail.com Tue Sep 30 20:21:54 2008
From: cakebread at gmail.com (Rob Cakebread)
Date: Tue, 30 Sep 2008 11:21:54 -0700
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <20080930175201.106863A409C@sparrow.telecommunity.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
<20080930162559.GA11804@amk-desktop.matrixgroup.net>
<20080930175201.106863A409C@sparrow.telecommunity.com>
Message-ID: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com>
On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
>>
>> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
>> > FWIW, pyinstall can collect all the packages before installing any of
>> > them. You do have to download all packages, though, as that's the only
>> > way to get the metadata.
>>
>> Does the DOAP output for a package not contain enough metadata?
>
> Nope. And it can't possibly do so, unless it contains dependency data for
> every possible variation of the package. For example, a package might
> dynamically declare dependency on ctypes, depending on whether you're
> installing it for Python 2.4 or Python 2.5. (Dependencies can also be
> platform-specific and build-option-specific, as well as
> Python-version-specific.)
>
Not to mention the DOAP vocabulary lacks a way to describe dependency
information. This is planned but it has to be well thought out because of
all the variations Philip mentions.
The good news is much of this dependency info is already in existence in
Linux distributions. Take a Gentoo ebuild, for example. It has separate
run-time, build-time and test dependency info, dependencies based on
enabled features, and dependencies based on the version of Python used.
Ebuilds also have metadata mapping the PyPI name to the Gentoo
package name, so it'll be easy enough to create a database with all this info.
I'm working on this now at http://doapspace.org/ where you can find DOAP
for Python packages with a bit more metadata than the DOAP supplied on
PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME )
From martin at v.loewis.de Tue Sep 30 22:08:25 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 30 Sep 2008 22:08:25 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E23F2D.3000802@simplistix.co.uk>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
Message-ID: <48E28739.7080901@v.loewis.de>
> That said, I didn't see any indication of what I consider to be a
> critical failure in PyPI: No dependency metadata prior to downloading
> the package.
Contributions are welcome. The source code of PyPI is available
publically, and I'm willing to accept patches. I won't have time
to work on this in the next 12 months myself.
Regards,
Martin
From chris at simplistix.co.uk Tue Sep 30 22:26:35 2008
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 30 Sep 2008 21:26:35 +0100
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E28739.7080901@v.loewis.de>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de>
Message-ID: <48E28B7B.3060602@simplistix.co.uk>
Martin v. L?wis wrote:
>> That said, I didn't see any indication of what I consider to be a
>> critical failure in PyPI: No dependency metadata prior to downloading
>> the package.
>
> Contributions are welcome. The source code of PyPI is available
> publically,
Where?
> and I'm willing to accept patches. I won't have time
> to work on this in the next 12 months myself.
These two don't seem to go hand in hand and don't really seem to fit my
experiene of the catalog-sig :-(
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
From ziade.tarek at gmail.com Tue Sep 30 22:41:12 2008
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 30 Sep 2008 22:41:12 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk>
<20080930153822.GJ26878@phare.normalesup.org>
<48E24897.8010800@colorstudy.com>
<20080930162559.GA11804@amk-desktop.matrixgroup.net>
<20080930175201.106863A409C@sparrow.telecommunity.com>
<9b06ffb10809301121r589e1784je1300fb2632ab46@mail.gmail.com>
Message-ID: <94bdd2610809301341j60830399s37cf310939783b86@mail.gmail.com>
On Tue, Sep 30, 2008 at 8:21 PM, Rob Cakebread <cakebread at gmail.com> wrote:
> On Tue, Sep 30, 2008 at 10:51 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
>> At 12:25 PM 9/30/2008 -0400, A.M. Kuchling wrote:
>>>
>>> On Tue, Sep 30, 2008 at 10:41:11AM -0500, Ian Bicking wrote:
>>> > FWIW, pyinstall can collect all the packages before installing any of
>>> > them. You do have to download all packages, though, as that's the only
>>> > way to get the metadata.
>>>
>>> Does the DOAP output for a package not contain enough metadata?
>>
>> Nope. And it can't possibly do so, unless it contains dependency data for
>> every possible variation of the package. For example, a package might
>> dynamically declare dependency on ctypes, depending on whether you're
>> installing it for Python 2.4 or Python 2.5. (Dependencies can also be
>> platform-specific and build-option-specific, as well as
>> Python-version-specific.)
>>
>
> Not to mention the DOAP vocabulary lacks a way to describe dependency
> information. This is planned but it has to be well thought out because of
> all the variations Philip mentions.
out of curiosity, :
- can a RDF-based database can possibly handle such a graph ?
- would it make sense for PyPI to query the doap server to get those
dependency infos ?
-
>
> The good news is much of this dependency info is already in existence in
> Linux distributions. Take a Gentoo ebuild, for example. It has separate
> run-time, build-time and test dependency info, dependencies based on
> enabled features, and dependencies based on the version of Python used.
>
> Ebuilds also have metadata mapping the PyPI name to the Gentoo
> package name, so it'll be easy enough to create a database with all this info.
>
> I'm working on this now at http://doapspace.org/ where you can find DOAP
> for Python packages with a bit more metadata than the DOAP supplied on
> PyPI ( http://doapspace.org/doap/py/PYPI_PKG_NAME )
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>
--
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
From martin at v.loewis.de Tue Sep 30 22:49:12 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 30 Sep 2008 22:49:12 +0200
Subject: [Catalog-sig] [Distutils] PEP for distutils
In-Reply-To: <48E28B7B.3060602@simplistix.co.uk>
References: <94bdd2610809280555p12c0e326r4a867bd3b67efbd9@mail.gmail.com>
<48E23F2D.3000802@simplistix.co.uk> <48E28739.7080901@v.loewis.de>
<48E28B7B.3060602@simplistix.co.uk>
Message-ID: <48E290C8.4030008@v.loewis.de>
Chris Withers wrote:
>> Contributions are welcome. The source code of PyPI is available
>> publically,
>
> Where?
https://svn.python.org/packages/trunk/
>> and I'm willing to accept patches. I won't have time
>> to work on this in the next 12 months myself.
>
> These two don't seem to go hand in hand and don't really seem to fit my
> experiene of the catalog-sig :-(
Saying what? that I do have time in the next twelve months? That's nice
to hear.
Regards,
Martin