"Sridhar Ratnakumar" writes:

What are the possible use-cases (by the tools) of these new fields would be?

One of the important consumers of package metadata is PyPI. I imagine these fields will be presented to the user along with other data about the package. -- \ “I got an answering machine for my phone. Now when someone | `\ calls me up and I'm not home, they get a recording of a busy | _o__) signal.” —Steven Wright | Ben Finney

On Mon, Nov 23, 2009 at 10:20 AM, Floris Bruynooghe wrote:

On Sun, Nov 22, 2009 at 04:23:33PM -0800, Sridhar Ratnakumar wrote:

...

On Sun, 22 Nov 2009 14:52:11 -0800, Tarek Ziadé wrote:

...

Hello As suggested in Catalog-SIG by some people, I would like to propose the addition of three more fields for 1.2: "Repository-URL"    A string containing the URL for the package's repository.   Example:        http://svn.python.org/projects/python/trunk/ "Repository-Browse-URL"     A string containing the URL for the package's browsable repository.   Example:      http://svn.python.org/view/python/trunk "Bug-Tracker-URL"   A string containing the URL for the package's bug tracker   Example:      http://bugs.python.org/

What are the possible use-cases (by the tools) of these new fields would be? I can imagine Repository-URL to be of use (eg: pip install IPython==dev), but have no idea why the other two should be added.

Same as for Author and Author-email etc: more information for the (human) users.  I think this is very valuable when browsing packages on e.g. PyPI.

Yes. What is happening right now is that people are adding sometimes these info in the long_description, but there's no way to display these informations at PyPI in a specific location on the page, like a portlet for example. PyPI is extracting urls out of long_description to put them in the simple index for tools like easy_install to process them, but that's it. If you look at plone.org (which is PyPI -compatible) : http://plone.org/products/jyu.pathkey Each project can display in the "project resources" some of these links, but that's currently done manually. Having them in the metadata will make it easier for PyPI and other similar servers to get them and display them, and will encourage the developers to use them from within their setup.py This proposal was triggered in the discussion about the rating/commenting system : someone said that we would have much more value in displaying these links, for people to find the code and the tracker. Tarek

Tarek Ziadé wrote:

Hello

As suggested in Catalog-SIG by some people, I would like to propose the addition of three more fields for 1.2:

"Repository-URL" A string containing the URL for the package's repository. Example: http://svn.python.org/projects/python/trunk/

"Repository-Browse-URL" A string containing the URL for the package's browsable repository. Example: http://svn.python.org/view/python/trunk

"Bug-Tracker-URL" A string containing the URL for the package's bug tracker Example: http://bugs.python.org/

Any thoughts ?

Hmmm, I thought I replied to this but don't see my mail in the archives: Yes to all of the above, could we also please have: Mailing-List-URL Change-Log-URL Documentation-URL I see the primary consumer of all 6 of these being PyPI, to give potential package users more direct information about where to go in a standard format. thanks, Chris

Tarek Ziadé writes:

On Sat, Nov 28, 2009 at 11:33 PM, Chris Withers wrote:

...

Change-Log-URL

I don't see a lot of projects with a changelog file available online, rather a changelog file in the release in CHANGELOG and sometimes a changelog displayed in the long_description field, so it's shown at PyPI.

I think that's a poor practice; the long description shouldn't need to change every release and shouldn't include a changelog.

Can you detail on this one ?

I would like to see more projects making their changelog a separate document, so it could be referenced simply and separately from other information about the project. This field would encourage that by making it more visible.

...

Documentation-URL

How would this one be different from Home-page ?

The home page is a “first impressions” page. It usually doesn't jump straight into a user's guide and technical documentation and interface specification. Many projects have a separate index page specifically for documentation; the URL of that page would be the value for this field. -- \ “Value your freedom or you will lose it, teaches history. | `\ “Don't bother us with politics,” respond those who don't want | _o__) to learn.” —Richard Stallman, 2002 | Ben Finney

Tarek Ziadé writes:

How about making it "Change-Log" in that case, and have the content of the changelog directly added in the metadata ? Rather than an url.

That might be good, if the changelog was in a separate file. It's quite commonly a very long text file, as it retains the change history of the code base indefinitely. -- \ “Shepherds … look after their sheep so they can, first, fleece | `\ them and second, turn them into meat. That's much more like the | _o__) priesthood as I know it.” —Christopher Hitchens, 2008-10-29 | Ben Finney

Tarek Ziadé writes:

On Sun, Nov 29, 2009 at 2:19 PM, Ben Finney wrote:

...

[having the changelog in the package metadata] might be good, if the changelog was in a separate file. It's quite commonly a very long text file, as it retains the change history of the code base indefinitely.

Yes, and if it's reStructuredText based, we will be able to do what people do actually, by integrating a reST-based CHANGELOG file at the end of long_description:

Let's not get ahead of ourselves :-) Asking developers to declare where their changelog file is will be much easier to get than asking them to change the format of that file. Stick to a simple standard of “this file contains the changelog”, which will let people simply move their existing changelog of whatever format into that file. Having it in a predictable location would be a big step forward, and an easy one to enforce. -- \ “Our products just aren't engineered for security.” —Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__) development, 2002 | Ben Finney

Tarek Ziadé writes:

On Sun, Nov 29, 2009 at 3:05 PM, Ben Finney wrote:

...

Stick to a simple standard of “this file contains the changelog”, which will let people simply move their existing changelog of whatever format into that file. Having it in a predictable location would be a big step forward, and an easy one to enforce.

That doesn't differ from what we have today with long_description:

That ignores what I thought we'd just agreed in this thread: that the changelog should be *in a separate file*. Putting it into one field among many ignores the fact that it's often long, and growing every release. -- \ “It ain't so much the things we don't know that get us in | `\ trouble. It's the things we know that ain't so.” —Artemus Ward | _o__) (1834–1867), U.S. journalist | Ben Finney

On Sun, Nov 29, 2009 at 10:32 PM, Ben Finney wrote:

Tarek Ziadé writes:

...

On Sun, Nov 29, 2009 at 3:05 PM, Ben Finney wrote:

...

Stick to a simple standard of “this file contains the changelog”, which will let people simply move their existing changelog of whatever format into that file. Having it in a predictable location would be a big step forward, and an easy one to enforce.

That doesn't differ from what we have today with long_description:

That ignores what I thought we'd just agreed in this thread: that the changelog should be *in a separate file*. Putting it into one field among many ignores the fact that it's often long, and growing every release.

It can be a file of course in your distribution, but at the end, it has to be a field in the metadata, like long_description is. Here's an example of what could be done in your setup.py: ==== with open('README.txt') as f: README = f.read() with open('CHANGELOG') as f: CHANGELOG = f.read() setup(name='foo', ..., long_description=README, changelog=CHANGELOG) ==== Regards Tarek

Chris Withers wrote:

Tarek Ziadé wrote:

...

On Sat, Nov 28, 2009 at 11:33 PM, Chris Withers wrote: [..]

...

Yes to all of the above, could we also please have:

Mailing-List-URL

+1

...

Change-Log-URL

I don't see a lot of projects with a changelog file available online,

...except every single Zope-related egg ;-)

These currently get put in the long-description, often up at the top, usually making the actual description totally unreadable.

Having the option to put a url in a standard field would encourage people to just put a link to the changelog.

Why does it need to be in a standard field? Who's the consumer of this, and what are they doing with the value? Can't you just stick the URL in the long description?

On Mon, Nov 30, 2009 at 1:20 PM, Ben Finney wrote:

Tarek Ziadé writes:

...

On Sun, Nov 29, 2009 at 10:32 PM, Ben Finney wrote:

...

[…] I thought we'd just agreed in this thread: that the changelog should be *in a separate file*. Putting it into one field among many ignores the fact that it's often long, and growing every release.

It can be a file of course in your distribution, but at the end, it has to be a field in the metadata,

I don't understand the distinction you're making here. What is the difference between “in your distribution” versus “in the end”?

You have to make a distinction between the "long_description" option in setup.py, and the "Description" metadata field, that lands in the PKG-INFO file (==PEP 345 fields) The PKG-INFO file, that contains all the metadata, is built by a call to "python setup.py sdist" for example , is shipped within your distribution, and read for instance by PyPI. IOW, if we are going to add a distinct field for "Changelog", this will not be a file path, but an URL *or* a raw text exactly like the "Description" field. Having a file for your changelog, that you are reading and using to fill a metadata field is just a implementation detail. Regards Tarek

On Mon, Nov 30, 2009 at 8:17 PM, Ian Bicking wrote:

On Sun, Nov 22, 2009 at 4:52 PM, Tarek Ziadé wrote:

...

As suggested in Catalog-SIG by some people, I would like to propose the addition of three more fields for 1.2:

"Repository-URL"    A string containing the URL for the package's repository.   Example:        http://svn.python.org/projects/python/trunk/

I like this in concept, but it's a bit vague as just a URL.  What kind of repository is it?  Sniffing is possible, but doesn't work that well.  For pip I require something like hg+http://... (with exceptions for schemes that are self-describing, like svn: and git:).

I didn't think that Pip would use this field to do some automatic process, I think the use case was to simply display that url at PyPI. OTHO, Repository-Browse-URL already plays that role, so it would be interesting to provide in "Repository-URL", urls that could be processed by installers. Do you have a particular use case in mind ?

It's also unclear whether it should point to the trunk, a branch, or a tag (related to the release).  As a URL that's only even possible for svn, as git/hg/bzr all point to the repository and URLs can't point to revisions, tags, etc.  (pip has some extensions to the URL format to accommodate these, but they aren't formally defined at this time).

Good point. I think this would end up being the trunk/default path *or* the root of their repository, because I doubt developers will update this field for every release they have. And, as you said, in mercurial for instance, we would need to "hg up some_tag" to have a similar feature than svn branches. So at the end, maybe we could just keep "Repository-Browse-URL" and drop the other one unless we can come up with some use cases for automatic processing of the url. Tarek.

On Mon, Nov 30, 2009 at 01:17:18PM -0600, Ian Bicking wrote:

On Sun, Nov 22, 2009 at 4:52 PM, Tarek Ziadé wrote:

...

As suggested in Catalog-SIG by some people, I would like to propose the addition of three more fields for 1.2:

"Repository-URL" A string containing the URL for the package's repository. Example: http://svn.python.org/projects/python/trunk/

I like this in concept, but it's a bit vague as just a URL. What kind of repository is it? Sniffing is possible, but doesn't work that well. For pip I require something like hg+http://... (with exceptions for schemes that are self-describing, like svn: and git:).

Very good point, the reason I proposed these was because I know Debian uses them but I failed to properly look at their experience. Actually turns out that currently they use: Vcs-Svn, Vcs-Git, etc. as well as Vcs-Browser. But searching around they are also having this discussion and looking that changing that because it doens't scale (needing a new field for each new VCS...). How about two fields? Repository-VCS and Repository-URL where the first is something like "git", "hg", "svn" etc.

It's also unclear whether it should point to the trunk, a branch, or a tag (related to the release). As a URL that's only even possible for svn, as git/hg/bzr all point to the repository and URLs can't point to revisions, tags, etc. (pip has some extensions to the URL format to accommodate these, but they aren't formally defined at this time).

It's probably fine to leave that up to the developer and not specify it, PyPI's just needs to display it to the user. Whatever the developer sees fit. Coming up with a schema that is and future proof and works with all VCSes out there will be pretty hard. Regards Floris

On Mon, Nov 30, 2009 at 4:30 PM, Tarek Ziadé wrote:

...

On Sun, Nov 22, 2009 at 4:52 PM, Tarek Ziadé wrote:

...

As suggested in Catalog-SIG by some people, I would like to propose the addition of three more fields for 1.2:

"Repository-URL" A string containing the URL for the package's repository. Example: http://svn.python.org/projects/python/trunk/

I like this in concept, but it's a bit vague as just a URL. What kind of repository is it? Sniffing is possible, but doesn't work that well. For pip I require something like hg+http://... (with exceptions for schemes

On Mon, Nov 30, 2009 at 8:17 PM, Ian Bicking wrote: that

...

are self-describing, like svn: and git:).

I didn't think that Pip would use this field to do some automatic process, I think the use case was to simply display that url at PyPI.

OTHO, Repository-Browse-URL already plays that role, so it would be interesting to provide in "Repository-URL", urls that could be processed by installers.

Do you have a particular use case in mind ?

I don't really. There is an active use case for links like: http://repo-location/MyPackage/trunk#egg=MyPackage-dev which are used to get the dev version. These links should go to a svn index (which both easy_install and pip understand specifically) or to some automatically-generated tarball (which I believe most of the other VCSs generate through their popular frontends, e.g., http://bitbucket.org/ianb/webob/get/tip.zip). If something more like that was added (more like Trunk-URL), that would be genuinely useful. Barring that, I'd recommend removing the Repository-URL field from the recommendation until we have a clearer use case. Right now, the better use case would be a field giving the checkout command, e.g.,: Repository-Checkout: hg clone http://bitbucket.org/ianb/webob But I don't think that is particularly useful, only that it is more useful than Repository-URL. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker

On Tue, Dec 1, 2009 at 10:11 AM, Tarek Ziadé wrote:

The value I see in that field is that:

- it helps developers structurize a bit their project description. e.g. letting them separate  the Description of the package from the changelog if any.

Package creators can already do this by including a link to the change log if they so choose.

- it will let PyPI separate the front page of the project from its changelog, e.g. in this page:  http://pypi.python.org/pypi/zc.buildout, instead of having the changelog here, it could be a link  to another page for the project.

Again, package creators can already do this.

How does that sounds ?

I don't understand the value of this proposal. We don't need more overhead for package authors to wade through, even if they end up ignoring it. No... *especially* if they end up ignoring it. -- Benji York

On Tue, Dec 1, 2009 at 10:50 AM, Tarek Ziadé wrote: [snip a reiteration of the perceived benefits]

So I guess you are also against the addition of "Repository-URL" and "Bug-Tracker-URL" too ? Since those can be links in the long_description as well.

I am. Don't get me wrong, I don't think these proposals will kill kittens or anything, it's just that every addition has a cost. I (apparently) see that cost as being higher than others do, therefore I feel the small benefit is swamped by the cost. -- Benji York

On Tue, Dec 1, 2009 at 9:11 AM, Tarek Ziadé wrote:

On Tue, Dec 1, 2009 at 4:19 AM, Ian Bicking wrote: [..]

...

If something more like that was added (more like Trunk-URL), that would be genuinely useful. Barring that, I'd recommend removing the Repository-URL field from the recommendation until we have a clearer use case. Right now, the better use case would be a field giving the checkout command, e.g.,: Repository-Checkout: hg clone http://bitbucket.org/ianb/webob But I don't think that is particularly useful, only that it is more useful than Repository-URL.

Here's a proposal then, that seems to synthetize what people have been saying:

Let's drop "Repository-Browse-URL" and keep a single "Repository-URL" field, which is a free URL that can take any URL form. e.g. a browsable url, or a git/hg url etc.

I prefer Repository-Browse-URL, as it is more explicit in its use: it's a link that someone using a browser can usefully click through to. I expect it will be displayed as such on PyPI. So this link is good: http://github.com/cloudkick/libcloud And this link is bad: git://github.com/cloudkick/libcloud.git A similar distinction exists for code.google.com projects. Now for "Change-Log" vs "Change-Log-URL", I think the first one is

better, because that's what people are already doing in their packages (when they add their changelog at the end of their long_description option), and it's not hard for PyPI to store it and display it, besides Description.

This seems fine to me. Is ReST allowed? Could one potentially just do: `Changes http://myproject.com/changes.html`_ ? And then essentially the changelog would be a single link? I'm not sure if that's a good idea. Would it be too vague to say that if the change log is a single URL that PyPI should link directly through to the change log instead of displaying the link? (The exact UI for PyPI hasn't been proposed, but if it's something like a tab with changes, that tab could actually link offsite) -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker

Benji York wrote:

On Tue, Dec 1, 2009 at 10:50 AM, Tarek Ziadé wrote: [snip a reiteration of the perceived benefits]

...

So I guess you are also against the addition of "Repository-URL" and "Bug-Tracker-URL" too ? Since those can be links in the long_description as well.

I am.

Don't get me wrong, I don't think these proposals will kill kittens or anything, it's just that every addition has a cost. I (apparently) see that cost as being higher than others do, therefore I feel the small benefit is swamped by the cost.

I agree with Benji that adding new fields has a cost and should be avoided. Just because we can add structure doesn't mean we should. In the specific case of Repository-URL and Bug-Tracker-URL, the idea was that PyPI would change its behavior if these fields were present (by not using the rating system). So I'm okay with those. But for other fields, like Repository-Checkout and Change-Log-URL, I haven't seen a use case yet. Eric.

Tarek Ziadé wrote:

How about making it "Change-Log" in that case, and have the content of the changelog directly added in the metadata ? Rather than an url.

Well, I wouldn't object to that if it was added in addition to Change-Log-URL, but bear in mind this makes much more work for PyPI than Change-Log-URL. Change-Log requires a new page, and integration with the package page. Change-Log-URL just requires a new url field added on the existing package page. Change-Log-URL also gives the freedom for package maintainers to keep their changelog wherever they like and refer to it. This might not be on PyPI; my changelogs are becoming sphinx pages, and while those pages are currently on packages.python.org, they will eventually be on http://simplistix.couk/software. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2009/12/1 Eric Smith :

Benji York wrote:

...

On Tue, Dec 1, 2009 at 10:50 AM, Tarek Ziadé wrote: [snip a reiteration of the perceived benefits]

...

So I guess you are also against the addition of "Repository-URL" and "Bug-Tracker-URL" too ? Since those can be links in the long_description as well.

I am.

Don't get me wrong, I don't think these proposals will kill kittens or anything, it's just that every addition has a cost.  I (apparently) see that cost as being higher than others do, therefore I feel the small benefit is swamped by the cost.

I agree with Benji that adding new fields has a cost and should be avoided. Just because we can add structure doesn't mean we should.

In the specific case of Repository-URL and Bug-Tracker-URL, the idea was that PyPI would change its behavior if these fields were present (by not using the rating system). So I'm okay with those. But for other fields, like Repository-Checkout and Change-Log-URL, I haven't seen a use case yet.

Ok so, if everyone agrees on those two, I am proposing to add for now : - Repository-Browse-URL : url to a web-browseable repository (no distinction whether its the trunk, a branch etc) - Bug-Tracker-URL : url to the web tracker and skip the others, as they seem controversial. I'll post the proposal at Catalog-SIG as well, Tarek

Floris Bruynooghe writes:

I'm still uneasy about forcing everyone to have a browsable URL

I think the intention is that all of these new fields are optional. No-one would be forced to put any of them into their package. -- \ “Every sentence I utter must be understood not as an | `\ affirmation, but as a question.” —Niels Bohr | _o__) | Ben Finney