From richardjones at optushome.com.au Thu Apr 20 11:35:42 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Thu, 20 Apr 2006 19:35:42 +1000
Subject: [Catalog-sig] PyPI enhancement wish items?
Message-ID: <200604201935.42426.richardjones@optushome.com.au>
Please fire away. I may have some time to work on this in the near future.
Current stuff I know about:
- PEP for metadata 1.2
http://www.python.org/peps/pep-0345.html
(though this is not finished nor has it been commented on)
- incorporate Ian Bicking's XML-RPC API
http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html
- auto-generate download_urls for package uploads
- command-line tool to query pypi and fetch entries (is this necessary
given easy_install?)
- DOAP support
- better searching
- tooltips for field labels
- change notification emails
- admin interface for user details editing
Richard
From ianb at colorstudy.com Thu Apr 20 18:05:14 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 20 Apr 2006 11:05:14 -0500
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <200604201935.42426.richardjones@optushome.com.au>
References: <200604201935.42426.richardjones@optushome.com.au>
Message-ID: <4447B13A.1020500@colorstudy.com>
Richard Jones wrote:
> Please fire away. I may have some time to work on this in the near future.
> Current stuff I know about:
>
> - PEP for metadata 1.2
> http://www.python.org/peps/pep-0345.html
> (though this is not finished nor has it been commented on)
The most notable addition seems to be Requires. Would this be read from
install_requires? It seems like this has to go in Setuptools (to
generate the PKG-INFO) as well as PyPI (to consume it).
Setuptools has the extra features, which could also be shown, probably
more as Recommended than as Required.
Setuptools doesn't allow for requirements like C libraries, or Python
versions. It would be nice if it did (at least the Python versions). I
don't know if Requires in the metadata will include those kinds of
requirements?
> - incorporate Ian Bicking's XML-RPC API
> http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html
> - auto-generate download_urls for package uploads
> - command-line tool to query pypi and fetch entries (is this necessary
> given easy_install?)
I think it's still useful; easy_install only works if you know what you
are looking for.
Maybe it could be integrated into easy_install... but it might be best
if it was a stand-alone script for now, and maybe integrate into nest later.
Better keyword searching support would be good too, especially through
XML-RPC or some RESTish interface. E.g.:
/pypi/packages/wsgi+application
Gives you a list of everything with the wsgi and application keywords.
I think Kevin worked around this, but he was having problems searching
for multiple keywords as well.
> - DOAP support
> - better searching
+2! A single search box on the front page would be most appreciated.
> - tooltips for field labels
> - change notification emails
> - admin interface for user details editing
Comments might be nice, but sadly require maintenance, for spam and
content. A more constrained thing might be links to other packages, so
you could see what else is
Some other things:
Download tracking would be nice. I know at least one person has been
reluctant to move all his packages to the Cheese Shop because it didn't
count downloads and SF did. And I hate SF downloading so so so much.
Changing the template to match the new look would be nice, of course.
I look forward to the improvements!
--
Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
From pje at telecommunity.com Thu Apr 20 18:49:14 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 20 Apr 2006 12:49:14 -0400
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <200604201935.42426.richardjones@optushome.com.au>
Message-ID: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com>
At 07:35 PM 4/20/2006 +1000, Richard Jones wrote:
>Please fire away. I may have some time to work on this in the near future.
>Current stuff I know about:
>
>- PEP for metadata 1.2
> http://www.python.org/peps/pep-0345.html
> (though this is not finished nor has it been commented on)
I think that PEP needs to go back to the drawing board; it emphasizes too
much the syntax of fields, while leaving their semantics entirely
void. For example, it dictates what format projects may specify version
numbers in, using a scheme that's too strict for even Python itself to
comply with. But then it leaves the meaning of requirements to be
community-defined. There's little point in having machine-readable data
with no machine-comprehensible semantics. That is, why make people force
fit stuff into a form that machines will never use anyway?
>- auto-generate download_urls for package uploads
This seems unnecessary to me, since easy_install reads the upload links
just fine where they are, but of course I'm biased. :)
>- command-line tool to query pypi and fetch entries (is this necessary
> given easy_install?)
A stable, documented interface to perform the operations that easy_install
does now via screen-scraping and URL interpretation would be useful.
The other things I'd add to the list are (decreasing priority):
1) the ability to treat project names and versions as case-insensitive,
while removing extraneous characters (as in pkg_resources.safe_name()) for
purposes both of searching and determining name uniqueness when registering.
2) Compute "cheesecake" scores for modified entries (using only the metrics
that don't actually run any of the package's code, of course) and display
them prominently. :)
3) Provide better explanation as to what to put in the fields: encourage
people distributing via Sourceforge to put their showfiles.php URL in as
the download URL, or any other page that has actual clickable links to
download files.
4) More strongly encourage people to use "setup.py register", by having the
web interface generate a setup.py containing the information they filled
in, and suggesting that they use it, while making it hard to find the
button that will go ahead and put the data in from the web. This can be
billed as a new convenience feature to automatically generate a setup.py
and help people improve their cheesecake scores. ;-)
From grig at gheorghiu.net Thu Apr 20 19:00:34 2006
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Thu, 20 Apr 2006 10:00:34 -0700 (PDT)
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com>
Message-ID: <20060420170034.4957.qmail@web54512.mail.yahoo.com>
--- "Phillip J. Eby" <pje at telecommunity.com> wrote:
> 2) Compute "cheesecake" scores for modified entries (using only the
> metrics
> that don't actually run any of the package's code, of course) and
> display
> them prominently. :)
Cheesecake! Cheesecake! I'm all for that! :-)
I've kind of neglected the cheesecake project in the last couple of
months, but if you guys express interest in it, I'm way more motivated
to dust it off and improve it...
I was actually thinking that cheesecake will never take off and gain
mind share unless I actually start displaying its scores for the PyPI
packages in a very prominent way :-)
Grig
From martin at v.loewis.de Thu Apr 20 21:54:18 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 20 Apr 2006 21:54:18 +0200
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <200604201935.42426.richardjones@optushome.com.au>
References: <200604201935.42426.richardjones@optushome.com.au>
Message-ID: <4447E6EA.3050105@v.loewis.de>
Richard Jones wrote:
> Please fire away. I may have some time to work on this in the near future.
> Current stuff I know about:
- Rating system for packages
- procedures for "taking over" entries should the original owner of the
entry go away.
Regards,
Martin
From ianb at colorstudy.com Thu Apr 20 22:07:21 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 20 Apr 2006 15:07:21 -0500
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com>
References: <5.1.1.6.0.20060420123202.01e21bc0@mail.telecommunity.com>
Message-ID: <4447E9F9.7040306@colorstudy.com>
Phillip J. Eby wrote:
>>- auto-generate download_urls for package uploads
>
>
> This seems unnecessary to me, since easy_install reads the upload links
> just fine where they are, but of course I'm biased. :)
It might be nice if PyPI can figure out SF download links. Then if SF
changes its format again PyPI can be updated and easy_install won't have
to be. It's a lot easier to update the one and only PyPI than every
easy_install.
>>- command-line tool to query pypi and fetch entries (is this necessary
>> given easy_install?)
>
>
> A stable, documented interface to perform the operations that easy_install
> does now via screen-scraping and URL interpretation would be useful.
Would that be the XML-RPC interface?
> The other things I'd add to the list are (decreasing priority):
>
> 1) the ability to treat project names and versions as case-insensitive,
> while removing extraneous characters (as in pkg_resources.safe_name()) for
> purposes both of searching and determining name uniqueness when registering.
I'm guessing that means: if someone goes to /pypi/foo and the real name
is Foo, then redirect them to /pypi/Foo.
> 2) Compute "cheesecake" scores for modified entries (using only the metrics
> that don't actually run any of the package's code, of course) and display
> them prominently. :)
Or alternately, have a way of uploading scores (from a trusted source),
and then the scores can be calculated offline on a less important
server. This will make it easier to develop Cheesecake too, since it
can be updated without updating PyPI.
I'm guessing the Cheesecake results can be summarized by a single score
and then some human-readable HTML describing the score.
--
Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
From richardjones at optushome.com.au Fri Apr 21 00:26:27 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 21 Apr 2006 08:26:27 +1000
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <4447B13A.1020500@colorstudy.com>
References: <200604201935.42426.richardjones@optushome.com.au>
<4447B13A.1020500@colorstudy.com>
Message-ID: <200604210826.27224.richardjones@optushome.com.au>
On Friday 21 April 2006 02:05, Ian Bicking wrote:
> Download tracking would be nice. I know at least one person has been
> reluctant to move all his packages to the Cheese Shop because it didn't
> count downloads and SF did. And I hate SF downloading so so so much.
This is in place now. For example, see
http://cheeseshop.python.org/pypi/roundup
Richard
From richardjones at optushome.com.au Fri Apr 21 00:51:59 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Fri, 21 Apr 2006 08:51:59 +1000
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <200604201935.42426.richardjones@optushome.com.au>
References: <200604201935.42426.richardjones@optushome.com.au>
Message-ID: <200604210851.59678.richardjones@optushome.com.au>
On Thursday 20 April 2006 19:35, Richard Jones wrote:
> - incorporate Ian Bicking's XML-RPC API
> http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html
*cough* of course, the TODO list I was looking at is out of date, as this is
already done :)
>>> import xmlrpclib
>>> server = xmlrpclib.ServerProxy('http://cheeseshop.python.org/pypi')
>>> server.package_releases('roundup')
['1.1.1']
>>> server.package_urls('roundup', '1.1.1')
[{'url':
'http://cheeseshop.python.org/packages/any/r/roundup/roundup-1.1.1.win32.exe',
'packagetype': 'bdist_wininst'}, {'url':
'http://cheeseshop.python.org/packages/source/r/roundup/roundup-1.1.1.tar.gz',
'packagetype': 'sdist'}]
Richard
From pje at telecommunity.com Fri Apr 21 01:53:16 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 20 Apr 2006 19:53:16 -0400
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <200604210851.59678.richardjones@optushome.com.au>
References: <200604201935.42426.richardjones@optushome.com.au>
<200604201935.42426.richardjones@optushome.com.au>
Message-ID: <5.1.1.6.0.20060420194753.041682b0@mail.telecommunity.com>
At 08:51 AM 4/21/2006 +1000, Richard Jones wrote:
>On Thursday 20 April 2006 19:35, Richard Jones wrote:
> > - incorporate Ian Bicking's XML-RPC API
> > http://mail.python.org/pipermail/catalog-sig/2005-May/000634.html
>
>*cough* of course, the TODO list I was looking at is out of date, as this is
>already done :)
>
> >>> import xmlrpclib
> >>> server = xmlrpclib.ServerProxy('http://cheeseshop.python.org/pypi')
> >>> server.package_releases('roundup')
>['1.1.1']
> >>> server.package_urls('roundup', '1.1.1')
>[{'url':
>'http://cheeseshop.python.org/packages/any/r/roundup/roundup-1.1.1.win32.exe',
>
>'packagetype': 'bdist_wininst'}, {'url':
>'http://cheeseshop.python.org/packages/source/r/roundup/roundup-1.1.1.tar.gz',
>
>'packagetype': 'sdist'}]
Sweet! It's fast, too, especially on search. I'll definitely upgrade
easy_install to start using this, although alas, not today.
From pje at telecommunity.com Sat Apr 22 00:22:15 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 21 Apr 2006 18:22:15 -0400
Subject: [Catalog-sig] Reject or update PEP 243?
Message-ID: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com>
I just noticed that the 2.5 What's New references PEP 243 in relation to
the distutils' "upload" command, but in fact the upload command implements
a different mechanism entirely. About the only thing the PEP and
implementation have in common is that they do a POST to python.org/pypi and
have a protocol_version field of "1". Nothing else is the same.
Perhaps someone who knows how the actual implementation works could rewrite
the PEP, or write another one?
From richardjones at optushome.com.au Sat Apr 22 03:09:27 2006
From: richardjones at optushome.com.au (Richard Jones)
Date: Sat, 22 Apr 2006 11:09:27 +1000
Subject: [Catalog-sig] Reject or update PEP 243?
In-Reply-To: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com>
References: <5.1.1.6.0.20060421181756.01e3bc00@mail.telecommunity.com>
Message-ID: <200604221109.27540.richardjones@optushome.com.au>
On Saturday 22 April 2006 08:22, Phillip J. Eby wrote:
> I just noticed that the 2.5 What's New references PEP 243 in relation to
> the distutils' "upload" command, but in fact the upload command implements
> a different mechanism entirely. About the only thing the PEP and
> implementation have in common is that they do a POST to python.org/pypi and
> have a protocol_version field of "1". Nothing else is the same.
It'll probably be easier to just rewrite the PEP. I'm not entirely sure I know
how the inner workings of the upload command work (how distutils does command
chaining, for example) but I could update the protocol if I found the time
for it.
Richard
From pje at telecommunity.com Sat Apr 22 18:04:43 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 22 Apr 2006 12:04:43 -0400
Subject: [Catalog-sig] [Python-Dev] setuptools: past, present, future
In-Reply-To: <e2d0rs$r30$1@sea.gmane.org>
References: <5.1.1.6.0.20060421134259.0419f318@mail.telecommunity.com>
<e2cb2p$f1s$1@sea.gmane.org> <e2cv9t$n24$1@sea.gmane.org>
<ca471dc20604220324n31f20ea5o36fc8d3f9d59d30@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060422115700.0448f138@mail.telecommunity.com>
At 12:34 PM 4/22/2006 +0200, Fredrik Lundh wrote:
>Guido van Rossum wrote:
>
> > Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be
> > diverse -- yes, I've heard people say that this is something that
> > Python sorely lacks; but I've also heard from more than one person
> > that CPAN sucks from a quality perspective. So I think we shouldn't
> > focus on emulating CPAN; rather, we should solve the problems we
> > actually have.
>
>the first problem seems to be to define what those problems really
>are ;-)
>
>(as for the CPAN quality, any public repository will end up being full
>of crap; I don't see any way to work around that. automatic scoring
>based on superficial aspects
The purpose of automated scoring on superficial aspects isn't so much to
ensure quality as it is to ensure *accessibility*, both in the sense of
being able to install the thing, and meet some basic levels of having
documentation.
If something is accessible and trivial to install, then the market can
decide which packages are better to actually use.
>or ratings by small numbers of anonymous
>visitors are probably among the worst ways to distinguish crap from
>good stuff; for quality, you need initiatives like
>
> http://code.enthought.com/enthon/
>
>and other "fat python" projects.
Actually, every project that uses other projects' code can now be a "chubby
python" by expressing dependencies. Really, one of the best ratings of a
package's quality (or at least popularity) is going to be how many other
projects depend on it. If everybody uploaded eggs to the Cheeseshop, it'd
be possible to show links to "projects that use this project's code" by
reading the dependency metadata with pkg_resources. (Not to mention
"projects that this project uses").
From grig at gheorghiu.net Sat Apr 22 18:49:15 2006
From: grig at gheorghiu.net (Grig Gheorghiu)
Date: Sat, 22 Apr 2006 09:49:15 -0700 (PDT)
Subject: [Catalog-sig] PyPI enhancement wish items?
In-Reply-To: <20060420170034.4957.qmail@web54512.mail.yahoo.com>
Message-ID: <20060422164915.63521.qmail@web54512.mail.yahoo.com>
Richard,
Do you think it would be possible to start some integration of my
Cheesecake project into PyPI? Here's the homepage of the project:
http://pycheesecake.org/
For a given package, I'm computing scores for installability,
documentation and code 'kwalitee'. Details are on the homepage.
I'd love to see what other things people think would be worth
measuring.
Grig
--- Grig Gheorghiu <grig at gheorghiu.net> wrote:
> --- "Phillip J. Eby" <pje at telecommunity.com> wrote:
>
> > 2) Compute "cheesecake" scores for modified entries (using only the
> > metrics
> > that don't actually run any of the package's code, of course) and
> > display
> > them prominently. :)
>
> Cheesecake! Cheesecake! I'm all for that! :-)
>
> I've kind of neglected the cheesecake project in the last couple of
> months, but if you guys express interest in it, I'm way more
> motivated
> to dust it off and improve it...
>
> I was actually thinking that cheesecake will never take off and gain
> mind share unless I actually start displaying its scores for the PyPI
> packages in a very prominent way :-)
>
> Grig
>
> _______________________________________________
> Catalog-sig mailing list
> Catalog-sig at python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>