From amk@amk.ca Sun Oct 13 21:40:29 2002
From: amk@amk.ca (A.M. Kuchling)
Date: Sun, 13 Oct 2002 16:40:29 -0400
Subject: [Catalog-sig] Withdrawing PEP 262
Message-ID: <20021013164029.A26682@nyman.amk.ca>
PEP 262 is a proposal for a database of installed Python packages. I
wrote it as a step toward implementing automatic installation of
Python software. After playing with Fink on MacOS X a bit, I've
decided to withdraw the PEP, because I now think that Python shouldn't
invent its own database; package handling and dependency tracking
should be the operating system's job.
Accordingly, I'm marking PEP 262 as 'Status: Deferred', and will
request that it be removed from consideration for Python 2.3. If
someone else wants to work on it, feel free to take over the PEP and
resurrect it.
--amk
From cliechti@gmx.net Sun Oct 13 22:55:56 2002
From: cliechti@gmx.net (Chris Liechti)
Date: Sun, 13 Oct 2002 23:55:56 +0200
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <20021013164029.A26682@nyman.amk.ca>
Message-ID: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock>
Am 13.10.2002 22:40:29, schrieb "A.M. Kuchling" <amk@amk.ca>:
>PEP 262 is a proposal for a database of installed Python packages. I
>wrote it as a step toward implementing automatic installation of
>Python software. After playing with Fink on MacOS X a bit, I've
>decided to withdraw the PEP, because I now think that Python shouldn't
>invent its own database; package handling and dependency tracking
>should be the operating system's job.
i use Debian Linux, which has a great package management system too
and it's very nice when all the dependencies are installed along with
a python extension. i understand you're exitement.
but... not all systems have a package management system, e.g. on
windows rules chaos. what can we do there?
and there are some many different packaging systems...
the problem of different package management systems can be partly
soved by distutils. there are some packaging types already there (but
i miss deb). the other problem is that a extension creator need to make
all those formats and provide the downloads.
and how about dependecies to onther packages. it's impossible
for an author to cope with all of them for all different systems.
(simplest to do will be to create a package with no dependecies, that way
a bit confort goes away but its more motivating for the maintainer to create
different package downloads, so that they're at least there.)
i admit that i'm less interested in an installed files database (within python)
rather than in a good way to download modules from a repository.
if there was a browser that lets one select the extensions one wishes and
then just press OK, that would be great. (maybe that could even be called
within a running python application to request the user to install an addon.
run in a separete process to solve the problem with different GUI systems)
i think it would be a pythonic solution when that browser would integrate
with the systems package manager. as a fallback it could still use
"setup.py install" (loosing remove capability but that's not too important as
systems without a package manager have a mess anyway ;-)
that would mean that there must be a repository of modules (at python.org?)
with downloads of zip, rpm, deb, etc access through http and ftp and easy
upload (for a quick growth). and maybe some hierarchical package naming
system should be developed for that.
what do you think of such an approach?
chris
From martin@v.loewis.de Mon Oct 14 05:27:18 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 14 Oct 2002 06:27:18 +0200
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock>
References: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock>
Message-ID: <m3znthy7a1.fsf@mira.informatik.hu-berlin.de>
Chris Liechti <cliechti@gmx.net> writes:
> i use Debian Linux, which has a great package management system too
> and it's very nice when all the dependencies are installed along with
> a python extension. i understand you're exitement.
>
> but... not all systems have a package management system, e.g. on
> windows rules chaos.
That may be true in general, but on Windows, it is not. There is a
database of installed packages, complete with uninstallation etc.
In fact, I'm not aware of any modern system that does not have a
packaging infrastructure.
> i admit that i'm less interested in an installed files database
> (within python) rather than in a good way to download modules from a
> repository.
That's an entire different issue. The catalog system, which causes
this SIG's existence, is still an open problem.
> i think it would be a pythonic solution when that browser would integrate
> with the systems package manager.
I agree, and I think this is Andrew's rationale for withdrawing the
PEP.
Regards,
Martin
From DavidA@ActiveState.com Mon Oct 14 05:52:21 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Sun, 13 Oct 2002 21:52:21 -0700
Subject: [Catalog-sig] Withdrawing PEP 262
References: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock> <m3znthy7a1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3DAA4D85.7010000@ActiveState.com>
Martin v. Loewis wrote:
>Chris Liechti <cliechti@gmx.net> writes:
>
>
>
>>i use Debian Linux, which has a great package management system too
>>and it's very nice when all the dependencies are installed along with
>>a python extension. i understand you're exitement.
>>
>>but... not all systems have a package management system, e.g. on
>>windows rules chaos.
>>
>>
>
>That may be true in general, but on Windows, it is not. There is a
>database of installed packages, complete with uninstallation etc.
>
In fact what I think is wrong with both Windows and many other systems'
is that they tend to be 'one-level' systems -- it's hard to define a
"subsystem" installation. It'd be nice if every python module installed
didn't show up in the ever-growing list of programs under Add/Remove
Programs. Or are there subtleties to the Windows Installer that I
haven't heard of?
--david
From thomas.heller@ion-tof.com Mon Oct 14 10:43:09 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 14 Oct 2002 11:43:09 +0200
Subject: [Catalog-sig] Withdrawing PEP 262
References: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock> <m3znthy7a1.fsf@mira.informatik.hu-berlin.de> <3DAA4D85.7010000@ActiveState.com>
Message-ID: <01d401c27366$1adfacc0$e000a8c0@thomasnotebook>
From: "David Ascher" <DavidA@ActiveState.com>
> Martin v. Loewis wrote:
>
> >Chris Liechti <cliechti@gmx.net> writes:
> >
> >
> >
> >>i use Debian Linux, which has a great package management system too
> >>and it's very nice when all the dependencies are installed along with
> >>a python extension. i understand you're exitement.
> >>
> >>but... not all systems have a package management system, e.g. on
> >>windows rules chaos.
> >>
> >>
> >
> >That may be true in general, but on Windows, it is not. There is a
> >database of installed packages, complete with uninstallation etc.
Well, it's not really a database, it is just a listing of files and
registry entries which must be removed at uninstallation time.
> >
> In fact what I think is wrong with both Windows and many other systems'
> is that they tend to be 'one-level' systems -- it's hard to define a
> "subsystem" installation. It'd be nice if every python module installed
> didn't show up in the ever-growing list of programs under Add/Remove
> Programs. Or are there subtleties to the Windows Installer that I
> haven't heard of?
No, currently not.
It would be possible to change this, but is it really worth the effort?
I've heard some complaints that the python root directory is populated
with these xxx-wininst.log and Removexxx.exe files, but why bother?
Thomas
From AASterling@Hotpop.com Mon Oct 14 20:36:42 2002
From: AASterling@Hotpop.com (Aaron Sterling)
Date: Mon, 14 Oct 2002 15:36:42 -0400
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock>
Message-ID: <HYSZWRLA0TPXRH73QLQP94LHNHOIC.3dab1cca@the-pad>
10/13/2002 5:55:56 PM, Chris Liechti <cliechti@gmx.net> wrote:
>i admit that i'm less interested in an installed files database (within python)
>rather than in a good way to download modules from a repository.
That is my view as well, I also agree with you implication that for the second to be done
well the first must be in place.
>if there was a browser that lets one select the extensions one wishes and
>then just press OK, that would be great. (maybe that could even be called
>within a running python application to request the user to install an addon.
>run in a separete process to solve the problem with different GUI systems)
>
>i think it would be a pythonic solution when that browser would integrate
>with the systems package manager. as a fallback it could still use
>"setup.py install" (loosing remove capability but that's not too important as
>systems without a package manager have a mess anyway ;-)
>
>that would mean that there must be a repository of modules (at python.org?)
Perhaps a good way to do it would be to have a table that can be easily downloaded at
python.org. The table would consist of package name in one coloumn and the direct url/s
(suitable for automatic download) of the resource as the other coloumn. If the desired module
is not available from any of the given urls, their could be a repository of the more
frequently used/standard library modules at the site to be downloaded as a last resort.
The table could be included with the standard library and accessed on the website if the
local copy were six months (I pulled it out of the air, it could be any age) old or older. It
would also be accessed when the local copy did not contain the required module. It could then
be stored as the new local copy until a new copy was required according to the specified
material. This scheme would have the advantage of saving bandwidth on the python website.
An alternate way to do it(and a much cooler way of doing it in my book), would be to select a
port on the python.org site and attach a server to it for a very simple protocol that could
be developed and implemented in about half an hour. The server would simply recieve the name
of the desired package and dispatch the url from which it could be downloaded. This would
have the advantage of allowing the server to keep track of requests for packages, thus
allowing usage data to be collected(or not). This could be used to decide which packages to
add to the standard library. Adding the most frequently used packages to the standard library
would have the effect of cutting bandwidth use on the server. Like the first solution, the
table would also be maintained locally and the main table at the website would only be
accessed under conditions similiar to those in the first solution. It also goes without
saying that the code to do these things would be a package so that Large organizations would
be able to run their own package servers. ImportError could even be changed in such a manner
that(at the administrator/users discretion for obvious security reasons) if a required
package was not on the system, it could be downloaded and installed automatically. That would
be cool(Is their a programming system that provides that functionality now?). Security
concerns could be overcome to some degree by having automatic installation download
exclusively from the python.org website.
for either solution new submissions would be added by mailing the name of the package and the
url to a specified mailbox at the website. A simple script could add them on a daily basis.
I think that the first plan is definitely doable, I favor the second(I know my voice carries
no weight). It goes without saying that somebody will think that the second solution is
overkill, but is that the general concensus?
If --any-- intererest whatsoever is expressed in either of these plans, I would be more then
willing to further flesh them out or work with anyone on further fleshing them out, in fact I
am fleshing the second one out at the moment.
Aaron
From cliechti@gmx.net Mon Oct 14 22:21:00 2002
From: cliechti@gmx.net (Chris Liechti)
Date: Mon, 14 Oct 2002 23:21:00 +0200
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <HYSZWRLA0TPXRH73QLQP94LHNHOIC.3dab1cca@the-pad>
Message-ID: <MJGA310OID97ICZTICECFCOM1TB0SO.3dab353c@rock>
Am 14.10.2002 21:36:42, schrieb Aaron Sterling <AASterling@hotpop.com>:
>10/13/2002 5:55:56 PM, Chris Liechti <cliechti@gmx.net> wrote:
>>i admit that i'm less interested in an installed files database (within python)
>>rather than in a good way to download modules from a repository.
>
>That is my view as well, I also agree with you implication that for the second to be done
>well the first must be in place.
>
>>if there was a browser that lets one select the extensions one wishes and
>>then just press OK, that would be great. (maybe that could even be called
>>within a running python application to request the user to install an addon.
>>run in a separete process to solve the problem with different GUI systems)
>>
>>i think it would be a pythonic solution when that browser would integrate
>>with the systems package manager. as a fallback it could still use
>>"setup.py install" (loosing remove capability but that's not too important as
>>systems without a package manager have a mess anyway ;-)
>>
>>that would mean that there must be a repository of modules (at python.org?)
>
>Perhaps a good way to do it would be to have a table that can be easily downloaded at
>python.org. The table would consist of package name in one coloumn and the direct url/s
>(suitable for automatic download) of the resource as the other coloumn. If the desired module
>is not available from any of the given urls, their could be a repository of the more
>frequently used/standard library modules at the site to be downloaded as a last resort.
>The table could be included with the standard library and accessed on the website if the
>local copy were six months (I pulled it out of the air, it could be any age) old or older. It
>would also be accessed when the local copy did not contain the required module. It could then
>be stored as the new local copy until a new copy was required according to the specified
>material. This scheme would have the advantage of saving bandwidth on the python website.
>
>An alternate way to do it(and a much cooler way of doing it in my book), would be to select a
>port on the python.org site and attach a server to it for a very simple protocol that could
>be developed and implemented in about half an hour. The server would simply recieve the name
>of the desired package and dispatch the url from which it could be downloaded.
no no...reason: firewalls. there must be a way to access the list through a http proxy when no other
protocols are passed trough a firewall (i have this situation where i work). because of that i'd say
the priorites should be the following 1. http, 2. ftp, 3. custom protocol,if any.
an another good reason: standards like a web server are easier to use. it's easy to convince
the IT departement that they should open a directory for the module repository while it is likely
to be very difficult to convince them to run a special server app.
look how Debian is working. there is a Packages.tgz on the server with a list of the available stuff.
and it can be accessed throught htpp and ftp, realy easy and just working.
> This would
>have the advantage of allowing the server to keep track of requests for packages, thus
>allowing usage data to be collected(or not). This could be used to decide which packages to
>add to the standard library. Adding the most frequently used packages to the standard library
>would have the effect of cutting bandwidth use on the server. Like the first solution, the
>table would also be maintained locally and the main table at the website would only be
>accessed under conditions similiar to those in the first solution. It also goes without
>saying that the code to do these things would be a package so that Large organizations would
>be able to run their own package servers. ImportError could even be changed in such a manner
>that(at the administrator/users discretion for obvious security reasons) if a required
>package was not on the system, it could be downloaded and installed automatically. That would
>be cool(Is their a programming system that provides that functionality now?). Security
>concerns could be overcome to some degree by having automatic installation download
>exclusively from the python.org website.
sure that'd be cool, but that should be only allowed with signed packages and otherwise only
with a BIG warning box...
>for either solution new submissions would be added by mailing the name of the package and the
>url to a specified mailbox at the website. A simple script could add them on a daily basis.
i think there should be two categories. one trusted, where the submissions must be signed and
go through some process where different people can look at it and a second easy to upload
repository, so that the number of packagaes can quickly grow. the quality of the submissions will
be lower in the second one but there might be some pearls...
>I think that the first plan is definitely doable, I favor the second(I know my voice carries
>no weight). It goes without saying that somebody will think that the second solution is
>overkill, but is that the general concensus?
>
>If --any-- intererest whatsoever is expressed in either of these plans, I would be more then
>willing to further flesh them out or work with anyone on further fleshing them out, in fact I
>am fleshing the second one out at the moment.
From AASterling@Hotpop.com Tue Oct 15 05:45:14 2002
From: AASterling@Hotpop.com (Aaron Sterling)
Date: Tue, 15 Oct 2002 00:45:14 -0400
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <MJGA310OID97ICZTICECFCOM1TB0SO.3dab353c@rock>
Message-ID: <FADCO7DC85ZXJGUD95C74XLIVQOK.3dab9d5a@the-pad>
10/14/2002 5:21:00 PM, Chris Liechti <cliechti@gmx.net> wrote:
>no no...reason: firewalls.
>sure that'd be cool, but that should be only allowed with signed packages and
otherwise only
>with a BIG warning box...
Oh...Yeah... Man the real world is a drag. Especially the security aspect of it.
That being said, let me list some rough specifications for such a system(the sane one)
and see if anyone agrees with any of them.
1) As Chris pointed out, the catalog should be divided into two levels, signed and
unsigned.
2) The catalog(I propose calling it the Python Package Catalog, or PPC, at least for
the purpose of this discussion) should be structured thusly. Three rfc822 fields: DATE,
PPC-Signed, PPC-Unsigned. This divides the catalog into two subcatalogs. Each entry in
a sub-catalog occupies one and only one line. This line takes the form <package name>
<SPACE> <package infofile URL> <SPACE> <package website> <SPACE> <package summary>. The
package name is the name of the package, the package infofile URL is the URL of the
package infofile(described in 4), the package website is the packages' website. The
package summary is an eighty charachter summary of the package. This is for use in the
browser.
3) Access to the primary PPC at python.org should be kept to a minimum to conserve
bandwidth. This can be facilitated by keeping a local copy of the PPC on each machine.
This carries with the extra benefit of redundency. When it is time to download a
package, the Local PPC(LPPC) is consulted. If the DATE field in the LPPC is expired a
fresh copy is downloaded from python.org.
4) This is a hack I think. It would be preferable to simply have one universal format
for all packages on all platforms, that not being the case, we need some means of
redirecting general requests for a package into a concrete URL for that specific
platform. This can be accomplished with infofiles. Infofiles would be plaintext, either
ASCII or LATIN-1. They would essentially be a catalog of the different distributions of
the package for different platforms. Each platform for which that package is supported
would have one and only one line given to it. This line would take the form <platform>
<SPACE> <platform specific URL>. The platform is simply sys.platform so that the code
is guaranteed to know what it is. The plaform specific URL is the url of the package
for that specific platform.
5) In addition to the LPPC, there should be the IPPC (Installed PPC), this is almost
the same format as the PPC, but contains only the subset of the PPC that is installed
on the local system. The change in format is the substitution of a reference to the
packages' local package database entry for the infofile reference.
6) The stucture of the local package database would be essentially the same as under
PEP 262
There is obviously a lot of stuff that needs to be fleshed out, such as where do these
files live and the UI for the Package Browser, to give just two examples. As a whole
though, how does it hold up?
Aaron
From lucio@movilogic.com Tue Oct 15 14:24:13 2002
From: lucio@movilogic.com (Lucio Torre)
Date: Tue, 15 Oct 2002 10:24:13 -0300
Subject: [Catalog-sig] Withdrawing PEP 262
References: <FADCO7DC85ZXJGUD95C74XLIVQOK.3dab9d5a@the-pad>
Message-ID: <3DAC16FD.8030506@movilogic.com>
Aaron,
i a bit lost in time, i hope this is still relevant. comments below.
Aaron Sterling wrote:
>10/14/2002 5:21:00 PM, Chris Liechti <cliechti@gmx.net> wrote:
>
>
>
>>no no...reason: firewalls.
>>
>>
>
>
>
>>sure that'd be cool, but that should be only allowed with signed packages and
>>
>>
>otherwise only
>
>
>>with a BIG warning box...
>>
>>
>
>Oh...Yeah... Man the real world is a drag. Especially the security aspect of it.
>
>That being said, let me list some rough specifications for such a system(the sane one)
>and see if anyone agrees with any of them.
>
>
I can testify for the imposrtance of this issue!.. On the other hand,
simplicity should also be considered a plus.
>
>1) As Chris pointed out, the catalog should be divided into two levels, signed and
>unsigned.
>
>2) The catalog(I propose calling it the Python Package Catalog, or PPC, at least for
>the purpose of this discussion) should be structured thusly. Three rfc822 fields: DATE,
>PPC-Signed, PPC-Unsigned. This divides the catalog into two subcatalogs. Each entry in
>a sub-catalog occupies one and only one line. This line takes the form <package name>
><SPACE> <package infofile URL> <SPACE> <package website> <SPACE> <package summary>. The
>package name is the name of the package, the package infofile URL is the URL of the
>package infofile(described in 4), the package website is the packages' website. The
>package summary is an eighty charachter summary of the package. This is for use in the
>browser.
>
>
Im sure i lost something, but where are dependencies? What about when
you want to install this you should have that installed first?
Thats a great feature.
Also, being a debian fan, i recommend allowing each instalation to
configure it sources. Python.org could provide a basic one, parnassus
another. something like that.
i dont find cerificates too important. i dont really know the authors of
the code i download, and i dont think python.org should tell anyone
about the security/autenticity of my code.
>3) Access to the primary PPC at python.org should be kept to a minimum to conserve
>bandwidth. This can be facilitated by keeping a local copy of the PPC on each machine.
>This carries with the extra benefit of redundency. When it is time to download a
>package, the Local PPC(LPPC) is consulted. If the DATE field in the LPPC is expired a
>fresh copy is downloaded from python.org.
>
I like the 'apt-get update' way better. Why force update? New versions
and incompatibilities always arise that should be considered.
>
>4) This is a hack I think. It would be preferable to simply have one universal format
>for all packages on all platforms, that not being the case, we need some means of
>redirecting general requests for a package into a concrete URL for that specific
>platform. This can be accomplished with infofiles. Infofiles would be plaintext, either
>ASCII or LATIN-1. They would essentially be a catalog of the different distributions of
>the package for different platforms. Each platform for which that package is supported
>would have one and only one line given to it. This line would take the form <platform>
><SPACE> <platform specific URL>. The platform is simply sys.platform so that the code
>is guaranteed to know what it is. The plaform specific URL is the url of the package
>for that specific platform.
>
>
Here we also have issues with python versions. What if im still using
python 2.7 and python 3000 is out? I should be able to select the
packages that are compatible with python 2.7
>5) In addition to the LPPC, there should be the IPPC (Installed PPC), this is almost
>the same format as the PPC, but contains only the subset of the PPC that is installed
>on the local system. The change in format is the substitution of a reference to the
>packages' local package database entry for the infofile reference.
>
>6) The stucture of the local package database would be essentially the same as under
>PEP 262
>
>There is obviously a lot of stuff that needs to be fleshed out, such as where do these
>files live and the UI for the Package Browser, to give just two examples. As a whole
>though, how does it hold up?
>
>
The way i see it is like this: (sorry for the extra noise in this touchy
issue)
$INSTALLDB/Packages - A Database of Installed Python Packages (PEP 262)
$INSTALLDB/Config/Sources.py - A sources list. Something like this:
sources = {
"python.org":{
"url":"http://python.org/",
"desc":"Default source",
}
"parnasuss": {
etc...
}
}
As you can see, im implying the use of python source as the format of
the configuration files. I think its better because:
a- its more felixible
b- we already have a parser
c- everyone knows how to read it
d- security issues with "pakcage info files" can be solved easily
$INSTALLDB/Cache/$SOURCE.package.list
List of packages. This file should be very similar to the sources.py
file, and include the requiered information of a packe. I
$INSTALLDB/Keys/*
Public keys of people you trust.
And usage should go like this:
dsitutils update
check every source for updates. perform a head on every url,
download new versions.
distutils install package [-s SOURCE]
look for the package in cache. if its found in more than one source,
complain and stop. require usage of 'source'
check prerequisites. in they are not met, say so and stop.
download. (http)
check for signatures.
install.
etc.
I think that we should keep these goals in mind:
1- No automation/intelligence until we are sure it can be done right. I
dont want automatic dependency matching and endeless loops like dselect.
2- it should not force us to keep an up to date installation of python,
but help us in doing so.
3- it should be easily wrapped. So i can use debian to keep my package
database or make a sample inno setup tool for the same thing.
just my 2c
lucio
>
>
From thomas.heller@ion-tof.com Wed Oct 16 18:43:40 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 16 Oct 2002 19:43:40 +0200
Subject: [Catalog-sig] Withdrawing PEP 262
In-Reply-To: <3DAA4D85.7010000@ActiveState.com>
References: <BA375KFD62XVPMEC322YYU63HCHGZY.3da9ebec@rock>
<m3znthy7a1.fsf@mira.informatik.hu-berlin.de>
<3DAA4D85.7010000@ActiveState.com>
Message-ID: <znte8ek3.fsf@ion-tof.com>
David Ascher <DavidA@ActiveState.com> writes:
> Martin v. Loewis wrote:
>
> >Chris Liechti <cliechti@gmx.net> writes:
> >
> >
>
> >>i use Debian Linux, which has a great package management system too
> >>and it's very nice when all the dependencies are installed along with
> >>a python extension. i understand you're exitement.
> >>
> >>but... not all systems have a package management system, e.g. on
> >>windows rules chaos.
> >>
>
> >
> >That may be true in general, but on Windows, it is not. There is a
> >database of installed packages, complete with uninstallation etc.
> >
I was wondering if it would make sense (now that PEP262 is no longer),
to convert these *-wininst.log files somewhat more into package
database components.
Currently these files contain a list of files created, directories
created, registry keys and values added to the system by the
installation program.
All this is used to cleanly remove the package from the system if
it is no longer needed.
Other info in these files is:
- the windows installer exe pathname used to install the package
- the name (including the version) of the package installed, for
example Numeric-20.3
- the command line for removing this package (used by the
add/remove programs Windows applet.
This information could be used also by a python program to verify
that all files are still present, to display the name and version,
to remove the package, and maybe more.
To extend the possibilities, would it make sense to include more
information into these files, starting with the metadata info
for the package at least, see PEP 242.
> In fact what I think is wrong with both Windows and many other
> systems' is that they tend to be 'one-level' systems -- it's hard to
> define a "subsystem" installation. It'd be nice if every python
> module installed didn't show up in the ever-growing list of programs
> under Add/Remove Programs. Or are there subtleties to the Windows
> Installer that I haven't heard of?
>
Given the above, it would be easy to write a slick command line or gui
python app with a nice user interface for displaying, removing,
verifying, and maybe sometime downloading and installing packages.
Thomas
From rjones@ekit-inc.com Tue Oct 22 08:32:45 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Tue, 22 Oct 2002 17:32:45 +1000
Subject: [Catalog-sig] Distutils package index online
Message-ID: <200210221732.45600.rjones@ekit-inc.com>
So I bit the bullet on the train this morning (what an odd phrase.... ;)
I have written a simple CGI and distutils extension that handles registration
of distutils package metadata in a central index. The intention is that this
system is run on python.org, mostly to lend it an air of officiality ;)
It's simple, it's bare-bones, it works:
http://mechanicalcat.net:8081/
I just figured we needed _something_ to get out there to start the ball
rolling. And I'm serious about the python.org part...
Richard
From thomas.heller@ion-tof.com Tue Oct 22 13:47:34 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 22 Oct 2002 14:47:34 +0200
Subject: [Catalog-sig] RFC: pypan - a Python package manager
Message-ID: <iszuod21.fsf@ion-tof.com>
I've hacked together a simple Python Package manager.
It is available here:
<http://starship.python.net/crew/theller/pypan/>
Suggestions for a better name are welcome...
It is only tested with Python 2.2, but it should also run under 2.0
and 2.1.
Currently works only under Windows, but I hope someone will join me
and extend it to linux.
For information, please look at the web page.
---------------------
Here are the general concepts (which are mostly derived from ciphon):
pypan can list the packages installed on the system (those installed
from bdist_wininst packages). It does not maintain a database of
installed packages, the list is created at runtime by using the
information from the install log files (I assume Linux package formats
like .rpm or .deb could provide the same information). See the 'list'
command.
The package repository is a simple directory tree on the server,
although pypan can also use a local directory tree.
At the top of the directory structure is a 'packages.xml' file which
lists the contained files together with their PKG-INFO. pypan can
handle source distributions built by distutils (.zip, .tar.gz), as
well as binary windows distributions built with bdist_wininst.
The 'packages.xml' file is built by a 'build_packagelist.py' script
which is included.
If a source distribution is installed, a windows installer is built on
the fly and run. Unfortunately, these windows installers can not yet
run in 'silent' mode, but I plan to add this later.
For your convenience I've uploaded (additionally to pypan itself)
several PyChecker distributions to the repository.
This is alpha software - please take care when using it.
Of course I am *much* interested in any feedback, bug reports, flames...
Regards,
Thomas
From rjones@ekit-inc.com Wed Oct 23 02:09:55 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Wed, 23 Oct 2002 11:09:55 +1000
Subject: [Catalog-sig] Trove information
Message-ID: <200210231109.55429.rjones@ekit-inc.com>
So why didn't Trove information make it into the distutils metadata? It came
up numerous times on the mailing list, but didn't actually make it into the
metadata set...
Richard
From martin@v.loewis.de Wed Oct 23 07:13:18 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 23 Oct 2002 08:13:18 +0200
Subject: [Catalog-sig] Trove information
In-Reply-To: <200210231109.55429.rjones@ekit-inc.com>
References: <200210231109.55429.rjones@ekit-inc.com>
Message-ID: <m3znt565tt.fsf@mira.informatik.hu-berlin.de>
Richard Jones <rjones@ekit-inc.com> writes:
> So why didn't Trove information make it into the distutils metadata? It came
> up numerous times on the mailing list, but didn't actually make it into the
> metadata set...
Either because there was no agreement, or because nobody ever
contributed patches (probably both).
Regards,
Martin
From rjones@ekit-inc.com Wed Oct 23 07:24:06 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Wed, 23 Oct 2002 16:24:06 +1000
Subject: [Catalog-sig] Online catalog now includes basic searching
Message-ID: <200210231624.06412.rjones@ekit-inc.com>
Again, proof-of-concept rather than fully-featured, but the online distutils
catalog now has searching.
That address again :)
http://mechanicalcat.net:8081/
Richard
From rjones@ekit-inc.com Wed Oct 23 07:24:44 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Wed, 23 Oct 2002 16:24:44 +1000
Subject: [Catalog-sig] Trove information
In-Reply-To: <m3znt565tt.fsf@mira.informatik.hu-berlin.de>
References: <200210231109.55429.rjones@ekit-inc.com> <m3znt565tt.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210231624.44848.rjones@ekit-inc.com>
On Wed, 23 Oct 2002 4:13 pm, Martin v. Loewis wrote:
> Richard Jones <rjones@ekit-inc.com> writes:
> > So why didn't Trove information make it into the distutils metadata? It
> > came up numerous times on the mailing list, but didn't actually make it
> > into the metadata set...
>
> Either because there was no agreement, or because nobody ever
> contributed patches (probably both).
I've asked the same question on the distutils mailing list (where I should
have asked in the first place) and will see.
Richard
From s-thapa-11@alumni.uchicago.edu Fri Oct 25 20:21:22 2002
From: s-thapa-11@alumni.uchicago.edu (Suchandra Thapa)
Date: 25 Oct 2002 14:21:22 -0500
Subject: [Catalog-sig] RFC: pypan - a Python package manager
In-Reply-To: <7kg64dfk.fsf@ion-tof.com>
References: <iszuod21.fsf@ion-tof.com> <1035306997.23711.63.camel@hepcat>
<ptu25q5v.fsf@ion-tof.com> <1035309484.23711.84.camel@hepcat>
<wuo64yq2.fsf@ion-tof.com> <1035565579.1571.61.camel@hepcat>
<7kg64dfk.fsf@ion-tof.com>
Message-ID: <1035573683.1575.92.camel@hepcat>
--=-nDsWAenhgWjtl1pvJF/N
Content-Type: multipart/alternative; boundary="=-4xzlh8jQ4jc2Y2Sk5TQt"
--=-4xzlh8jQ4jc2Y2Sk5TQt
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
On Fri, 2002-10-25 at 12:48, Thomas Heller wrote:
Shouldn't we take this thread to the catalog-sig, or at least
parts of it? The discussion pypan vs. ciphon?
I have the impression that some discussion also lets other people
jump in... They see that something is happing.
=20
Sure, I've cc'ed the message to the catalog-sig list.
And also I'm still hacking on pypan, and would be interested in
comments ;-)
That's the rub. There have been several attempts to get this
functionality for python, but my impression is that most of died because
the authors didn't get much feedback and therefore gradually lost
interest in continuing.
> I'm still not sure what the best approach is however. I don't really
> want to have functionality that is available only in some cases but
> trying to offer a remove or uninstall command on all platforms would
> require changes to distutils to track what files are being installed =
(I
> think). =20
=20
IMO we should try to avoid requiring changes to distutils, and this
seems also what the withdrawn PEP 262 was about.
=20
I agree. However, I'm not sure how else to make an uninstall command
available without modifying the distutils source. Windows and rpm based
systems are fine since we can use the uninstaller or the rpm command to
remove modules. Providing a reliable method on other systems seems to
be a more difficult problem. I originally considered just removing
the appropriate directory in /usr/lib/pythonx.x/site-packages but that
doesn't catch everything and may not even work.
> Yeah, I've had more time recently so I've been working on ciphon here
> and there. I usually get more time to work on ciphon in the weekends=
so
> I try to get things done then. The next items on my todo list are
> allowing users to run ciphon from the command line (e.g. ciphon insta=
ll
> mymodule) and to allow http servers to be used as well. If you have =
any
> suggestions on other things that I should work on I'm open to
> suggestions. The two changes I mentioned above probably won't take t=
o
> long to do.
=20
Command line is a good idea: already present in pypan.
Yeah, I didn't really think about it initially but after looking at the
pypan sources and how it worked, it made a lot of sense to have that
available.
In pypan I used urllib - it handles FTP servers, HTTP servers, and
local files "file:..." as well. Seems pretty universal to me.
=20
I was thinking about going along the same lines. I'll need to check how
well the urllib handles ftp downloads from Bernstein's anonftpd though.
Another suggestion: sending bug reports via email doesn't work per
default on windows - it's very uncommon to have an smtp server running
on the machine you work on. If this feature is to stay, there should
probably be a config option 'smtpserver'.
I forgot to ask about how well that worked on Windows. I'll change the
configuration files and the ciphon config command to allow the smtp
server to be set. I think I'll keep it around since it seems like it
would be useful for developers to be able to get an automated bug report
with a stack trace emailed to the bugs mailing list. I'm planning on
extending it a little further so that users can send a bug report
manually and to give better reporting of errors.
--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------
--=-4xzlh8jQ4jc2Y2Sk5TQt
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; CHARSET=3DUTF-8">
<META NAME=3D"GENERATOR" CONTENT=3D"GtkHTML/1.0.2">
</HEAD>
<BODY>
On Fri, 2002-10-25 at 12:48, Thomas Heller wrote:
<BR>
<FONT SIZE=3D"3"></FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>Shouldn't we take this thr=
ead to the catalog-sig, or at least</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>parts of it? The discussion pyp=
an vs. ciphon?</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>I have the impression that some=
discussion also lets other people</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>jump in... They see that someth=
ing is happing.</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">Sure, I've cc'ed the message to the catalog-sig list.</FON=
T>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>And also I'm still hacking=
on pypan, and would be interested in</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>comments ;-)</FONT></FONT></I><=
/PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">That's the rub. There have been several attempts to =
get this functionality for python, but my impression is that most of died b=
ecause the authors didn't get much feedback and therefore gradually lost in=
terest in continuing.</FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> I'm still not sure wh=
at the best approach is however. I don't really</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> want to have functionality=
that is available only in some cases but</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> trying to offer a remove o=
r uninstall command on all platforms would</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> require changes to distuti=
ls to track what files are being installed (I</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> think). </FONT></FONT></I=
>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>IMO we should try to avoid requ=
iring changes to distutils, and this</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>seems also what the withdrawn P=
EP 262 was about.</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">I agree. However, I'm not sure how else to make an u=
ninstall command available without modifying the distutils source. Wi=
ndows and rpm based systems are fine since we can use the uninstaller or th=
e rpm command to remove modules. Providing a reliable method on other=
systems seems to be a more difficult problem. I original=
ly considered just removing the appropriate directory in /usr/lib/pythonx.x=
/site-packages but that doesn't catch everything and may not even wor=
k.</FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> Yeah, I've had more t=
ime recently so I've been working on ciphon here</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> and there. I usually get =
more time to work on ciphon in the weekends so</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> I try to get things done t=
hen. The next items on my todo list are</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> allowing users to run ciph=
on from the command line (e.g. ciphon install</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> mymodule) and to allow htt=
p servers to be used as well. If you have any</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> suggestions on other thing=
s that I should work on I'm open to</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> suggestions. The two chan=
ges I mentioned above probably won't take to</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> long to do.</FONT></FONT><=
/I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>Command line is a good idea: al=
ready present in pypan.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">Yeah, I didn't really think about it initially but after l=
ooking at the pypan sources and how it worked, it made a lot of sense to ha=
ve that available.</FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>In pypan I used urllib - i=
t handles FTP servers, HTTP servers, and</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>local files "file:..."=
; as well. Seems pretty universal to me.</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">I was thinking about going along the same lines. I'l=
l need to check how well the urllib handles ftp downloads from Bernstein's =
anonftpd though.</FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>Another suggestion: sendin=
g bug reports via email doesn't work per</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>default on windows - it's very =
uncommon to have an smtp server running</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>on the machine you work on. If =
this feature is to stay, there should</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>probably be a config option 'sm=
tpserver'.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">I forgot to ask about how well that worked on Windows. I'l=
l change the configuration files and the ciphon config command to allow the=
smtp server to be set. I think I'll keep it around since it seems li=
ke it would be useful for developers to be able to get an automated bug rep=
ort with a stack trace emailed to the bugs mailing list. I'm planning=
on extending it a little further so that users can send a bug report manua=
lly and to give better reporting of errors.</FONT>
<BR>
<FONT SIZE=3D"3"></FONT>
<TABLE CELLSPACING=3D"0" CELLPADDING=3D"0" WIDTH=3D"100%">
<TR>
<TD>
<PRE>--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------</PRE>
</TD>
</TR>
</TABLE>
</BODY>
</HTML>
--=-4xzlh8jQ4jc2Y2Sk5TQt--
--=-nDsWAenhgWjtl1pvJF/N
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iEYEABECAAYFAj25mbIACgkQ6nShCjt5AZLuDQCfbJWe3tPDIejcmxiM07CiTzY3
MnUAn3+sF5aX2ZwBqAsR/Aqjsu+xkZbR
=nkSz
-----END PGP SIGNATURE-----
--=-nDsWAenhgWjtl1pvJF/N--
From thomas.heller@ion-tof.com Fri Oct 25 20:55:57 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 25 Oct 2002 21:55:57 +0200
Subject: [Catalog-sig] RFC: pypan - a Python package manager
In-Reply-To: <1035573683.1575.92.camel@hepcat>
References: <iszuod21.fsf@ion-tof.com> <1035306997.23711.63.camel@hepcat>
<ptu25q5v.fsf@ion-tof.com> <1035309484.23711.84.camel@hepcat>
<wuo64yq2.fsf@ion-tof.com> <1035565579.1571.61.camel@hepcat>
<7kg64dfk.fsf@ion-tof.com> <1035573683.1575.92.camel@hepcat>
Message-ID: <znt22t4z.fsf@ion-tof.com>
Suchandra Thapa <s-thapa-11@alumni.uchicago.edu> writes:
> On Fri, 2002-10-25 at 12:48, Thomas Heller wrote:
>
>
> Shouldn't we take this thread to the catalog-sig, or at least
> parts of it? The discussion pypan vs. ciphon?
> I have the impression that some discussion also lets other people
> jump in... They see that something is happing.
>
>
> Sure, I've cc'ed the message to the catalog-sig list.
>
> And also I'm still hacking on pypan, and would be interested in
> comments ;-)
>
> That's the rub. There have been several attempts to get this
> functionality for python, but my impression is that most of died because
> the authors didn't get much feedback and therefore gradually lost
> interest in continuing.
The chicken and egg issue. It's difficult to reach the critical mass
where the system is actually used.
Question for other readers of this list (David, maybe): How's pyppm
going? I don't hear much from it.
>
> > I'm still not sure what the best approach is however. I don't
> > really want to have functionality that is available only in
> > some cases but trying to offer a remove or uninstall command
> > on all platforms would require changes to distutils to track
> > what files are being installed (I think).
>
> IMO we should try to avoid requiring changes to distutils, and
> this seems also what the withdrawn PEP 262 was about.
>
>
> I agree. However, I'm not sure how else to make an uninstall
> command available without modifying the distutils source. Windows
> and rpm based systems are fine since we can use the uninstaller or
> the rpm command to remove modules. Providing a reliable method on
> other systems seems to be a more difficult problem. I originally
> considered just removing the appropriate directory in
> /usr/lib/pythonx.x/site-packages but that doesn't catch everything
> and may not even work.
Right - this cannot work.
IMO we should concentrate first on those systems where a 'native
package manager' is used. At least this is the approach I will
continue in pypan (Hm, really a strange name. Think I'll change it to
'pypit' or PythonPit, PythonTrove. Difficult for a non-native english
speaker ;-).
Since I only use Windows - except sometimes, when managing my starship
area - this is my main interest.
But a 'remove' feature is really needed for a package manager, I
think.
> Another suggestion: sending bug reports via email doesn't work
> per default on windows - it's very uncommon to have an smtp
> server running on the machine you work on. If this feature is to
> stay, there should probably be a config option 'smtpserver'.
>
> I forgot to ask about how well that worked on Windows.
It works on windows if you change 'localhost' to a real smtp server.
> I'll change the configuration files and the ciphon config command to
> allow the smtp server to be set. I think I'll keep it around since
> it seems like it would be useful for developers to be able to get an
> automated bug report with a stack trace emailed to the bugs mailing
> list. I'm planning on extending it a little further so that users
> can send a bug report manually and to give better reporting of
> errors.
This is only useful if ciphon is in active use by real users.
IMO for the developers it would be better to see the traceback
directly. Maybe a CIPHON_DEBUG environment var or something like that?
My usual work style is to catch exceptions in a sys.excepthook installed
in my sitecustomize file - this recipe is in the Python Cookbook.
This excepthook starts the debugger (pdb), and lets me inspect
the state of the program immediately.
By the way, there are a lot of unqualified try/except clauses in
ciphon.py, which make debugging a pain IMO, and also sometimes mask
programming errors.
Thomas
From DavidA@ActiveState.com Fri Oct 25 21:45:28 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 25 Oct 2002 13:45:28 -0700
Subject: [Catalog-sig] RFC: pypan - a Python package manager
References: <iszuod21.fsf@ion-tof.com> <1035306997.23711.63.camel@hepcat> <ptu25q5v.fsf@ion-tof.com> <1035309484.23711.84.camel@hepcat> <wuo64yq2.fsf@ion-tof.com> <1035565579.1571.61.camel@hepcat> <7kg64dfk.fsf@ion-tof.com> <1035573683.1575.92.camel@hepcat> <znt22t4z.fsf@ion-tof.com>
Message-ID: <3DB9AD68.5010009@ActiveState.com>
Thomas Heller wrote:
> The chicken and egg issue. It's difficult to reach the critical mass
> where the system is actually used.
>
> Question for other readers of this list (David, maybe): How's pyppm
> going? I don't hear much from it.
PyPPM is pretty stagnant. There are a few problems w/ PyPPM from where I'm sitting.
1) Unlike in the Perl world, we have to manage both the code repositories and
the binary package building & distributing processes. In Perl, CPAN exists
(ActiveState need do nothing there), and PPM is "relatively" easy (it's still
hard, since many many modules fail to build on some OSes or their reqs aren't
clear, or ...). It's a maintenance nightmare. I'm not convinced that PyPPM
will live much longer. I'm very interested in alternatives that provide value
to our users and that don't have the high maintenance cost.
2) Distutils, while better than nothing, is still very young. There are (lots
of) bugs (in codepaths that most people don't touch), but mostly it's not really
designed to do what we need it to do. People tend to write setup.py scripts for
one particular goal ("setup.py install"), and extracting the relevant
information to get "setup.py makeppd" to work takes a large amount of work for
each package for each platform. I argued as hard as I could in distutils' early
days that we needed a declarative syntax to describe packages rather than Python
code, but that wasn't chosen. I still think I was right =). I realize that I
could pick up the mantle of distutils and speak with code. However, changing
such a fundamental aspect involves more work than it's worth to me right now.
3) The chicken/egg problem hits us hard -- the infrastructure isn't in place,
and putting in infrastructure is expensive. Making it easier for Python users
to install modules is hardly where ActiveState can expect to reap the largest
rewards for its efforts in our larger picture.
4) The first implementation of PyPPM was sub-par. I'd rather rewrite it from
scratch.
5) Distutils is "too" good in some ways. The windows installers, for example,
solve the problem well enough for enough users (and, crucially, package
authors), that there isn't a huge pressure on us to do it better/differently.
As I've said before, I'm not convinced that Python modules belong at the same
level of granularity as Python itself, but that's a second-order problem.
Just a few thoughts.
--david
From s-thapa-11@alumni.uchicago.edu Fri Oct 25 22:45:09 2002
From: s-thapa-11@alumni.uchicago.edu (Suchandra Thapa)
Date: 25 Oct 2002 16:45:09 -0500
Subject: [Catalog-sig] RFC: pypan - a Python package manager
Message-ID: <1035582310.1575.126.camel@hepcat>
--=-9eNyJx/Y+2Xf/DSv/lNS
Content-Type: multipart/mixed; boundary="=-V29YUkY6rPi32loKGNgD"
--=-V29YUkY6rPi32loKGNgD
Content-Type: multipart/alternative; boundary="=-ORxt5hDhwgcniyLIBxRs"
--=-ORxt5hDhwgcniyLIBxRs
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
On Fri, 2002-10-25 at 14:55, Thomas Heller wrote:=20
The chicken and egg issue. It's difficult to reach the critical mass
where the system is actually used.
It might be useful to see how CPAN got started. =20
But a 'remove' feature is really needed for a package manager, I
think.
Since, ciphon already differentiates rpm and win32 installs from I can
run the uninstall=20
for those cases and do nothing in the general one (for now at least).
> I'll change the configuration files and the ciphon config command to
> allow the smtp server to be set. I think I'll keep it around since
> it seems like it would be useful for developers to be able to get an
> automated bug report with a stack trace emailed to the bugs mailing
> list. I'm planning on extending it a little further so that users
> can send a bug report manually and to give better reporting of
> errors.
=20
My usual work style is to catch exceptions in a sys.excepthook installe=
d
in my sitecustomize file - this recipe is in the Python Cookbook.
This excepthook starts the debugger (pdb), and lets me inspect
the state of the program immediately.
I'll see if I can get something similar into the current code.
By the way, there are a lot of unqualified try/except clauses in
ciphon.py, which make debugging a pain IMO, and also sometimes mask
programming errors.
=20
There are a bit of places in ciphon where the exception handling is
sub-par. That's something that I need to work on.
--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------
--=-ORxt5hDhwgcniyLIBxRs
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; CHARSET=3DUTF-8">
<META NAME=3D"GENERATOR" CONTENT=3D"GtkHTML/1.0.2">
</HEAD>
<BODY>
On Fri, 2002-10-25 at 14:55, Thomas Heller wrote:=20
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>The chicken and egg issue.=
It's difficult to reach the critical mass</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>where the system is actually us=
ed.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">It might be useful to see how CPAN got started. </FO=
NT>
<BR>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>But a 'remove' feature is =
really needed for a package manager, I</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>think.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">Since, ciphon already differentiates rpm and win32 install=
s from I can run the uninstall </FONT>
<BR>
<FONT SIZE=3D"3">for those cases and do nothing in the general one (for now=
at least).</FONT>
<BR>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> I'll change the confi=
guration files and the ciphon config command to</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> allow the smtp server to b=
e set. I think I'll keep it around since</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> it seems like it would be =
useful for developers to be able to get an</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> automated bug report with =
a stack trace emailed to the bugs mailing</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> list. I'm planning on ext=
ending it a little further so that users</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> can send a bug report manu=
ally and to give better reporting of</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> errors.</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>My usual work style is to catch=
exceptions in a sys.excepthook installed</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>in my sitecustomize file - this=
recipe is in the Python Cookbook.</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>This excepthook starts the debu=
gger (pdb), and lets me inspect</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>the state of the program immedi=
ately.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">I'll see if I can get something similar into the current c=
ode.</FONT>
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>By the way, there are a lo=
t of unqualified try/except clauses in</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>ciphon.py, which make debugging=
a pain IMO, and also sometimes mask</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>programming errors.</FONT></FON=
T></I>
</PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">There are a bit of places in ciphon where the exception ha=
ndling is sub-par. That's something that I need to work on.</FONT>
<BR>
<TABLE CELLSPACING=3D"0" CELLPADDING=3D"0" WIDTH=3D"100%">
<TR>
<TD>
<PRE>--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------</PRE>
</TD>
</TR>
</TABLE>
<BR>
</BODY>
</HTML>
--=-ORxt5hDhwgcniyLIBxRs--
--=-V29YUkY6rPi32loKGNgD
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: base64
LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjEuMC42IChHTlUv
TGludXgpCkNvbW1lbnQ6IEZvciBpbmZvIHNlZSBodHRwOi8vd3d3LmdudXBnLm9yZwoKaUVZRUFC
RUNBQVlGQWoyNXV3OEFDZ2tRNm5TaENqdDVBWkpFNndDZlZidUFPc2Qya3RJYUNDVUNlZDFSck0z
ZgpHTGtBb0lic0llSmJvUlJobDlwVlBhNnZ5V0YySmg4RQo9NGd3RAotLS0tLUVORCBQR1AgU0lH
TkFUVVJFLS0tLS0K
--=-V29YUkY6rPi32loKGNgD--
--=-9eNyJx/Y+2Xf/DSv/lNS
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iEYEABECAAYFAj25u2UACgkQ6nShCjt5AZJIQgCgkz5FRt9ZUnZKSyILSLBOi89o
ZfUAoJjfR15bBRWw6ExgCQDW/4SKpc9n
=dFRs
-----END PGP SIGNATURE-----
--=-9eNyJx/Y+2Xf/DSv/lNS--
From DavidA@ActiveState.com Fri Oct 25 22:53:57 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 25 Oct 2002 14:53:57 -0700
Subject: [Catalog-sig] RFC: pypan - a Python package manager
References: <1035582310.1575.126.camel@hepcat>
Message-ID: <3DB9BD75.5020405@ActiveState.com>
Suchandra Thapa wrote:
> On Fri, 2002-10-25 at 14:55, Thomas Heller wrote:
>
> /The chicken and egg issue. It's difficult to reach the critical mass///
> /where the system is actually used.///
>
> It might be useful to see how CPAN got started.
The standard perl way. A huge amount of energy, a lot of people actively
working together, and a lack of fear of hacks. CPAN is nothing but a replicated
FTP server w/ a bit of authentication and a lot of library science =). There's
almost 0 meta-data, no distribution model. It was also started at a time when
there were no alternatives -- only a few people had access to permanent ftp
sites or web sites from which to publish.
--david
From s-thapa-11@alumni.uchicago.edu Fri Oct 25 23:04:31 2002
From: s-thapa-11@alumni.uchicago.edu (Suchandra Thapa)
Date: 25 Oct 2002 17:04:31 -0500
Subject: [Catalog-sig] RFC: pypan - a Python package manager
In-Reply-To: <3DB9BD75.5020405@ActiveState.com>
References: <1035582310.1575.126.camel@hepcat>
<3DB9BD75.5020405@ActiveState.com>
Message-ID: <1035583471.1574.144.camel@hepcat>
--=-Jc7o67b+Sw67N3s9g8Jg
Content-Type: multipart/alternative; boundary="=-8v1viCvHgfUO3Dn8lIYL"
--=-8v1viCvHgfUO3Dn8lIYL
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
On Fri, 2002-10-25 at 16:53, David Ascher wrote:
Suchandra Thapa wrote:
> On Fri, 2002-10-25 at 14:55, Thomas Heller wrote:
>=20
> /The chicken and egg issue. It's difficult to reach the critical mass=
///
> /where the system is actually used.///
>=20
> It might be useful to see how CPAN got started.=20
=20
The standard perl way. A huge amount of energy, a lot of people active=
ly=20
working together, and a lack of fear of hacks. CPAN is nothing but a r=
eplicated=20
FTP server w/ a bit of authentication and a lot of library science =3D)=
. There's=20
almost 0 meta-data, no distribution model. It was also started at a ti=
me when=20
there were no alternatives -- only a few people had access to permanent=
ftp=20
sites or web sites from which to publish.
Couldn't we do the same thing? Have package distribution done using
http or ftp servers then add more sophisticated package handling done
later. As long as we provide a place of updating the original package
manager over the internet, we should be able to make changes in the
distribution method somewhat seamless. =20
We could probably get a fairly good userbase for a package manager just
by providing a few packages like PIL, numeric python, mxtools, and
win32all. =20
--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------
--=-8v1viCvHgfUO3Dn8lIYL
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; CHARSET=3DUTF-8">
<META NAME=3D"GENERATOR" CONTENT=3D"GtkHTML/1.0.2">
</HEAD>
<BODY>
On Fri, 2002-10-25 at 16:53, David Ascher wrote:
<BLOCKQUOTE>
<PRE><FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>Suchandra Thapa wrote:</FO=
NT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> On Fri, 2002-10-25 at 14:5=
5, Thomas Heller wrote:</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> /The chicken and egg issue=
. It's difficult to reach the critical mass///</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> /where the system is actua=
lly used.///</FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>> It might be useful to see =
how CPAN got started. </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I></FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>The standard perl way. A huge =
amount of energy, a lot of people actively </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>working together, and a lack of=
fear of hacks. CPAN is nothing but a replicated </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>FTP server w/ a bit of authenti=
cation and a lot of library science =3D). There's </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>almost 0 meta-data, no distribu=
tion model. It was also started at a time when </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>there were no alternatives -- o=
nly a few people had access to permanent ftp </FONT></FONT></I>
<FONT COLOR=3D"#737373"><FONT SIZE=3D"3"><I>sites or web sites from which t=
o publish.</FONT></FONT></I></PRE>
</BLOCKQUOTE>
<FONT SIZE=3D"3">Couldn't we do the same thing? Have package distribu=
tion done using http or ftp servers then add more sophisticated package han=
dling done later. As long as we provide a place of updating the origi=
nal package manager over the internet, we should be able to make changes in=
the distribution method somewhat seamless. </FONT>
<BR>
<FONT SIZE=3D"3"></FONT>
<BR>
<FONT SIZE=3D"3">We could probably get a fairly good userbase for a package=
manager just by providing a few packages like PIL, numeric python, m=
xtools, and win32all. </FONT>
<BR>
<FONT SIZE=3D"3"></FONT>
<TABLE CELLSPACING=3D"0" CELLPADDING=3D"0" WIDTH=3D"100%">
<TR>
<TD>
<PRE>--=20
------------------------------------------------------------------
Suchandra S. Thapa=20
s-thapa-11@alumni.uchicago.edu
------------------------------------------------------------------</PRE>
</TD>
</TR>
</TABLE>
</BODY>
</HTML>
--=-8v1viCvHgfUO3Dn8lIYL--
--=-Jc7o67b+Sw67N3s9g8Jg
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iEYEABECAAYFAj25v+8ACgkQ6nShCjt5AZKb/ACggMDioy7dkzKr6Jb6XYP6kDTb
RgwAni07K4d7p4a6Gwb8PFx2NFJMqHe+
=J+wi
-----END PGP SIGNATURE-----
--=-Jc7o67b+Sw67N3s9g8Jg--
From Chris.Barker@noaa.gov Fri Oct 25 23:35:16 2002
From: Chris.Barker@noaa.gov (Chris Barker)
Date: Fri, 25 Oct 2002 15:35:16 -0700
Subject: [Catalog-sig] RFC: pypan - a Python package manager
References: <1035582310.1575.126.camel@hepcat> <3DB9BD75.5020405@ActiveState.com>
Message-ID: <3DB9C724.DDBA2A2C@noaa.gov>
David Ascher wrote:
> > /The chicken and egg issue. It's difficult to reach the critical mass///
> > /where the system is actually used.///
> >
> > It might be useful to see how CPAN got started.
>
> The standard perl way. A huge amount of energy, a lot of people actively
> working together, and a lack of fear of hacks. CPAN is nothing but a replicated
> FTP server w/ a bit of authentication and a lot of library science =).
This history can be instructive.
I've always thought that the community as a whole was kind of going
about this backward. There is a lot of talk, and a fair bit of coding
about the technology of a Python Module Catalog, but no one actually
building the catalog.
When I go to set up Python on a new machine (or upgrade). I have to go
to a bunch of different web sites to get the packages I generally use,
but for the most part, getting them to install has been pretty painless.
Some I just copy, some I ./configure; make; make install, some I
./setup.py build, etc. On Windows, most of them come with installers.
If I could just get them from one web site I'd be a whole lot happier
right there. If they all installed the same way that would be great too.
I'd like it even more if there was one installer that installed a
"complete" system. There is no such thing, of course, but there are a
limited number of packages that are in really general use (PIL, mxTools,
Numeric, ...). All I would be wasting is some disk space and download
bandwidth to download a pile of stuff in one fell swoop. If a particular
set of packages where semi-officially known as CompletePython version
2.2 (or whatever), then when you wanted to distribute your code, you
could just require CompletePython 2.2, and your users would have what
they need.
I guess what I'm getting at is an expanded "batteries included"
philosophy. It is great that the standard library has as much as it
does, but it's not enough. Trying to fold all that other stuff into the
standard library wouldn't work, of course, but if we managed to
establish a list of commonly used third party packages, each maintained
by different folks, but obtainable from one source, we'd have a great
start.
I'm inspired by the old "Python on Linux" site (sorry, I can't remember
who put that together). All I had to do was go there, download
everything, and do an rpm -U *, and I had most of the stuff I needed.
What it would take to get this going is someone to host the site, and a
small group of folks to decide on a package list and start populating
the site. As it grows, it would be useful to build the tools to automate
the whole thing more, but we could get it started without any fancy
technology.
I proposed something like this on c.l.p a while back (1yr or so??). I
got surprisingly little support, and I just couldn't do it alone, so it
died. We really need a core group of a few folks (ideally one with
experience in building packages/installers on each of the major
platforms).
What I envision is being able to go to www.completepython.org, click on
version 2.2, click on Windows|OS_X|Linux and get a list of maybe 25-50
packages, any of which I could click on, or a "Download All" button.
That's it. Maybe if that package list grows to 100s (or thousands) it
would require more organization, but let's cross that bridge when we
come to it.
Yow! that rant got a lot longer than I had planned. Thanks for bearing
with me, if you got this far.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
From DavidA@ActiveState.com Fri Oct 25 23:48:14 2002
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 25 Oct 2002 15:48:14 -0700
Subject: [Catalog-sig] RFC: pypan - a Python package manager
References: <1035582310.1575.126.camel@hepcat> <3DB9BD75.5020405@ActiveState.com> <3DB9C724.DDBA2A2C@noaa.gov>
Message-ID: <3DB9CA2E.5070707@ActiveState.com>
Chris Barker wrote:
> David Ascher wrote:
>
> I've always thought that the community as a whole was kind of going
> about this backward. There is a lot of talk, and a fair bit of coding
> about the technology of a Python Module Catalog, but no one actually
> building the catalog.
That doesn't surprise me, if you consider why people do things in this community
-- because they enjoy it. I'm just as guilty of that as anyone. I, like most
of the community, like to design, code, and talk, but not really manage. It
takes a person who likes to manage (with the coddling, arm-twisting, persistence
and rigor that any kind of management implies) to get this job done. IMO, of
course.
--david
From andy@agmweb.ca Fri Oct 25 23:56:40 2002
From: andy@agmweb.ca (Andy McKay)
Date: Fri, 25 Oct 2002 15:56:40 -0700
Subject: [Catalog-sig] RFC: pypan - a Python package manager
References: <1035582310.1575.126.camel@hepcat> <3DB9BD75.5020405@ActiveState.com> <3DB9C724.DDBA2A2C@noaa.gov>
Message-ID: <000501c27c79$c910e590$89505018@cr582427a>
> > The standard perl way. A huge amount of energy, a lot of people
actively
> > working together, and a lack of fear of hacks. CPAN is nothing but a
replicated
> > FTP server w/ a bit of authentication and a lot of library science =).
>
> This history can be instructive.
I think thats part of the problem, the Perl thing started out as something
quite simple. Many of the proposals have been over analysed with far too
many features to the degree that people only get so far before time
pressures or interest drops off. I've seen proposals that want things
available by HTTP, XML-RPC, RSS, FTP, Web Services etc..., and cope with
every different way imaginable of uploading a module.
Although Zope.org may have (many) problems, but its simply a case of making
a tar ball and put some info about it. Sure it doesn't have the automated
installer, but as a simple repository it does a basic job.
While I'm not saying its a good solution, it makes the point that a simple
solution would really help as the first step.
--
Andy McKay
www.agmweb.ca
From rjones@ekit-inc.com Sat Oct 26 00:31:26 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Sat, 26 Oct 2002 09:31:26 +1000
Subject: [Catalog-sig] New proposal, with PEP
Message-ID: <200210260931.26702.rjones@ekit-inc.com>
[
I've now posted three messages on the subject of my online catalog effort -
four if you count the message I sent to the distutils sig. None of those
posts have generated any feedback to me. Not even "piss off, you're wasting
our precious time". That's a bit damn disheartening. People have looked at
and used the prototype though, so I suppose at least the posts weren't
ignored. I'm persisting regardless, because I believe I've got a good idea,
and I have friends who also believe I'm on to a good thing too. I also know
there's a history of little support for these projects.
I have now written a draft PEP which follows. Comments are welcome.
Please note the specific limitations of scope in the Abstract.
]
PEP: XXX
Title: Distutils Enhancements
Version: $Revision: 1.2 $
Last-Modified: $Date: 2002/10/25 07:08:28 $
Author: Richard Jones <rjones@ekit-inc.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 24-Oct-2002
Python-Version: 2.3
Post-History:
Abstract
========
This PEP proposes several extensions to the distutils packaging
system [1]_. These enhancements include a central package index, tools
for submitting package information to the index and extensions to the
package metadata to include Trove [2]_ information.
This PEP does not address either issues of package dependency, nor
centralised storage of packages. Nor is it proposing a local
database of packages as described in PEP 262 [6]_.
Existing package repositories such as the Vaults of Parnassus [3]_,
CPAN [4]_ and PAUSE [5]_ will be investigated as prior art in this
field.
Rationale
=========
Python programmers have long needed a simple method of discovering
existing modules and systems available for their use. It is arguable
that the existence of these systems for other languages have been a
significant contribution to their popularity. The existence of the
catalog-sig, and the many discussions there indicate that there is a
large population of users who recognise this need.
The introduction of the distutils packaging system to Python
simplified the process of distributing shareable code, and included
mechanisms for capture of package metadata, but did little with the
metadata save ship it with the package.
The server should be hosted in the python.org domain, giving it an air
of legitimacy that existing catalog efforts do not have.
The interface for submitting information to the catalog should be as
simple as possible - hopefully just one command-line command for
more regular users.
Issues of package dependency are not addressed due to the complexity
of such a system. The original PEP which proposed such a system was
dropped as the author realised that platform packaging system (RPM,
apt, etc) already handle dependencies, installation and removal.
Issues of package dissemination (storage on a central server) are
not addressed because they require assumptions about availability of
storage and bandwidth that I am not in a position to make.
Specification
=============
The specification takes three parts, the `web interface`_, the
`distutils register command`_ and the `distutils trove
categorisation`_.
Web Interface
-------------
A web interface is implemented over a simple store. The interface is
available through the python.org domain, either directly or as
packages.python.org.
The store has columns for all metadata fields. The (name, version)
double is used as a uniqueness key. Additional submissions for an
existing (name, version) will result in an *update* operation.
The web interface implements the following commands:
**index**
Lists known packages, optionally filtered. An additional HTML page,
**search**, presents a form to the user which is used to customise
the index view. The index will include a browsing interface like
that presented in the Trove interface design section 4.3. The
results will be paginated, sorted alphabetically and only showing
the most recent version. Most recent version information will be
determined using the distutils LooseVersion class.
**display**
Displays information about the package. All fields are displayed as
plain text. The "url" (or "home_page") field is hyperlinked.
**submit**
Accepts a POST form submission of metadata about a package. The
"name" and "version" fields are mandatory, as they uniquely identify
an entry in the index. Submit will automatically determine whether
to create a new entry or updating an existing entry. The metadata
is checked for correctness where appropriate - specifically the
Trove discriminators are compared with the allowed set. An update will
update all information about the package based on the new submitted
information.
**user**
Registers a new user with the index. Requires username, password and
email address. Passwords will be stored on the server as SHA hashes.
If the username exists:
1. If valid HTTP Basic auth is provided, the password and email
address are updated with the submission information, or
2. If no valid auth is provided, the user is informed that the login
is already taken.
Registration will be a three-step process, involving:
1. User submission of details to the *register* command,
2. Web server sending email to the user's email address with a URL
to visit to confirm registration with a random one-time key, and
3. User visits URL with the key and confirms registration.
Several user Roles will exist:
Admin
Can assign Owner Role - they decide who may submit for a given
package name.
Owner
Owns a package name, may assign Maintainer Role for that name
Maintainer
Can submit and update info for a particular package name
**password_reminder**
Sends a password reminder to the user's email address.
The **submit** command will require HTTP Basic authentication,
preferrably over an HTTPS connection.
Distutils Register Command
--------------------------
An additional distutils command, "register" is implemented which
posts the package metadata to the central server. The register command
automatically handles user registration; the user is presented with
three options:
1. login and submit package information
2. register as a new packager
3. send password reminder email
On UN*X systems, the user will be prompted at exit to save their
username/password to a file in their home directory in the file
``.pythonpackagerc``. A similar system could be used on Windows, I
suppose.
Notification of changes to a package entry will be sent to all users
who have submitted information about the package. That is, the original
submitter and any subsequent updaters.
The register command will include a --verify option which performs a
test submission to the server without actually committing the data.
The server will perform its submission verification checks as usual
and report any errors it would have reported during a normal
submission. This is useful for verifying correctness of Trove
discriminators.
Distutils Trove Categorisation
------------------------------
The Trove concept of *discriminators* will be added to the metadata
set available to package authors. The list of discriminators will be
available through the web, and added to the package like so::
setup(
name = "roundup",
version = __version__,
discriminators = [
'Development Status :: 4 - Beta',
'Environment :: Console (Text Based)',
'Environment :: Web Environment',
'Intended Audience :: End Users/Desktop',
'Intended Audience :: Developers',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: Python License',
'Operating System :: MacOS X',
'Operating System :: Microsoft :: Windows',
'Operating System :: POSIX',
'Programming Language :: Python',
'Topic :: Communications :: Email',
'Topic :: Office/Business',
'Topic :: Software Development :: Bug Tracking',
],
url = 'http://sourceforge.net/projects/roundup/',
...
)
It was decided that strings would be used for the discriminator
entries due to the deep nesting that would be involved in a more
formal Python structure.
The original Trove specification that discriminator namespaces be
separated by slashes ("/") unfortunately collides with many of the
names having slashes in them (eg. "OS/2"). The double-colon solution
(" :: ") implemented by Sourceforge and Freshmeat gets around this
limitation.
The list of discriminator values on the module index has been merged
from Freshmeat (with their permission) and Sourceforge (awaiting their
approval). This list will be made available through the web interface
as a text list which may then be copied to the ``setup.py`` file.
Reference Implementation
========================
Reference code and demonstration server are available at
http://mechanicalcat.net:8081/
===== ===================================================
Done Feature
===== ===================================================
Y Submission
Y Index
Y Display
Y Search
N User rego
N Trove
N User verification
N Password reminder
===== ===================================================
In the three days 22nd and 23rd October after the first announcement to
the catalog-sig (22nd) and distutils-sig (23rd), the prototype had 50
visitors (not including myself), three of whom used the register command
to submit package information.
References
==========
.. [1] distutils packaging system
(http://www.python.org/doc/current/lib/module-distutils.html)
.. [2] Trove
(http://tuxedo.org/~esr/trove/)
.. [3] Vaults of Parnassus
(http://www.vex.net/parnassus/)
.. [4] CPAN
(http://www.cpan.org/)
.. [5] PAUSE
(http://pause.cpan.org/)
.. [6] PEP 262, A Database of Installed Python Packages
(http://www.python.org/peps/pep-0262.html)
Copyright
=========
This document has been placed in the public domain.
Acknowledgements
================
Anthony Baxter and Toby Sargeant for encouragement and feedback
during initial drafting.
The many participants of the distutils and catalog SIGs for their
ideas over the years.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:
From martin@v.loewis.de Sat Oct 26 09:23:20 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 26 Oct 2002 10:23:20 +0200
Subject: [Catalog-sig] RFC: pypan - a Python package manager
In-Reply-To: <1035583471.1574.144.camel@hepcat>
References: <1035582310.1575.126.camel@hepcat>
<3DB9BD75.5020405@ActiveState.com> <1035583471.1574.144.camel@hepcat>
Message-ID: <m3fzutip6v.fsf@mira.informatik.hu-berlin.de>
Suchandra Thapa <s-thapa-11@alumni.uchicago.edu> writes:
> Couldn't we do the same thing? Have package distribution done using
> http or ftp servers then add more sophisticated package handling done
> later.
I think what is lacking is the huge amount of energy and the lot of
people actively working together.
Regards,
Martin
From martin@v.loewis.de Sat Oct 26 09:37:43 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 26 Oct 2002 10:37:43 +0200
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210260931.26702.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com>
Message-ID: <m3bs5hioiw.fsf@mira.informatik.hu-berlin.de>
Richard Jones <rjones@ekit-inc.com> writes:
> I've now posted three messages on the subject of my online catalog effort -
> four if you count the message I sent to the distutils sig. None of those
> posts have generated any feedback to me. Not even "piss off, you're wasting
> our precious time". That's a bit damn disheartening.
I think this is the problem. If you get disheartened easily, expect to
give up sooner or later, like everybody else before you.
Every time somebody comes along and offers some kind of catalog, I ask
whether they actually want to operate it, or whether they only want to
write the software - or perhaps even just define a spec.
I'm not interested in specs, and I'm only mildly interested in
software. What I want to see is the catalog in operation.
> People have looked at and used the prototype though, so I suppose at
> least the posts weren't ignored. I'm persisting regardless, because
> I believe I've got a good idea, and I have friends who also believe
> I'm on to a good thing too. I also know there's a history of little
> support for these projects.
I think you are misinterpreting history. People used to get quite
involved in discussing specs; these days, they might be tired because
of fear that the project won't progress beyond the spec phase.
As for contributions to the software: Most software was good enough
when released initially.
As for actually using the infrastructure: That was typically
impossible because of the lack of infrastructure to use.
What other support would you expect?
Regards,
Martin
From lac@strakt.com Sat Oct 26 09:35:54 2002
From: lac@strakt.com (Laura Creighton)
Date: Sat, 26 Oct 2002 10:35:54 +0200
Subject: [Catalog-sig] RFC: pypan - a Python package manager
In-Reply-To: Message from Chris Barker <Chris.Barker@noaa.gov>
of "Fri, 25 Oct 2002 15:35:16 PDT." <3DB9C724.DDBA2A2C@noaa.gov>
References: <1035582310.1575.126.camel@hepcat> <3DB9BD75.5020405@ActiveState.com> <3DB9C724.DDBA2A2C@noaa.gov>
Message-ID: <200210260835.g9Q8Zsen027437@ratthing-b246.strakt.com>
This is what the PBF is planning to do with Python-in-a-Tie. What packages
do you think belong in this release? I am trying to get packages sent
and tested on the Snake Farm, but it turns out that a fair number of
people simply don't care if their packages run on Solaris, HP or
other architectures right now.
Laura Creighton
Chris Barker wrote:
>
> When I go to set up Python on a new machine (or upgrade). I have to go
> to a bunch of different web sites to get the packages I generally use,
> but for the most part, getting them to install has been pretty painless.
> Some I just copy, some I ./configure; make; make install, some I
> ./setup.py build, etc. On Windows, most of them come with installers.
>
> If I could just get them from one web site I'd be a whole lot happier
> right there. If they all installed the same way that would be great too.
>
> I'd like it even more if there was one installer that installed a
> "complete" system. There is no such thing, of course, but there are a
> limited number of packages that are in really general use (PIL, mxTools,
> Numeric, ...). All I would be wasting is some disk space and download
> bandwidth to download a pile of stuff in one fell swoop. If a particular
> set of packages where semi-officially known as CompletePython version
> 2.2 (or whatever), then when you wanted to distribute your code, you
> could just require CompletePython 2.2, and your users would have what
> they need.
>
> I guess what I'm getting at is an expanded "batteries included"
> philosophy. It is great that the standard library has as much as it
> does, but it's not enough. Trying to fold all that other stuff into the
> standard library wouldn't work, of course, but if we managed to
> establish a list of commonly used third party packages, each maintained
> by different folks, but obtainable from one source, we'd have a great
> start.
>
> I'm inspired by the old "Python on Linux" site (sorry, I can't remember
> who put that together). All I had to do was go there, download
> everything, and do an rpm -U *, and I had most of the stuff I needed.
>
> What it would take to get this going is someone to host the site, and a
> small group of folks to decide on a package list and start populating
> the site. As it grows, it would be useful to build the tools to automate
> the whole thing more, but we could get it started without any fancy
> technology.
>
> I proposed something like this on c.l.p a while back (1yr or so??). I
> got surprisingly little support, and I just couldn't do it alone, so it
> died. We really need a core group of a few folks (ideally one with
> experience in building packages/installers on each of the major
> platforms).
>
> What I envision is being able to go to www.completepython.org, click on
> version 2.2, click on Windows|OS_X|Linux and get a list of maybe 25-50
> packages, any of which I could click on, or a "Download All" button.
> That's it. Maybe if that package list grows to 100s (or thousands) it
> would require more organization, but let's cross that bridge when we
> come to it.
>
> Yow! that rant got a lot longer than I had planned. Thanks for bearing
> with me, if you got this far.
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> NOAA/OR&R/HAZMAT (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@noaa.gov
>
> _______________________________________________
> Catalog-sig mailing list
> Catalog-sig@python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
From rjones@ekit-inc.com Sat Oct 26 13:05:44 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Sat, 26 Oct 2002 22:05:44 +1000
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <m3bs5hioiw.fsf@mira.informatik.hu-berlin.de>
References: <200210260931.26702.rjones@ekit-inc.com> <m3bs5hioiw.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210262205.44480.rjones@ekit-inc.com>
On Sat, 26 Oct 2002 6:37 pm, Martin v. Loewis wrote:
> Richard Jones <rjones@ekit-inc.com> writes:
> > I've now posted three messages on the subject of my online catalog effort
> > - four if you count the message I sent to the distutils sig. None of
> > those posts have generated any feedback to me. Not even "piss off, you're
> > wasting our precious time". That's a bit damn disheartening.
>
> I think this is the problem. If you get disheartened easily, expect to
> give up sooner or later, like everybody else before you.
Note that I continued on to say that I have not given up. Just that some
feedback (either positive, negative _or_ ambivalent) would be nice.
> Every time somebody comes along and offers some kind of catalog, I ask
> whether they actually want to operate it, or whether they only want to
> write the software - or perhaps even just define a spec.
>
> I'm not interested in specs, and I'm only mildly interested in
> software. What I want to see is the catalog in operation.
It's in operation right now (minus the user-based features which are pretty
trivial to implement) at the hosting I use. I've had several users querying
the catalog and submitting information to it.
It's not in operation at python.org though, which I see as being important to
its acceptance. I don't have any control over that. Hence the PEP. Ongoing
support would be minimal, and I'd be gladly putting my hand up to take on
that job, assuming I can have access to the machine python.org is hosted on.
> > People have looked at and used the prototype though, so I suppose at
> > least the posts weren't ignored. I'm persisting regardless, because
> > I believe I've got a good idea, and I have friends who also believe
> > I'm on to a good thing too. I also know there's a history of little
> > support for these projects.
>
> I think you are misinterpreting history. People used to get quite
> involved in discussing specs; these days, they might be tired because
> of fear that the project won't progress beyond the spec phase.
>
> As for contributions to the software: Most software was good enough
> when released initially.
>
> As for actually using the infrastructure: That was typically
> impossible because of the lack of infrastructure to use.
>
> What other support would you expect?
For this to work, I need:
1. the web interface installed on python.org, and either someone there to
look after it or access so I can look after it,
2. the "register" command included in the next non-patch python distribution
(ie. 2.3), and
3. optionally have the distutils metadata expanded to include Trove
discriminators
I deliberately minimised the infrastructure requirement of the implementation
so that it could run with little impact on python.org. I could just leave it
running on my host, but that'd have nowhere near the legitemacy of it running
on python.org.
Richard
From doug@hellfly.net Sat Oct 26 13:44:47 2002
From: doug@hellfly.net (Doug Hellmann)
Date: Sat, 26 Oct 2002 08:44:47 -0400
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210260931.26702.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com>
Message-ID: <200210261244.IAA17993@branagan.hellfly.net>
On Friday 25 October 2002 7:31 pm, Richard Jones wrote:
> [
> I've now posted three messages on the subject of my online catalog effort -
> four if you count the message I sent to the distutils sig. None of those
> posts have generated any feedback to me. Not even "piss off, you're wasting
> our precious time". That's a bit damn disheartening. People have looked at
> and used the prototype though, so I suppose at least the posts weren't
> ignored. I'm persisting regardless, because I believe I've got a good idea,
> and I have friends who also believe I'm on to a good thing too. I also know
> there's a history of little support for these projects.
I missed your first post, but I like the PEP and I've tried out the reference
implementation with some success by submitting HappyDoc. Definitely enough
to make me want to see the rest of it implemented. You couldn't have made
the interface any easier, that's for sure!
Doug
From martin@v.loewis.de Sat Oct 26 19:37:00 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 26 Oct 2002 20:37:00 +0200
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210262205.44480.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com>
<m3bs5hioiw.fsf@mira.informatik.hu-berlin.de>
<200210262205.44480.rjones@ekit-inc.com>
Message-ID: <m31y6dnj1v.fsf@mira.informatik.hu-berlin.de>
Richard Jones <rjones@ekit-inc.com> writes:
> Note that I continued on to say that I have not given up. Just that some
> feedback (either positive, negative _or_ ambivalent) would be nice.
Ok, so here's my negative feedback:
Why would anybody want to use such a thing? As long as it contains
only a few dozen packages, I really don't see a need.
To make it larger, I think it must accommodate a wider range of
packages, not just those that get installed through distutils. But if
that is the way to go, how is this different from the Vaults, or
Freshmeat? If I were to look for Python packages, I'd look at
http://freshmeat.net/browse/178
Nicely organized by Trove category and all.
> It's not in operation at python.org though, which I see as being
> important to its acceptance. I don't have any control over
> that. Hence the PEP. Ongoing support would be minimal, and I'd be
> gladly putting my hand up to take on that job, assuming I can have
> access to the machine python.org is hosted on.
I'd be happy to install it on python.org, just give me instructions
(in private email).
> 1. the web interface installed on python.org, and either someone there to
> look after it or access so I can look after it,
I can certainly help here.
> 2. the "register" command included in the next non-patch python
> distribution (ie. 2.3), and
That will be difficult; you need to propose it on python-dev.
For the time being, you should find a means to do without.
> 3. optionally have the distutils metadata expanded to include Trove
> discriminators
That will be really hard, I guess - people won't see the need to add
this to their setup calls. It would also cause the packages to break
on older distutils installations.
Regards,
Martin
From rjones@ekit-inc.com Sat Oct 26 23:54:29 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Sun, 27 Oct 2002 08:54:29 +1000
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <m31y6dnj1v.fsf@mira.informatik.hu-berlin.de>
References: <200210260931.26702.rjones@ekit-inc.com> <200210262205.44480.rjones@ekit-inc.com> <m31y6dnj1v.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210270954.30130.rjones@ekit-inc.com>
On Sun, 27 Oct 2002 5:37 am, Martin v. Loewis wrote:
> Richard Jones <rjones@ekit-inc.com> writes:
> > Note that I continued on to say that I have not given up. Just that some
> > feedback (either positive, negative _or_ ambivalent) would be nice.
>
> Ok, so here's my negative feedback:
>
> Why would anybody want to use such a thing? As long as it contains
> only a few dozen packages, I really don't see a need.
>
> To make it larger, I think it must accommodate a wider range of
> packages, not just those that get installed through distutils.
It's trivial to provide a form interface to manually submit package
information. I will do so now.
> But if
> that is the way to go, how is this different from the Vaults, or
> Freshmeat? If I were to look for Python packages, I'd look at
Because:
1. there's no integration with distutils, and consequently no one-shot,
trivial mechanism for submitting metadata,
2. neither of the above are hosted at python.org, and hence don't have any of
the legitemacy that that hosting would bring, and
3. Freshmeat is a pain to use, and only supports open-source Linux projects
(or at a minimum open-source projects that are available on Linux).
> > It's not in operation at python.org though, which I see as being
> > important to its acceptance. I don't have any control over
> > that. Hence the PEP. Ongoing support would be minimal, and I'd be
> > gladly putting my hand up to take on that job, assuming I can have
> > access to the machine python.org is hosted on.
>
> I'd be happy to install it on python.org, just give me instructions
> (in private email).
Installation on python.org can wait until I've finished the code, but thanks
for the offer! It's good to know that if this is accepted then the python.org
step is possible :)
> > 1. the web interface installed on python.org, and either someone there to
> > look after it or access so I can look after it,
>
> I can certainly help here.
>
> > 2. the "register" command included in the next non-patch python
> > distribution (ie. 2.3), and
>
> That will be difficult; you need to propose it on python-dev.
Hence the PEP. I _think_ I'm following correct procedure here...
> For the time being, you should find a means to do without.
The "register" command could almost be considered the most important component
of this proposal, making it trivial for people to submit information to the
central index.
> > 3. optionally have the distutils metadata expanded to include Trove
> > discriminators
>
> That will be really hard, I guess - people won't see the need to add
> this to their setup calls.
People will add it when they want their package to be easier to find. One of
the features I've built into the specification is the notion of updating the
metadata - so if they want to add in a longer description at a later date,
they can just edit their setup and re-invoke the register command. Same thing
with adding or editing the trove discriminators.
> It would also cause the packages to break
> on older distutils installations.
That's a problem, yes. I'm not sure what the solution would be. Older packages
run with the new code wouldn't have a problem. Newer packages run with older
code will break on the "trove" keyword argument to setup(). I'm not sure what
the solution would be.
Richard
From cliechti@gmx.net Sun Oct 27 00:18:54 2002
From: cliechti@gmx.net (Chris Liechti)
Date: Sun, 27 Oct 2002 01:18:54 +0200
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210270954.30130.rjones@ekit-inc.com>
Message-ID: <2U2WTPIEZTCB2ZHBVQOKLKB92ZSC76.3dbb22de@rock>
Am 27.10.2002 00:54:29, schrieb Richard Jones <rjones@ekit-inc.com>:
>On Sun, 27 Oct 2002 5:37 am, Martin v. Loewis wrote:
>> It would also cause the packages to break
>> on older distutils installations.
>
>That's a problem, yes. I'm not sure what the solution would be. Older packages
>run with the new code wouldn't have a problem. Newer packages run with older
>code will break on the "trove" keyword argument to setup(). I'm not sure what
>the solution would be.
how about a try... catch and a second setup method?
from distutils.core import setup
try:
from distutils.core import extended_setup
except ImportError:
setup(...)
else:
extended_setup(...)
or maybe it's possible to get the distultils version? whatever, i think that solvabe
with a bit more code in the setup script. maybe not that nice 'cause the setup
script get more complicated but it would be optional to use the extened
way anyway.
chris
From martin@v.loewis.de Sun Oct 27 08:49:46 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Oct 2002 09:49:46 +0100
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210270954.30130.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com>
<200210262205.44480.rjones@ekit-inc.com>
<m31y6dnj1v.fsf@mira.informatik.hu-berlin.de>
<200210270954.30130.rjones@ekit-inc.com>
Message-ID: <m3lm4kuuz7.fsf@mira.informatik.hu-berlin.de>
Richard Jones <rjones@ekit-inc.com> writes:
> 2. neither of the above are hosted at python.org, and hence don't
> have any of the legitemacy that that hosting would bring, and
In my eyes, Freshmeat has enough legitemacy on its own.
> > That will be difficult; you need to propose it on python-dev.
>
> Hence the PEP. I _think_ I'm following correct procedure here...
Sure. At some point, you'd need to submit it to the PEP editor, see
PEP 1.
> That's a problem, yes. I'm not sure what the solution would
> be. Older packages run with the new code wouldn't have a
> problem. Newer packages run with older code will break on the
> "trove" keyword argument to setup(). I'm not sure what the solution
> would be.
Unless you can find a hack (like putting data into a separate file for
the register command to find them there), I think there is no
solution. Wait until Python 2.4 or 2.5 for users to start using the
feature (i.e. when they are willing to give up compatibility with
older Python releases, for other reasons).
Regards,
Martin
From rjones@ekit-inc.com Sun Oct 27 11:40:07 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Sun, 27 Oct 2002 21:40:07 +1000
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <m3lm4kuuz7.fsf@mira.informatik.hu-berlin.de>
References: <200210260931.26702.rjones@ekit-inc.com> <200210270954.30130.rjones@ekit-inc.com> <m3lm4kuuz7.fsf@mira.informatik.hu-berlin.de>
Message-ID: <200210272240.07569.rjones@ekit-inc.com>
On Sun, 27 Oct 2002 7:49 pm, Martin v. Loewis wrote:
> Richard Jones <rjones@ekit-inc.com> writes:
> > 2. neither of the above are hosted at python.org, and hence don't
> > have any of the legitemacy that that hosting would bring, and
>
> In my eyes, Freshmeat has enough legitemacy on its own.
Sure, but not in my eyes. And we're not alone in our respective opinions.
> > > That will be difficult; you need to propose it on python-dev.
> >
> > Hence the PEP. I _think_ I'm following correct procedure here...
>
> Sure. At some point, you'd need to submit it to the PEP editor, see
> PEP 1.
Sure, I just wanted to get some feedback on the proposal before going through
the formal process.
> > That's a problem, yes. I'm not sure what the solution would
> > be. Older packages run with the new code wouldn't have a
> > problem. Newer packages run with older code will break on the
> > "trove" keyword argument to setup(). I'm not sure what the solution
> > would be.
>
> Unless you can find a hack (like putting data into a separate file for
> the register command to find them there), I think there is no
> solution. Wait until Python 2.4 or 2.5 for users to start using the
> feature (i.e. when they are willing to give up compatibility with
> older Python releases, for other reasons).
So that would imply building the keyword in now and not advertising it?
Richard
From martin@v.loewis.de Sun Oct 27 21:23:55 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 27 Oct 2002 22:23:55 +0100
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210272240.07569.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com>
<200210270954.30130.rjones@ekit-inc.com>
<m3lm4kuuz7.fsf@mira.informatik.hu-berlin.de>
<200210272240.07569.rjones@ekit-inc.com>
Message-ID: <m3smyrk238.fsf@mira.informatik.hu-berlin.de>
Richard Jones <rjones@ekit-inc.com> writes:
> > Unless you can find a hack (like putting data into a separate file for
> > the register command to find them there), I think there is no
> > solution. Wait until Python 2.4 or 2.5 for users to start using the
> > feature (i.e. when they are willing to give up compatibility with
> > older Python releases, for other reasons).
>
> So that would imply building the keyword in now and not advertising it?
No, documenting it as "new in Python 2.x" (for x=3 or x=4) would
describe the problem precisely: it is then the user's choice whether
or not to use it.
Regards,
Martin
From Juergen Hermann" <jh@web.de Mon Oct 28 10:17:55 2002
From: Juergen Hermann" <jh@web.de (Juergen Hermann)
Date: Mon, 28 Oct 2002 12:17:55 +0200
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210270954.30130.rjones@ekit-inc.com>
Message-ID: <E1867uH-0004DC-00@smtp.web.de>
On Sun, 27 Oct 2002 08:54:29 +1000, Richard Jones wrote:
>> It would also cause the packages to break
>> on older distutils installations.
>
>That's a problem, yes. I'm not sure what the solution would be. Older p=
ackages
>run with the new code wouldn't have a problem. Newer packages run with =
older
>code will break on the "trove" keyword argument to setup(). I'm not sur=
e what
>the solution would be.
We already had such breakage with older distutils versions (pre 1.0 I th=
ink), and
my way to solve it was:
if hasattr(distutils.dist.DistributionMetadata, 'get_keywords'):
setup_args['keywords'] =3D "wiki web"
if hasattr(distutils.dist.DistributionMetadata, 'get_platforms'):
setup_args['platforms'] =3D "win32 posix"
apply(setup, (), setup_args)
Some official distutils "capability" mechanism would be nicer, of course=
. Don't
check version numbers!
Ciao, J=FCrgen
--
J=FCrgen Hermann, Developer
WEB.DE AG, http://webde-ag.de/
From lac@strakt.com Mon Oct 28 13:09:02 2002
From: lac@strakt.com (Laura Creighton)
Date: Mon, 28 Oct 2002 14:09:02 +0100
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: Message from Richard Jones <rjones@ekit-inc.com>
of "Sat, 26 Oct 2002 22:05:44 +1000." <200210262205.44480.rjones@ekit-inc.com>
References: <200210260931.26702.rjones@ekit-inc.com> <m3bs5hioiw.fsf@mira.informatik.hu-berlin.de> <200210262205.44480.rjones@ekit-inc.com>
Message-ID: <200210281309.g9SD92en006449@ratthing-b246.strakt.com>
I am interested in cataloging software for the ability to add
metadata; indeed it is only the ability to add metadata that makes
any extra work worthwhile. But I wonder if making the register command
a library function that works with various versions of Python would
not be a better idea. Then whenever somebody tried to download
something from the web interface on python.org the first thing that
would be checked is 'do you have this library' and if the answer is
no, then it would first install this library for you and then whatever
it is that you came here to get.
The Python Business Forum is very interested in this software, by the
way. Do you want to test it on the Snake Farm?
http://www.lysator.liu.se/~sfarmer/
Laura Creighton
From nobody@maui.dnsvault.com Wed Oct 30 00:27:03 2002
From: nobody@maui.dnsvault.com (Nobody)
Date: Tue, 29 Oct 2002 19:27:03 -0500
Subject: [Catalog-sig] A larger gold balance!
Message-ID: <E186ghL-0002DM-00@maui.dnsvault.com>
<html>
<head>
<title>Untitled</title>
</head>
<body>
<h1 align="center"><font size="+3"><font color="red">Hello all E-Gold and EVOcash account holders!</font></font></h1>
<P align=justify><b> We here at Your Gold Chance
are honest, reliable and willing to help you achieve what you want most..... A
larger e-gold balance! We have developed a program to give you a smart return
without the usual wait of hyip's. For speed and convenience we utilize online
digital currencies. We accept E-Gold and EVOcash.</b></P><B>
<P align=justify> All investments are based on a 33 week plan. We pay you 10% per week for a total of 33 weeks. You first payment starts 7 days after you invest and your capital is returned to you at the end of the 33th week.</P>
<P align=justify> The minimum Investment is $25.00USD and the maximum Investment is $25,000 USD. Do not send us any amount outside these parameters otherwise your investment will be returned without profit.<br></P></B>
<p align="left"><font size="+1"><font color="Red">If you are interested in this program visit site:</font></font></p>
<a href="http://www.yourgoldchance.net/invest.html"><font color="Blue">http://www.yourgoldchance.net/invest.html</font></a>
<p align="left"><font size="+1"><font color="Red">All additional information you can get here:</font></font></p>
<a href="http://www.yourgoldchance.net/"><font color="Blue">http://www.yourgoldchance.net/</font></a>
<p align="left"><font size="+1"><font color="Red">If you would like to contact us, please send us an email:</font></font></p>
<A href="mailto:yourgoldchance@yourgoldchance.net"><font color="Blue">yourgoldchance@yourgoldchance.net</font></A>
<p align="justify"><b>Our company will continue its best efforts to make good on every
payments on our program without any delay.</b></p>
<p><b>Thanks for choosing Your Gold Chance!<br>
Karen Andreozzi,<br>
President<br>
Yor Gold Chance, Inc.</b></p>
</body>
</html>
From rjones@ekit-inc.com Wed Oct 30 00:34:50 2002
From: rjones@ekit-inc.com (Richard Jones)
Date: Wed, 30 Oct 2002 11:34:50 +1100
Subject: [Catalog-sig] New proposal, with PEP
In-Reply-To: <200210281309.g9SD92en006449@ratthing-b246.strakt.com>
References: <200210260931.26702.rjones@ekit-inc.com> <200210262205.44480.rjones@ekit-inc.com> <200210281309.g9SD92en006449@ratthing-b246.strakt.com>
Message-ID: <200210301134.50475.rjones@ekit-inc.com>
On Tue, 29 Oct 2002 12:09 am, Laura Creighton wrote:
> I am interested in cataloging software for the ability to add
> metadata; indeed it is only the ability to add metadata that makes
> any extra work worthwhile.
What metadata would you be talking about adding?
> But I wonder if making the register command
> a library function that works with various versions of Python would
> not be a better idea.
Having the "register" command is central to the adoption of the proposal. It
will be made available as a download for people running pre-adoption versions
of Python (ie. currently 2.1 and 2.2 and hopefully not 2.3).
> Then whenever somebody tried to download
> something from the web interface on python.org the first thing that
> would be checked is 'do you have this library' and if the answer is
> no, then it would first install this library for you and then whatever
> it is that you came here to get.
I'm not sure how this works. Issues of package installation and dependency
have killed so many proposals so far, including PEP 262, that I'm
specifically not addressing them. They can be added later - we just need
something now to start the ball rolling.
> The Python Business Forum is very interested in this software, by the
> way. Do you want to test it on the Snake Farm?
> http://www.lysator.liu.se/~sfarmer/
Not sure what that buys me since there's nothing to actually build.
Richard
From amk@amk.ca Wed Oct 30 21:20:19 2002
From: amk@amk.ca (Andrew Kuchling)
Date: Wed, 30 Oct 2002 16:20:19 -0500
Subject: [Catalog-sig] URLs for Python packages wanted
Message-ID: <E1870GB-0006I9-00@ute.mems-exchange.org>
I'm making yet another attempt at writing a Python catalog. This one
will traverse a list of specified URLs and download and index any
source packages that those URLs point to. (The idea being, if you
have a directory that holds your code, you can simply drop new files
into it and they'll be indexed
To seed the initial database, I need the URLs of HTTP or FTP download
directories. If you have a significant amount of Distutils-packaged
software available, please mail me a URL for your collection. (For
example, most of my code is available from
http://www.amk.ca/files/python/ .)
Thanks!
--amk (www.amk.ca)
OTHELLO: Put out the light, and then put out the light.
-- _Othello_, V, ii
From thomas.heller@ion-tof.com Thu Oct 31 09:16:45 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 31 Oct 2002 10:16:45 +0100
Subject: [Catalog-sig] Re: URLs for Python packages wanted
References: <E1870GB-0006I9-00@ute.mems-exchange.org>
Message-ID: <lm4f9ddu.fsf@ion-tof.com>
Andrew Kuchling <akuchlin@mems-exchange.org> writes:
> I'm making yet another attempt at writing a Python catalog. This one
> will traverse a list of specified URLs and download and index any
> source packages that those URLs point to. (The idea being, if you
> have a directory that holds your code, you can simply drop new files
> into it and they'll be indexed
>
If you need code for extracting metadata (not the complete metadata is
included) and some other useful info from bdist_wininst created binary
windows installer packages, you can look into the make_packagelist.py
file in
http://starship.python.net/crew/theller/pypan/packages/uploaded/pypan-0.2.zip.
This is not a persitent URL.
Thomas
From thomas.heller@ion-tof.com Thu Oct 31 10:24:24 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 31 Oct 2002 11:24:24 +0100
Subject: [Catalog-sig] PEP 243 - Module Repository Upload Mechanism
Message-ID: <d6pqaotj.fsf@ion-tof.com>
I've been playing with a PEP 243 implementation recently.
To remind you, this is Sean Reifschneider's Module Repository Upload
Mechanism PEP: http://www.python.org/peps/pep-0243.html .
With some inspiration from his Swalow code, and some help from
himself, I've played with a CGI script implenting this.
Different from the PEP, I've not used his X-Swalow-XXX headers to
return results to the uploader, instead the cgi-script returns a HTML
page (if submitted from a web-browser), or plain text (if called
programmatically).
Additionally to the swalow code, this script accepts an OpenPGP
compatible signature for the uploaded file, which I create with GnuPG.
I've patched distutils to accept --submit and --sign options for the
bdist_wininst and sdist commands, which will shell out to GnuPG to
create a signature for the created distribution, and then upload the
distribution to the server.
The cgi script itself verifies the signature again a keyring on the
server, and, if successful, moves the uploaded file to the public area.
Here's the transcript of a sample session:
C:\pypan>python setup.py sdist --submit --sign
running sdist
reading manifest file 'MANIFEST'
creating pypan-0.2
creating pypan-0.2\PyPan
copying files to pypan-0.2...
copying README -> pypan-0.2
copying make_packagelist.py -> pypan-0.2
copying pypan.py -> pypan-0.2
copying setup.py -> pypan-0.2
copying PyPan\__init__.py -> pypan-0.2\PyPan
copying PyPan\avail.py -> pypan-0.2\PyPan
copying PyPan\install.py -> pypan-0.2\PyPan
copying PyPan\installed.py -> pypan-0.2\PyPan
copying PyPan\tarfile.py -> pypan-0.2\PyPan
C:\Xilinx_ISE\bin\nt\zip.exe -rq dist\pypan-0.2.zip pypan-0.2
removing 'pypan-0.2' (and everything under it)
You need a passphrase to unlock the secret key for
user: "Thomas Heller <theller@python.net>"
1024-bit DSA key, ID B4985CBA, created 2002-10-24
File `dist\pypan-0.2.zip.asc' exists. Overwrite (y/N)? y
created dist\pypan-0.2.zip.asc
submitting dist\pypan-0.2.zip
########
uploader: saving as ./uploads/pypan-0.2.zip
uploader: signature found
uploader: Signature saved as ./uploads/pypan-0.2.zip.asc
uploader: run 'gpg --batch --verify ./uploads/pypan-0.2.zip.asc ./uploads/pypan-0.2.zip 2>&1'
gpg: Warning: using insecure memory!
gpg: Signature made Thu Oct 31 04:48:23 2002 EST using DSA key ID B4985CBA
gpg: Good signature from "Thomas Heller <theller@python.net>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
gpg: Fingerprint: DB49 621F C353 2ACF 4954 1DF3 5818 2B59 B498 5CBA
uploader: seems to be a source archive file
uploader: parsed ok
uploader: final destination ./packages/uploaded/pypan-0.2.zip
uploader: final destination ./packages/uploaded/pypan-0.2.zip.asc
unknown filetype uploaded/pypan-0.2.zip.asc
unknown filetype uploaded/pypan-0.2.win32.exe.asc
unknown filetype packages.xml
uploader: now 13 packages
The lines following the '#######' is the text returned by the CGI script.
There is a security concern which has to be addressed, I'm aware of
that: when creating the signature you have to supply your passphrase
to the running distutils sdist or bdist_wininst command. I don't think
I like this.
Comments?
Thomas
From martin@v.loewis.de Thu Oct 31 19:08:49 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 31 Oct 2002 20:08:49 +0100
Subject: [Catalog-sig] PEP 243 - Module Repository Upload Mechanism
In-Reply-To: <d6pqaotj.fsf@ion-tof.com>
References: <d6pqaotj.fsf@ion-tof.com>
Message-ID: <m3fzumqvcu.fsf@mira.informatik.hu-berlin.de>
Thomas Heller <thomas.heller@ion-tof.com> writes:
> There is a security concern which has to be addressed, I'm aware of
> that: when creating the signature you have to supply your passphrase
> to the running distutils sdist or bdist_wininst command. I don't think
> I like this.
Are you really supplying it to distutils? I'd expect that gpg could
read it directly from the terminal...
Regards,
Martin
From thomas.heller@ion-tof.com Thu Oct 31 19:34:33 2002
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: 31 Oct 2002 20:34:33 +0100
Subject: [Catalog-sig] PEP 243 - Module Repository Upload Mechanism
In-Reply-To: <m3fzumqvcu.fsf@mira.informatik.hu-berlin.de>
References: <d6pqaotj.fsf@ion-tof.com>
<m3fzumqvcu.fsf@mira.informatik.hu-berlin.de>
Message-ID: <smym8ks6.fsf@ion-tof.com>
martin@v.loewis.de (Martin v. Loewis) writes:
> Thomas Heller <thomas.heller@ion-tof.com> writes:
>
> > There is a security concern which has to be addressed, I'm aware of
> > that: when creating the signature you have to supply your passphrase
> > to the running distutils sdist or bdist_wininst command. I don't think
> > I like this.
>
> Are you really supplying it to distutils? I'd expect that gpg could
> read it directly from the terminal...
No, I'm not supplying it to distutils. Maybe I wasn't clear, here is
the line of code:
os.system("gpg --armor --output %s.asc --detach-sig %s" % (filename, filename))
and gpg reads it from the console.
The user, however, runs
python setup.py sdist --sign --submit
and then is required to enter his passphrase. He might be concerned
where it goes...
Thomas
From martin@v.loewis.de Thu Oct 31 20:28:11 2002
From: martin@v.loewis.de (Martin v. Loewis)
Date: 31 Oct 2002 21:28:11 +0100
Subject: [Catalog-sig] PEP 243 - Module Repository Upload Mechanism
In-Reply-To: <smym8ks6.fsf@ion-tof.com>
References: <d6pqaotj.fsf@ion-tof.com>
<m3fzumqvcu.fsf@mira.informatik.hu-berlin.de>
<smym8ks6.fsf@ion-tof.com>
Message-ID: <m3y98epd44.fsf@mira.informatik.hu-berlin.de>
Thomas Heller <thomas.heller@ion-tof.com> writes:
> The user, however, runs
>
> python setup.py sdist --sign --submit
>
> and then is required to enter his passphrase. He might be concerned
> where it goes...
I think this is fine. The potential distutils users won't just type a
password if some program asks for it. So the documentation needs to
explain what the password is good for, and how precisely the
processing works.
Regards,
Martin