From moshez@zadka.site.co.il Thu Nov 9 02:16:32 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Thu, 09 Nov 2000 04:16:32 +0200
Subject: [Catalog-sig] Strawman: Debian
Message-ID: <E13thGd-00073b-00@darjeeling.zadka.site.co.il>
I don't know how many people here use Linux, and of those, I don't
know how many people use Debian GNU/Linux. Debian, as a distribution
has a couple of tools to manage distribution of several thousands
packages with complex interdependancy. I suggest, as a strawman,
that we outright steal these tools, and use them for managing Python
modules.
Let me summarize the procedures used in Debian to perform things
as installations, upgrades and removals of software.
The low-level is handled with dpkg -- Debian Packaging tool. It understands
the format of a .deb file (the format in which Debian packages are distributed).
It can check both the control information to see, e.g., that no higher version
of that package is already installed, and that all dependancies are installed
properly. Then it unpacks the distribution, and executes a configuration
script. (For example, in the Python .deb, the configuration script byte-compiles
all Python modules).
As a way to facilitate installations, Debian also has a tool called apt
(A Packaging Tool) which knows which packages exist in the Debian distribution,
and what their connections are (conflicts, dependancies, recommendations, etc.)
To impress everyone, I'll show an actual apt-get session:
moshez@darjeeling:~/prog/my/pmh$ sudo apt-get install xterm
Password:
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
libxaw7 xfree86-common xlibs
The following NEW packages will be installed:
libxaw7 xfree86-common xlibs xterm
0 packages upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 2233kB of archives. After unpacking 6984kB will be used.
Do you want to continue? [Y/n] y
Get:1 http://http.us.debian.org woody/main libxaw7 4.0.1-1 [202kB]
....gets other packages (my connection is slow, so I won't wait for it
to finish)...
...installs them....
...configures them....
Voila! I have "xterm" installed.
The .debs are built from source packages with a little extra control
information. If I want to change some .deb, I merely have to
apt-get source <package>
This downloads the original package source, the Debian diff, opens the
tarball and apply the diff. After I change the source a bit, I can
issue simple Makefile commands and I have an alternative .deb. I'll usually
change the version, so it won't get overridden by apt-get.
Of course, this is only the beginning: apt-get is truly amazing.
I think we can make this system (perhaps stealing only ideas, not code)
work just as well for Python.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From moshez@zadka.site.co.il Thu Nov 9 04:53:30 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Thu, 09 Nov 2000 06:53:30 +0200
Subject: [Catalog-sig] Strawman: Debian
Message-ID: <E13tjiS-0007B5-00@darjeeling.zadka.site.co.il>
I sent this before, when only AMK and I were subscribed.
Here it is again:
I don't know how many people here use Linux, and of those, I don't
know how many people use Debian GNU/Linux. Debian, as a distribution
has a couple of tools to manage distribution of several thousands
packages with complex interdependancy. I suggest, as a strawman,
that we outright steal these tools, and use them for managing Python
modules.
Let me summarize the procedures used in Debian to perform things
as installations, upgrades and removals of software.
The low-level is handled with dpkg -- Debian Packaging tool. It understands
the format of a .deb file (the format in which Debian packages are distributed).
It can check both the control information to see, e.g., that no higher version
of that package is already installed, and that all dependancies are installed
properly. Then it unpacks the distribution, and executes a configuration
script. (For example, in the Python .deb, the configuration script byte-compiles
all Python modules).
As a way to facilitate installations, Debian also has a tool called apt
(A Packaging Tool) which knows which packages exist in the Debian distribution,
and what their connections are (conflicts, dependancies, recommendations, etc.)
To impress everyone, I'll show an actual apt-get session:
moshez@darjeeling:~/prog/my/pmh$ sudo apt-get install xterm
Password:
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
libxaw7 xfree86-common xlibs
The following NEW packages will be installed:
libxaw7 xfree86-common xlibs xterm
0 packages upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 2233kB of archives. After unpacking 6984kB will be used.
Do you want to continue? [Y/n] y
Get:1 http://http.us.debian.org woody/main libxaw7 4.0.1-1 [202kB]
....gets other packages (my connection is slow, so I won't wait for it
to finish)...
...installs them....
...configures them....
Voila! I have "xterm" installed.
The .debs are built from source packages with a little extra control
information. If I want to change some .deb, I merely have to
apt-get source <package>
This downloads the original package source, the Debian diff, opens the
tarball and apply the diff. After I change the source a bit, I can
issue simple Makefile commands and I have an alternative .deb. I'll usually
change the version, so it won't get overridden by apt-get.
Of course, this is only the beginning: apt-get is truly amazing.
I think we can make this system (perhaps stealing only ideas, not code)
work just as well for Python.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From akuchlin@mems-exchange.org Wed Nov 8 21:17:51 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 08 Nov 2000 16:17:51 -0500
Subject: [Catalog-sig] Sources of inspiration
Message-ID: <E13tcbP-0007Ft-00@kronos.cnri.reston.va.us>
Other systems to look at:
Debian: apt (see Moshe's post)
FreeBSD: ports system
RPMfind: www.rpmfind.net
XEmacs: the 'packages' system
Perl: CPAN.pm
Perl: OSD/PPM
OSD was pointed out to me by David Ascher, and stands for Open
Software Distribution (OSD). See www.w3.org/tr/note-osd.html,
www.microsoft.com/workshop/management/osd/osdfaq.asp. "OSD is an XML
vocabulary for describing a software distribution package as a "bill
of materials" and defining relationships and dependencies among its
parts. OSD-based software distribution implementers include Marimba
Inc.'s Castanet, Microsoft's Internet Explorer, Novell Inc.'s ZENworks
and ActiveState Tool Corp.'s Win32 Perl."
--amk
From akuchlin@mems-exchange.org Wed Nov 8 21:18:25 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 08 Nov 2000 16:18:25 -0500
Subject: [Catalog-sig] Initial requirements
Message-ID: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us>
[This is an edited version of a posting I made to the Distutils-SIG
some months ago.]
There are a few steps needed to find and install a package:
1) Discovery : which module does what I need?
2) Download : where can I get a copy?
3) Security : is this actually from the package author, and not a
Trojan horse?
4) Installation : how do I set it up?
Distutils focuses on the hardest and most complicated step, #4.
Requirements from #1)
1.1) Users can browse through a list of available packages, browse
through a hierarchy like Parnassus, or do keyword searches.
1.2) Packages are also indexed by author and date, so you can see all
of Moshe's packages, or the 10 most recently changed packages.
1.3) For each author, their name, e-mail and home page URL are stored.
1.4) Information about a package is only extracted from the metadata
included in the package's setup.py file. This ensures that the info
is up-to-date, saves users the effort of entering data, and encourages
people to use the Distutils.
Requirements from #2)
2.1) Given a package name, software can retrieve one or more download
URLs for the package.
2.2) The catalog should keep copies of the code, not just point to
remote sites, in order to prevent disasters
2.3) It should be possible to mirror the catalog without too much
trouble, using a conventional FTP or HTTP mirroring script.
Requirements from #3)
3.1) Users can check a signature on the downloaded package,
using an external tool such as GnuPG.
3.2) Checking the signature is optional, and can be skipped if the
external tool isn't available. (We could implement our own signature
scheme with Python code, but that's a bad idea; security is hard, and
few people will bother to generate keys that are only useful only for
distributing Python modules.
Related requirement: the Distutils sdist and bdist_* commands should
have a --sign switch to sign the generated .tgz, .rpm, or whatever
file.
Requirements from #4) None
Not covered in these initial requirements: checking for updated
versions, tracking dependencies. These seem to be difficult problems
that require a database of Python packages installed on a system.
--amk
From robin@alldunn.com Wed Nov 8 21:38:11 2000
From: robin@alldunn.com (Robin Dunn)
Date: Wed, 8 Nov 2000 13:38:11 -0800
Subject: [Catalog-sig] Initial requirements
References: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us>
Message-ID: <015b01c049cc$31bdeb20$3225d2d1@ARES>
>
> Not covered in these initial requirements: checking for updated
> versions, tracking dependencies. These seem to be difficult problems
> that require a database of Python packages installed on a system.
>
As long as it's some client-side tool (command line or GUI) doing #1 and #2
and not just some web-based interface, and as long as all the package
meta-data is easily fetchable, then the tool should be able to manage this
without too much problem. It can keep track of what it has installed and
compare with the meta-data for currently available packages.
--
Robin Dunn
Software Craftsman
robin@AllDunn.com
http://wxPython.org Java give you jitters?
http://wxPROs.com Relax with wxPython!
From akuchlin@mems-exchange.org Wed Nov 8 21:45:31 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 8 Nov 2000 16:45:31 -0500
Subject: [Catalog-sig] Initial requirements
In-Reply-To: <015b01c049cc$31bdeb20$3225d2d1@ARES>; from robin@alldunn.com on Wed, Nov 08, 2000 at 01:38:11PM -0800
References: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us> <015b01c049cc$31bdeb20$3225d2d1@ARES>
Message-ID: <20001108164531.A27952@kronos.cnri.reston.va.us>
On Wed, Nov 08, 2000 at 01:38:11PM -0800, Robin Dunn wrote:
>without too much problem. It can keep track of what it has installed and
>compare with the meta-data for currently available packages.
What if the user installs packages w/o using any of the tools, by
downloading it and running "python setup.py install"? You'd really
need a Python-wide database of installed packages; at the very least
this is a question for the Distutils.
--amk
From robin@alldunn.com Wed Nov 8 23:54:14 2000
From: robin@alldunn.com (Robin Dunn)
Date: Wed, 8 Nov 2000 15:54:14 -0800
Subject: [Catalog-sig] Initial requirements
References: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us> <015b01c049cc$31bdeb20$3225d2d1@ARES> <20001108164531.A27952@kronos.cnri.reston.va.us>
Message-ID: <01a001c049df$33418340$3225d2d1@ARES>
>
> What if the user installs packages w/o using any of the tools, by
> downloading it and running "python setup.py install"?
Then you slap the user's hand and tell 'em not to do that! <wink>
> You'd really
> need a Python-wide database of installed packages; at the very least
> this is a question for the Distutils.
I agree distutils should update the DB when packages are installed, but
should it be the master of it? If so then I would think that it should also
be the one that checks dependencies, fetches dependent packages, etc., which
I think is overloading distutils responsibilities too much, and maybe limits
the ability of other tools from interacting with the data and also being a
package manager.
Also, the current entry point into distutils is a package's setup.py file
which wouldn't make much sense for a generic package manager. So there is
going to have to be a separate utility to front-end distutils anyway so why
not keep it all separate so that we can end up with different ones for
different needs. For example, a command-line version**, a tkinter version,
a wx version, one integrated with IDLE, Boa, PythonWin, or PythonWorks,
etc...
Here is what I was thinking of when I spent a few brain cycles on this a
couple weeks ago:
1. There should be a module (or package)** that is part of the standard
distribution that manages access to a local DB (could be as simple as a
shelve of pickled objects, or maybe an XML file) that contains the meta-data
about the installed packages. The module should handle reading the DB into
a collection of objects that can be read by the various package manager
tools, adding new meta-data objects to the DB, deleting ones that have been
uninstalled, locking the DB file, etc.
2. The distutils install command should (by default) use that module to
update the DB with the installed package's info, refuse to install if there
are dependencies missing, etc. There should be flags to disable both of
these in case the package tools want to update the DB themselves or
whatever.
3. The various package tools also use the module to access/update the DB,
but go the next step and know how to fetch the list of archive mirrors,
fetch the master package list from an archive, fetch a package and
dependencies from an archive, install and uninstall packages, etc. Much of
this could be factored out into the standard library module/package to go
along with the DB access, but it should all be designed to be extended by
the package tool so they have the freedom to customize behaviour (especially
the IDE integrations) and add whiz-bangs where appropriate.
4. While I agree that everybody should move to using distutils the simple
fact is that it's either going to take a long time to get there, or that not
everybody will do it. For this I think it would be nice to allow the
package tools and the archives to deal with non-distutils based packages,
perhaps simply by adding to the meta-data the name of a python file in the
distribution to execute to do the install. This could default to "setup.py
install" or something...
5. The package meta-data should also include information about pre-built
binary distributions for various platforms. The package tools can then
fetch the proper binary if available and install it (either un[g]zip it in
the right place, or execute an .exe, run rpm or whatever)
** As long as they are not tightly integrated with distutils perhaps the
command-line package tool and the library module could be distributed as
part of distutils for convenience.
Disclaimer: All this is just my rambling thoughts on the subject and I've
already changed my opinions a couple times since starting to write it... ;-)
There will probably be a better solution that everybody likes better, but
maybe some of these ideas will help out too.
--
Robin Dunn
Software Craftsman
robin@AllDunn.com
http://wxPython.org Java give you jitters?
http://wxPROs.com Relax with wxPython!
From guido@python.org Thu Nov 9 02:19:00 2000
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Nov 2000 21:19:00 -0500
Subject: [Catalog-sig] Initial requirements
In-Reply-To: Your message of "Wed, 08 Nov 2000 15:54:14 PST."
<01a001c049df$33418340$3225d2d1@ARES>
References: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us> <015b01c049cc$31bdeb20$3225d2d1@ARES> <20001108164531.A27952@kronos.cnri.reston.va.us>
<01a001c049df$33418340$3225d2d1@ARES>
Message-ID: <200011090219.VAA25330@cj20424-a.reston1.va.home.com>
> 4. While I agree that everybody should move to using distutils the simple
> fact is that it's either going to take a long time to get there, or that not
> everybody will do it. For this I think it would be nice to allow the
> package tools and the archives to deal with non-distutils based packages,
> perhaps simply by adding to the meta-data the name of a python file in the
> distribution to execute to do the install. This could default to "setup.py
> install" or something...
I think it's okay if a 3rd party creates a distutil'ed version of such
packages. After all this is exactly what the Debian and Red Hat
package managers do.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From robin@alldunn.com Thu Nov 9 02:43:26 2000
From: robin@alldunn.com (Robin Dunn)
Date: Wed, 8 Nov 2000 18:43:26 -0800
Subject: [Catalog-sig] Initial requirements
References: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us> <015b01c049cc$31bdeb20$3225d2d1@ARES> <20001108164531.A27952@kronos.cnri.reston.va.us> <01a001c049df$33418340$3225d2d1@ARES> <200011090219.VAA25330@cj20424-a.reston1.va.home.com>
Message-ID: <003c01c049f6$d6779ec0$0b01a8c0@ARES>
>
> I think it's okay if a 3rd party creates a distutil'ed version of such
> packages. After all this is exactly what the Debian and Red Hat
> package managers do.
Good point.
--
Robin Dunn
Software Craftsman
robin@AllDunn.com
http://wxPython.org Java give you jitters?
http://wxPROs.com Relax with wxPython!
From calvin@cs.uni-sb.de Thu Nov 9 21:46:37 2000
From: calvin@cs.uni-sb.de (Bastian Kleineidam)
Date: Thu, 9 Nov 2000 22:46:37 +0100 (CET)
Subject: [Catalog-sig] DB format and ChangeLog
In-Reply-To: <E13tcbx-0007Fv-00@kronos.cnri.reston.va.us>
Message-ID: <Pine.LNX.4.21.0011092238390.26076-100000@earth.cs.uni-sb.de>
Hi all,
I have two comments:
- make a readable DB format, no binary pickling solutions. Debian uses
a rfc822 DB with "name: value" entries.
- when I am installing new or updated versions of a package I am always
curious what has changed since the last version (bug fixes, new
features etc.). If the changes are important enough, I want to download
and install the new version. If not, then I wait for the next release.
So there should be an option to download/view only the changelog.
Bastian Kleineidam
From akuchlin@mems-exchange.org Thu Nov 9 22:10:48 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 9 Nov 2000 17:10:48 -0500
Subject: [Catalog-sig] Re: OSD (was: Too much good stuff)
In-Reply-To: <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>; from mwa@gate.net on Thu, Nov 09, 2000 at 02:26:47PM -0500
References: <20001109170158.404361D18F@dinsdale.python.org> <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>
Message-ID: <20001109171048.B16625@kronos.cnri.reston.va.us>
On Thu, Nov 09, 2000 at 02:26:47PM -0500, Mark W. Alexander wrote:
>I looked at that some time ago. Despite Microsoft's involvement, I
>have to say I think it's an outstanding idea to have the python
>catalogs be based on a w3c standard.
OSD is not a W3C standard; it's a W3C Note dated 1997, which means
it's purely informational and not any sort of standard. Anyone can
write something up and submit it as a W3C Note.
I finally read the OSD spec yesterday, and wasn't very impressed.
Packages are nodes, connected by arcs representing the dependencies.
It's dressed up with lots of XML fanciness, but it doesn't provide
anything very sophisticated. About the only other users of OSD seem
to be Marimba, and ActiveState's Perl Package Manager format.
Compatibility with the latter might be a good reason to use OSD.
--amk
From akuchlin@mems-exchange.org Thu Nov 9 22:23:54 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 9 Nov 2000 17:23:54 -0500
Subject: [Catalog-sig] Too much good stuff
In-Reply-To: <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>; from mwa@gate.net on Thu, Nov 09, 2000 at 02:26:47PM -0500
References: <20001109170158.404361D18F@dinsdale.python.org> <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>
Message-ID: <20001109172354.C16625@kronos.cnri.reston.va.us>
On Thu, Nov 09, 2000 at 02:26:47PM -0500, Mark W. Alexander wrote:
>What about external (non-python) dependencies that would not be in the
>DB? This would include pretty much any shared library that a wrapper
>module needs. Do we carry enough meta-data to cover all cascading
>dependencies? If so, how do we verify them? Search the system?
>Query the native package manager? (My head's starting to hurt....)
This is why I'm strongly against worrying about tracking dependencies
right now: the issue is complicated and messy to solve. A solution
that only handles storing and finding packages is much easier to do,
and already much better than what currently exists. Let's finish
listing requirements for 1 through 3 and design something first,
leaving dependencies for later.
--amk
From guido@python.org Thu Nov 9 22:46:50 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Nov 2000 17:46:50 -0500
Subject: [Catalog-sig] Too much good stuff
In-Reply-To: Your message of "Thu, 09 Nov 2000 17:23:54 EST."
<20001109172354.C16625@kronos.cnri.reston.va.us>
References: <20001109170158.404361D18F@dinsdale.python.org> <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>
<20001109172354.C16625@kronos.cnri.reston.va.us>
Message-ID: <200011092246.RAA32622@cj20424-a.reston1.va.home.com>
> On Thu, Nov 09, 2000 at 02:26:47PM -0500, Mark W. Alexander wrote:
> >What about external (non-python) dependencies that would not be in the
> >DB? This would include pretty much any shared library that a wrapper
> >module needs. Do we carry enough meta-data to cover all cascading
> >dependencies? If so, how do we verify them? Search the system?
> >Query the native package manager? (My head's starting to hurt....)
>
> This is why I'm strongly against worrying about tracking dependencies
> right now: the issue is complicated and messy to solve. A solution
> that only handles storing and finding packages is much easier to do,
> and already much better than what currently exists. Let's finish
> listing requirements for 1 through 3 and design something first,
> leaving dependencies for later.
Good point. Version and dependency tracking tend to be flamebait.
The distutils sig almost went up in flames over a discussion on how to
define version numbers. :-)
--Guido van Rossum (home page: http://www.python.org/~guido/)
From akuchlin@mems-exchange.org Thu Nov 9 23:03:59 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 09 Nov 2000 18:03:59 -0500
Subject: [Catalog-sig] New Web pages
Message-ID: <E13u0jf-0004N8-00@kronos.cnri.reston.va.us>
I've added the requirements document (slightly modified) and a list of
links to other cataloging systems to the SIG's pages at
www.python.org/sigs/catalog-sig/ . Suggestions, additional links, &c.
are welcome...
--amk
From Juergen Hermann" <jh@web.de Thu Nov 9 22:13:30 2000
From: Juergen Hermann" <jh@web.de (Juergen Hermann)
Date: Fri, 10 Nov 2000 00:13:30 +0200
Subject: [Catalog-sig] Too much good stuff
In-Reply-To: <200011092246.RAA32622@cj20424-a.reston1.va.home.com>
Message-ID: <m13u0sn-005FKHC@smtp.web.de>
On Thu, 09 Nov 2000 17:46:50 -0500, Guido van Rossum wrote:
>Good point. Version and dependency tracking tend to be flamebait.
>The distutils sig almost went up in flames over a discussion on how to
>define version numbers. :-)
Don't track or resolve it automatically, but enable us to yell.
It would be nice though if a package could print an error msg after
querying a central version database for a minimal needed version of a
utility package (i.e. create a version query API).
Bye, J=FCrgen
From moshez@zadka.site.co.il Fri Nov 10 08:23:10 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 10 Nov 2000 10:23:10 +0200
Subject: [Catalog-sig] Too much good stuff
In-Reply-To: Message from "Mark W. Alexander" <mwa@gate.net>
of "Thu, 09 Nov 2000 14:26:47 EST." <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>
References: <Pine.A41.4.21.0011091334420.50696-100000@seminole.gate.net>
Message-ID: <E13u9Sp-0007t8-00@darjeeling.zadka.site.co.il>
> > Message: 1
> > From: Moshe Zadka <moshez@zadka.site.co.il>
> > I don't know how many people here use Linux, and of those, I don't
> > know how many people use Debian GNU/Linux. Debian, as a distribution
> > has a couple of tools to manage distribution of several thousands
> > packages with complex interdependancy. I suggest, as a strawman,
> > that we outright steal these tools, and use them for managing Python
> > modules.
>
> I like the idea, but I don't think the implementation is universal
> enough.
Huh? Why don't you think that we can port apt-get/dpkg into anything
under the sun? (pun intended). It's just C code.
> I've got Linux desktops (multiple distributions) and HP
> and Solaris servers.
So? Nobody said apt-get/dpkg need to work only on Debian -- you just
need to change the place where they place the status file.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From akuchlin@mems-exchange.org Thu Nov 16 15:18:07 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 10:18:07 -0500
Subject: [Catalog-sig] First cut at metadata
Message-ID: <E13wQnf-0006CM-00@kronos.cnri.reston.va.us>
Here's a proposed list of metadata to track in the first version of a
catalog. All of these fields are fairly straightforward in semantics,
and don't drag us into tangled issues. Anything simple that's missing
from this list?
--amk
Information about an author
===========================
Name
Home page
GnuPG/PGP key (both the actual key, and just the ID)
E-mail address (used as ID?)
E-mail address public: y/n
Information about a package
===========================
Name
Version
Supported Platforms
Description
Keywords
Homepage URL
Author IDs
License
Download link
Date of release
Left out for now: Dependency information
Information about a document
============================
Name
Author
Description
URL of HTML version
URL of printable version
(Either URL can be omitted if not applicable.)
From akuchlin@mems-exchange.org Thu Nov 16 15:45:45 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 10:45:45 -0500
Subject: [Catalog-sig] UI for managing catalog
Message-ID: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
What types of interfaces do we need to support for managing the
catalog. A Web-based version, through CGI, is clearly required.
Do we want to require FTP? I'm inclined to say "no", since we want to
make it easy to run mirrors, and it's easier to run just a Web server
than a Web server + FTP server. On the other hand, it's harder to
mirror Web pages; we'll probably need to provide an index page that
points to all the files. The scripts underlying this system will run
on a single host, generating static pages that will get copied by
mirroring sites.
Here's a proposal, that hews very closely to the CPAN model. Users
register themselves as developers, and get an ID. For elegance,
we'll try to encourage re-using developer IDs from other registration
schemes: CPAN, Debian, SourceForge, etc.
Developers then have two options:
1) Upload their distribution through a CGI form, in either .zip or
.tgz format. A script will extract the setup.py file to get the
metadata; users could also enter all this manually, but keeping an
accurate setup.py is encouraged. If the developer has a public key, a
signature for the distribution will be checked as a protective measure
against hacked distributions.
2) Submit a URL from which the distribution can be downloaded. A
script will then fetch it, extract setup.py, and everything follows as
before.
We can then run automated scripts which check for updated versions of
software and automatically update the catalog.
I don't understand how WebDAV would fit into this, and therefore it
should be left out of the first version.
--amk
From guido@python.org Thu Nov 16 16:48:24 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 11:48:24 -0500
Subject: [Catalog-sig] First cut at metadata
In-Reply-To: Your message of "Thu, 16 Nov 2000 10:18:07 EST."
<E13wQnf-0006CM-00@kronos.cnri.reston.va.us>
References: <E13wQnf-0006CM-00@kronos.cnri.reston.va.us>
Message-ID: <200011161648.LAA10426@cj20424-a.reston1.va.home.com>
Looking good. Some nits:
> Information about an author
> ===========================
> Name
> Home page
> GnuPG/PGP key (both the actual key, and just the ID)
> E-mail address (used as ID?)
> E-mail address public: y/n
On the one hand I like using the email address as ID, because it
avoids having to remember my username for a site I don't visit very
often. On the other hand it may become a pain to change -- users
change their email address all the time and we must support this.
> Information about a package
> ===========================
> Name
> Version
> Supported Platforms
Should we give a fixed set of choices, or a free form field? The
problem with a fixed set of choices is that there are lots of subtle
variations, e.g. do you present Windows as a choice or Win95, Win98,
WinNT, Win2000, WinME. Ditto Unix or Linux or Red Hat Linux or Red
Hat Linux 6.1? The problem with free form is that it's hard to use in
a search (e.g. for many people Unix and Linux are synomyms).
> Description
> Keywords
> Homepage URL
> Author IDs
> License
I recommend to use the categories offered by sourceforge here.
> Download link
> Date of release
>
> Left out for now: Dependency information
Submittors could be urged to add this to the Description text if there
are special requirements.
> Information about a document
> ============================
> Name
> Author
> Description
> URL of HTML version
> URL of printable version
> (Either URL can be omitted if not applicable.)
What about downloadable versions?
--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido@python.org Thu Nov 16 16:51:53 2000
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Nov 2000 11:51:53 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: Your message of "Thu, 16 Nov 2000 10:45:45 EST."
<E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
Message-ID: <200011161651.LAA10447@cj20424-a.reston1.va.home.com>
> 2) Submit a URL from which the distribution can be downloaded. A
> script will then fetch it, extract setup.py, and everything follows as
> before.
Is this added complexity (for us) worth having?
--Guido van Rossum (home page: http://www.python.org/~guido/)
From bkc@murkworks.com Thu Nov 16 17:39:38 2000
From: bkc@murkworks.com (Brad Clements)
Date: Thu, 16 Nov 2000 12:39:38 -0500
Subject: [Catalog-sig] First cut at metadata
In-Reply-To: <20001116170200.D6A1A1D26F@dinsdale.python.org>
Message-ID: <3A13D1E6.21393.FFD1077@localhost>
> > Supported Platforms
>
> Should we give a fixed set of choices, or a free form field? The
> problem with a fixed set of choices is that there are lots of subtle
> variations, e.g. do you present Windows as a choice or Win95, Win98,
> WinNT, Win2000, WinME. Ditto Unix or Linux or Red Hat Linux or Red
> Hat Linux 6.1? The problem with free form is that it's hard to use in
> a search (e.g. for many people Unix and Linux are synomyms).
I think we should have a fixed set of choices. When we get to the point
of having a platform specific-websearch, it will make life a lot easier.
Then there's also the issue of binary vs. source for those packages that
have C code.
Can we take any design guidance from RPM as well as CPAN?
Although we don't need to get that sophisticated to start with, I think its
worthwhile having the basic information that could support that in the
future.
So, for example on dependencies, couldn't a package just point to other
packages in the system by "id". And have an option for "external
package" with some kind of human readable text field that describes it.
This way if we ever get automated download of packages with depends,
most things will work automatically unless you download a package
with an "external" dependency. At which point human intervention is
required.
Also, if you get a dependant package that doesn't support your platform,
it won't download either.
Brad Clements, bkc@murkworks.com (315)268-1000
http://www.murkworks.com (315)268-9812 Fax
netmeeting: ils://ils.murkworks.com AOL-IM: BKClements
From amos@digicool.com Thu Nov 16 18:36:32 2000
From: amos@digicool.com (Amos Latteier)
Date: Thu, 16 Nov 2000 10:36:32 -0800
Subject: [Catalog-sig] UI for managing catalog
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com>
Message-ID: <3A142930.7078C00D@digicool.com>
Guido van Rossum wrote:
>
> > 2) Submit a URL from which the distribution can be downloaded. A
> > script will then fetch it, extract setup.py, and everything follows as
> > before.
>
> Is this added complexity (for us) worth having?
I think so. In fact I already have rough code to do exactly this. It
downloads a distutils package, extracts the setup.py and parses it.
I'd like to see the distutils meta-data in setup.py used as much as
possible. If simply packing your stuff with distutils gets you into the
catalog easily, then I think this will motivate folks to use it.
In the long run, I'd like to distutils packing tools to be able to
contact the catalog and upload the package. So for example you could use
a distutils command like this
python setup.py catalog_dist
To build and submit a distribution to the catalog.
-Amos
--
Amos Latteier mailto:amos@digicool.com
Digital Creations http://www.digicool.com
From akuchlin@mems-exchange.org Thu Nov 16 19:28:37 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 16 Nov 2000 14:28:37 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <3A142930.7078C00D@digicool.com>; from amos@digicool.com on Thu, Nov 16, 2000 at 10:36:32AM -0800
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com> <3A142930.7078C00D@digicool.com>
Message-ID: <20001116142837.B24422@kronos.cnri.reston.va.us>
On Thu, Nov 16, 2000 at 10:36:32AM -0800, Amos Latteier wrote:
>Guido van Rossum wrote:
>> Is this added complexity (for us) worth having?
>I think so. In fact I already have rough code to do exactly this. It
>downloads a distutils package, extracts the setup.py and parses it.
And the hard part of this is processing the setup.py safely, not the
downloading, which is just a matter of using urllib.urlopen().
Amos, is this essentially the same code as discussed in the
distutils-sig some months, or have you made lots of changes to it?
> python setup.py catalog_dist
Yes! And this is an example of the catalog producing a change in the
Distutils and hence in the standard library... though if it misses the
deadline for 2.1, no big deal.
--amk
From amos@digicool.com Thu Nov 16 19:26:43 2000
From: amos@digicool.com (Amos Latteier)
Date: Thu, 16 Nov 2000 11:26:43 -0800
Subject: [Catalog-sig] UI for managing catalog
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com> <3A142930.7078C00D@digicool.com> <20001116142837.B24422@kronos.cnri.reston.va.us>
Message-ID: <3A1434F3.77D776D9@digicool.com>
Andrew Kuchling wrote:
>
> On Thu, Nov 16, 2000 at 10:36:32AM -0800, Amos Latteier wrote:
> >I think so. In fact I already have rough code to do exactly this. It
> >downloads a distutils package, extracts the setup.py and parses it.
>
> And the hard part of this is processing the setup.py safely, not the
> downloading, which is just a matter of using urllib.urlopen().
Yes, it's the same old code. It is not safe. I had problems getting
Rexec work here, and punted on it.
This is a problem, regardless of whether the catalog fetches the
distribution, or someone uploads it. In either case the catalog must run
potentially untrusted code to extract meta-data.
-Amos
--
Amos Latteier mailto:amos@digicool.com
Digital Creations http://www.digicool.com
From martin@loewis.home.cs.tu-berlin.de Thu Nov 16 19:43:03 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Thu, 16 Nov 2000 20:43:03 +0100
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <200011161651.LAA10447@cj20424-a.reston1.va.home.com> (message
from Guido van Rossum on Thu, 16 Nov 2000 11:51:53 -0500)
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com>
Message-ID: <200011161943.UAA00873@loewis.home.cs.tu-berlin.de>
> > 2) Submit a URL from which the distribution can be downloaded. A
> > script will then fetch it, extract setup.py, and everything follows as
> > before.
>
> Is this added complexity (for us) worth having?
I'd think so, yes. Developers are often proud of their software, so
they want it to be located on their home page. They will go through
the effort of registering it with the catalog, but they are less
inclined to place the distribution into multiple locations -
especially when internet bandwidth gets in the way.
I know that I was cursing internet bandwidth/SF/US business hours when
I was trying to upload PyXML 0.6.2 this week. It took ages to get it
onto SF, when I really wanted to go home. I had preferred to put it on
a local Web server, and SF fetch it for me.
Regards,
Martin
From martin@loewis.home.cs.tu-berlin.de Thu Nov 16 19:48:03 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Thu, 16 Nov 2000 20:48:03 +0100
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <3A142930.7078C00D@digicool.com> (message from Amos Latteier on
Thu, 16 Nov 2000 10:36:32 -0800)
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com> <3A142930.7078C00D@digicool.com>
Message-ID: <200011161948.UAA00917@loewis.home.cs.tu-berlin.de>
> In the long run, I'd like to distutils packing tools to be able to
> contact the catalog and upload the package.
That would be nice and useful, indeed.
Regards,
Martin
From moshez@zadka.site.co.il Fri Nov 17 14:20:18 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 16:20:18 +0200
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: Message from Andrew Kuchling <akuchlin@mems-exchange.org>
of "Thu, 16 Nov 2000 10:45:45 EST." <E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
Message-ID: <E13wmNG-00086t-00@darjeeling.zadka.site.co.il>
[Andrew]
> What types of interfaces do we need to support for managing the
> catalog. A Web-based version, through CGI, is clearly required.
Of course. By CGI, you probably mean "server side code" rather then
really CGI. CGI is an evil protocol, and will die soon anyway. Why
not use a real tool, like Zope?
[Andrew]
> Do we want to require FTP?
No. FTP is an evil protocol, and will die soon anyway.
[Andrew]
> ...it's harder to
> mirror Web pages;
Not if you have WebDAV. Note that you're talking about two different
things (we might decide to require them both on each CPyAN server, we
might not):
1. Packages mirror.
2. Searching interface.
These do not have to be necessarily related. An easy way to do it is to
have two cookies: "Preferred-Packages:" and "Preferred-Searching:". The
master searching server will redirect to the correct Preferred-Searching
server, which will then use the "Preferred-Packages" cookie to construct
URLs for the correct Packages mirror. We can talk a lot about ways to
do it, but I am pretty much sure it's doable. (I'm a web developer by
profession, so I do this kind of things for a living. You can trust me.)
One advantage would be that there can be fewer searching interfaces (which
is not as high a load on a server) then packages mirror. More, packages
mirror can use either FTP or WebDAV and the searching interface won't
care a bit.
That's talking only about the user side of things -- not the developer.
Note, BTW, that there can be other ways to access a web page then via
a web-browser. Especially if we subscribe to a very rigid set of queries,
it can be done by a Python module.
[Andrew]
> I don't understand how WebDAV would fit into this, and therefore it
> should be left out of the first version.
No! We don't have to use all of WebDAV, but one thing we could use is
that in WebDAV, you *can* get a list of all files in a directory. That's
critical for mirroring.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From moshez@zadka.site.co.il Fri Nov 17 14:23:21 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 16:23:21 +0200
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: Message from Guido van Rossum <guido@python.org>
of "Thu, 16 Nov 2000 11:51:53 EST." <200011161651.LAA10447@cj20424-a.reston1.va.home.com>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com>
Message-ID: <E13wmQE-00087C-00@darjeeling.zadka.site.co.il>
> > 2) Submit a URL from which the distribution can be downloaded. A
> > script will then fetch it, extract setup.py, and everything follows as
> > before.
>
> Is this added complexity (for us) worth having?
Well, it can be left out for the first version. I'm sure it's needed in
the long run: it's much easier on people (like me) whose home computer
is connected at low speed. I upload it anyway to my website, and I'd
rather CPyAN would download it without me having to sacrifice my limited
bandwidth for that.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From martin@loewis.home.cs.tu-berlin.de Thu Nov 16 19:56:55 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Thu, 16 Nov 2000 20:56:55 +0100
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <3A1434F3.77D776D9@digicool.com> (message from Amos Latteier on
Thu, 16 Nov 2000 11:26:43 -0800)
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <200011161651.LAA10447@cj20424-a.reston1.va.home.com> <3A142930.7078C00D@digicool.com> <20001116142837.B24422@kronos.cnri.reston.va.us> <3A1434F3.77D776D9@digicool.com>
Message-ID: <200011161956.UAA01036@loewis.home.cs.tu-berlin.de>
> Yes, it's the same old code. It is not safe. I had problems getting
> Rexec work here, and punted on it.
Can you elaborate on the problems, please? Is there something that
intrinsically can't work, or did you just not get through with it?
Regards,
Martin
From martin@loewis.home.cs.tu-berlin.de Fri Nov 17 10:21:28 2000
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Fri, 17 Nov 2000 11:21:28 +0100
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <E13wmNG-00086t-00@darjeeling.zadka.site.co.il> (message from
Moshe Zadka on Fri, 17 Nov 2000 16:20:18 +0200)
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <E13wmNG-00086t-00@darjeeling.zadka.site.co.il>
Message-ID: <200011171021.LAA00881@loewis.home.cs.tu-berlin.de>
> Of course. By CGI, you probably mean "server side code" rather then
> really CGI. CGI is an evil protocol, and will die soon anyway. Why
> not use a real tool, like Zope?
[...]
> No. FTP is an evil protocol, and will die soon anyway.
You have quite strong opinions on harmless software specifications :-)
I don't think the server-side implementation approach matters at the
moment; I'd rather not make Zope a prerequisite.
As for ftp, I personally consider it much more powerful than http for
file transfer. For example, it has a standard index facility (DIR),
which http doesn't - it is quite a pain to analyse an index.html
automatically.
> No! We don't have to use all of WebDAV, but one thing we could use is
> that in WebDAV, you *can* get a list of all files in a directory. That's
> critical for mirroring.
Ah, mirroring - that's another reason not to mandate Zope, and another
area where ftp does much better than anything else :-)
Regards,
Martin
From moshez@zadka.site.co.il Fri Nov 17 18:49:30 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Fri, 17 Nov 2000 20:49:30 +0200
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: Message from "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
of "Fri, 17 Nov 2000 11:21:28 +0100." <200011171021.LAA00881@loewis.home.cs.tu-berlin.de>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <E13wmNG-00086t-00@darjeeling.zadka.site.co.il> <200011171021.LAA00881@loewis.home.cs.tu-berlin.de>
Message-ID: <E13wqaB-0008Lk-00@darjeeling.zadka.site.co.il>
[Moshe]
> No. FTP is an evil protocol, and will die soon anyway.
[Martin v. Loewis]
> You have quite strong opinions on harmless software specifications :-)
Ummmm....harmless? FTP mixes OSI levels. Where I come from, it's a shooting
offense. Frequently, the trial can be skipped.
[Martin]
> I don't think the server-side implementation approach matters at the
> moment; I'd rather not make Zope a prerequisite.
Did you look at the part where I dissociate searching and grabbing packages?
[Martin]
> As for ftp, I personally consider it much more powerful than http for
> file transfer. For example, it has a standard index facility (DIR),
> which http doesn't
WebDAV has.
[Martin]
> - it is quite a pain to analyse an index.html
> automatically.
Luckily, we're not living in the '90s anymore. We have WebDAV. Why not
use it, instead of a brain dead protocol which cannot be made secure,
and will die an anguished death when IPv6 takes over the world?
And anyway, my proposed scheme allows for FTP mirrors (as opposed to
meta-data searchers) as well.
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From akuchlin@mems-exchange.org Fri Nov 17 15:39:27 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 17 Nov 2000 10:39:27 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <E13wmNG-00086t-00@darjeeling.zadka.site.co.il>; from moshez@zadka.site.co.il on Fri, Nov 17, 2000 at 04:20:18PM +0200
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <akuchlin@mems-exchange.org> <E13wmNG-00086t-00@darjeeling.zadka.site.co.il>
Message-ID: <20001117103927.B25735@kronos.cnri.reston.va.us>
On Fri, Nov 17, 2000 at 04:20:18PM +0200, Moshe Zadka wrote:
>[Andrew]
>Of course. By CGI, you probably mean "server side code" rather then
>really CGI. CGI is an evil protocol, and will die soon anyway. Why
>not use a real tool, like Zope?
We do not want to require people to install the massive bulk of Zope.
Ideally I'd like running a package-only mirror to simply require
running wget or some similar tool; a searchable mirror should simply
require the additional step of setting up a CGI script and mirroring
the additional index file(s).
--amk
From jeremy@alum.mit.edu Fri Nov 17 04:29:11 2000
From: jeremy@alum.mit.edu (Jeremy Hylton)
Date: Thu, 16 Nov 2000 23:29:11 -0500 (EST)
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <E13wqaB-0008Lk-00@darjeeling.zadka.site.co.il>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us>
<E13wmNG-00086t-00@darjeeling.zadka.site.co.il>
<200011171021.LAA00881@loewis.home.cs.tu-berlin.de>
<E13wqaB-0008Lk-00@darjeeling.zadka.site.co.il>
Message-ID: <14868.46103.298676.649511@bitdiddle.concentric.net>
>>>>> "MZ" == Moshe Zadka <moshez@zadka.site.co.il> writes:
MZ> [Moshe]
>> No. FTP is an evil protocol, and will die soon anyway.
MZ> [Martin v. Loewis]
>> You have quite strong opinions on harmless software
>> specifications :-)
MZ> Ummmm....harmless? FTP mixes OSI levels. Where I come from, it's
MZ> a shooting offense. Frequently, the trial can be skipped.
This conversation would be a lot more productive if we could avoid
useless discussions like this. (I realize it's hard, because I
haven't been able to avoid saying: The OSI model is useless.)
Calling a protocol "evil," does not help us make progress on the
catalog.
Jeremy
From moshez@zadka.site.co.il Sat Nov 18 00:45:38 2000
From: moshez@zadka.site.co.il (Moshe Zadka)
Date: Sat, 18 Nov 2000 02:45:38 +0200
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: Message from Andrew Kuchling <akuchlin@mems-exchange.org>
of "Fri, 17 Nov 2000 10:39:27 EST." <20001117103927.B25735@kronos.cnri.reston.va.us>
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <akuchlin@mems-exchange.org> <E13wmNG-00086t-00@darjeeling.zadka.site.co.il> <20001117103927.B25735@kronos.cnri.reston.va.us>
Message-ID: <E13ww8R-0000Iz-00@darjeeling.zadka.site.co.il>
[Andrew]
>Of course. By CGI, you probably mean "server side code" rather then
>really CGI. CGI is an evil protocol, and will die soon anyway. Why
>not use a real tool, like Zope?
[Moshe]
> We do not want to require people to install the massive bulk of Zope.
> Ideally I'd like running a package-only mirror to simply require
> running wget or some similar tool; a searchable mirror should simply
> require the additional step of setting up a CGI script and mirroring
> the additional index file(s).
Let me just note, that AFAIK, CPAN requires mod_perl.
But in any case, that is a mute issue, for the time being.
Could you start a list of open questions that we need to answer before
starting to implement something?
Here are some points to think about:
* Registration/package uploading interface
* Mirroring interface
* How to seperate the meta-data from the content -- in the Debian system,
e.g., each node has a complete copy of all meta-data locally. An "update"
operation updates it. Quite likely, the meta-data file will change a lot,
but requiring mirrors to generate it themselves is unrealistic.
* Completing meta-data: among other things, how to specify dependancies
on non-Python packages? (e.g., zlib)
* What format will the packages be in? I recommend specifying *one* *standard*
*format*, and if at all possible, make sure Python 2.1 supports it natively.
* How do we handle collisions between package names/module names? Unlike Perl,
Python has the problem that if there is a module "foo" there cannot be a
module "foo.bar" too.
* Who do we think will host the master archive? (Alternatives: SF, DC...)
--
Moshe Zadka <sig@zadka.site.co.il>
This is a signature anti-virus.
Please stop the spread of signature viruses!
From amos@digicool.com Fri Nov 17 20:29:49 2000
From: amos@digicool.com (Amos Latteier)
Date: Fri, 17 Nov 2000 15:29:49 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <200011161956.UAA01036@loewis.home.cs.tu-berlin.de>
Message-ID: <web-310577@digicool.com>
On Thu, 16 Nov 2000 20:56:55 +0100
"Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
wrote:
> > Yes, it's the same old code. It is not safe. I had
> problems getting
> > Rexec work here, and punted on it.
>
> Can you elaborate on the problems, please? Is there
> something that
> intrinsically can't work, or did you just not get through
> with it?
The problem is that you need to run python code (setup.py)
to extract
meta-data from a distutils distribution. This is part of the
distutils
design, and this has been discussed on the distutils list.
In the case
of the catalog, you may not trust the code that has either
been
uploaded, or that the catalog has fetched from a URL. My
though was
that you could use rexec to execute untrusted setup.py
files. However,
I don't know much about rexec and was never able to get it
to work
right with sample setup.py files. No doubt someone else
could do
better.
If it turns out the for some reason rexec can't provide a
solution to
this problem, I see some alternatives. We could find someway
to trust
the setup.py file, so a restricted execution environment
wouldn't be
necessary. Or we could require that meta-data come from
somewhere else
(such as manually filling out a web form). Or we could
enhance the
distutils to write meta-data to a text file in a simple
format when it
builds a distribution.
-Amos
From bkc@murkworks.com Sun Nov 19 15:21:10 2000
From: bkc@murkworks.com (Brad Clements)
Date: Sun, 19 Nov 2000 10:21:10 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <web-310577@digicool.com>
References: <200011161956.UAA01036@loewis.home.cs.tu-berlin.de>
Message-ID: <3A17A5EF.568.1EF154E8@localhost>
On 17 Nov 2000, at 15:29, Amos Latteier wrote:
> Or we could
> enhance the
> distutils to write meta-data to a text file in a simple
> format when it
> builds a distribution.
>
I think this is the best solution.
Brad Clements, bkc@murkworks.com (315)268-1000
http://www.murkworks.com (315)268-9812 Fax
netmeeting: ils://ils.murkworks.com AOL-IM: BKClements
From faassen@vet.uu.nl Mon Nov 20 14:08:07 2000
From: faassen@vet.uu.nl (Martijn Faassen)
Date: Mon, 20 Nov 2000 15:08:07 +0100
Subject: [Catalog-sig] Hi, and..
Message-ID: <20001120150807.A13900@vet.uu.nl>
Hi everybody,
Cool new SIG, I just found out about it. It seems that Tim Middleton,
the enigmatic creator of the Vaults of Parnassus has not yet joined
this party. It'd be appropriate to email him a notification in case he
missed it. Has someone done this yet? If not, could they?
His email address is x@vex.net
Regards,
Martijn
P.S. yes, I know Tim (or whatever his name is!) is deep in the PSU hierarchy,
but that's all the more reason to let him in on this. :)
From smendes@systemsfusion.com Mon Nov 20 16:28:30 2000
From: smendes@systemsfusion.com (Sandro Mendes)
Date: Mon, 20 Nov 2000 09:28:30 -0700
Subject: [Catalog-sig] Text Based Application
Message-ID: <3A19512E.633C7A18@systemsfusion.com>
Hi,
Does anybody know of a text based version of Tk? I'm looking for
something like ncurses for python?
Sandro
From fdrake@acm.org Mon Nov 20 16:34:49 2000
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 20 Nov 2000 11:34:49 -0500 (EST)
Subject: [Catalog-sig] Text Based Application
In-Reply-To: <3A19512E.633C7A18@systemsfusion.com>
References: <3A19512E.633C7A18@systemsfusion.com>
Message-ID: <14873.21161.513220.481463@cj42289-a.reston1.va.home.com>
Sandro Mendes writes:
> Does anybody know of a text based version of Tk? I'm looking for
> something like ncurses for python?
The curses module that comes Python uses ncurses; you may need to
enable the module _curses if you built from sources.
-Fred
--
Fred L. Drake, Jr. <fdrake at acm.org>
PythonLabs at Digital Creations
From akuchlin@mems-exchange.org Mon Nov 20 22:45:35 2000
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 20 Nov 2000 17:45:35 -0500
Subject: [Catalog-sig] UI for managing catalog
In-Reply-To: <E13ww8R-0000Iz-00@darjeeling.zadka.site.co.il>; from moshez@zadka.site.co.il on Sat, Nov 18, 2000 at 02:45:38AM +0200
References: <E13wREP-0006Cq-00@kronos.cnri.reston.va.us> <akuchlin@mems-exchange.org> <E13wmNG-00086t-00@darjeeling.zadka.site.co.il> <20001117103927.B25735@kronos.cnri.reston.va.us> <akuchlin@mems-exchange.org> <E13ww8R-0000Iz-00@darjeeling.zadka.site.co.il>
Message-ID: <20001120174535.A22219@kronos.cnri.reston.va.us>
On Sat, Nov 18, 2000 at 02:45:38AM +0200, Moshe Zadka wrote:
>Could you start a list of open questions that we need to answer before
>starting to implement something?
>* Registration/package uploading interface
>* Mirroring interface
>* How to seperate the meta-data from the content -- in the Debian system,
> e.g., each node has a complete copy of all meta-data locally. An "update"
> operation updates it. Quite likely, the meta-data file will change a lot,
> but requiring mirrors to generate it themselves is unrealistic.
>* Completing meta-data: among other things, how to specify dependancies
> on non-Python packages? (e.g., zlib)
I've given my opinion for most of these issues in previous postings:
1) file uploads and automated retrieval from your download site
2) mirroring through something simple like wget
3) not previously discussed -- I don't know.
4) avoiding dependencies for now because it's a rat's nest of issues
>* What format will the packages be in? I recommend specifying *one* *standard*
> *format*, and if at all possible, make sure Python 2.1 supports it natively.
.tar.gz and .zip seem like the two natural choices -- .tar.gz for
Unix, .zip for Windows people. Luckily code is available to read both
of these formats.
>* How do we handle collisions between package names/module names? Unlike Perl,
> Python has the problem that if there is a module "foo" there cannot be a
> module "foo.bar" too.
Unique naming; it's called mxODBC, for example, even though its
package name is ODBC or, soon, mx.ODBC.
>* Who do we think will host the master archive? (Alternatives: SF, DC...)
Beats me; at the moment I'd rather work on a strawman implementation
and worry about long-term hosting later.
--amk