CPAN functionality for python

[Sean Reifschneider]

Unfortunately, this week has been pretty busy.  I've been giving the idea
some thought again (it's been something I've wanted to work on for quite
a while), and just finally today had the chance to look at pythonsiphon.

I'm not really clear (from the code what) as to what it's job is to be.
It smells like it's wanting to be something that understands, given a
distutils package, how to install it.

My thoughts on that are that it's a job that's probably going to be
something subject largely to being handed off to appropriate scripts
based on the platform and some user input (for example, a user may
prefer to have packages they download installed in ~/lib/python1.5,
instead of /usr/lib/python1.5).

Ideally though, the tool should be able to deal with allowing the user
to select their preferred distribution media.  I'd prefer to see an ix86
RedHat 7.0 RPM, a SRPM, and then would fall back to a distutils file or
tar file.

As far as the server side for allowing users to query packages,
dependencies, and locations...

I had actually hoped to pound out some code on this, but that just
didn't happen.  I have built up a schema which I think encompasses
everything I see such a system wanting to track.  Initial versions
probably won't track all that, but I believe they should be considered.

The schema is attached below.

A quick overview of the tables.  I hate the name "items", but it's
the best I could come up with.  That's what I'm calling the general
module (things that can be imported) or package (for example, programs
like Sketch -- why not have them in here as well) description.  In
this case, let's use "sockserv".  The URL would point to the main
location you can find "sockserv" module information.

"Packages" are actually meant to describe what you can download.
This database will track multiple versions, include a checksum
and information on what platform it's meant for (for binary
distributions, for example).

Mirrors are handled by a special table.  The idea is that one should
be able to add mirrors without having to list all the items in that
particular mirror (again).  New entries should be able to be listed
without having to list all the mirrors they might reside on.  So,
I've come up with the idea that a mirror simply specifies a "prefix"
for the URL listed in the "locations" table.

To be honest, my plan is to have a special-case "null" mirror which
non-mirror members are listed under -- that would have an empty
prefix.

So, given a package, version, format, architecture, etc...  The
"locations" table will produce list of all locations which that
file can be found at.

Note that there is a dependency table which lists that one "item"
relies on another "item".

As far as the network interface goes, at the current moment I'm thinking
that queries will be submitted to a web server CGI as a POST, and the
results will be returned as the body as text/plain...  I'll probably use
netstrings (as discussed before) to encode things for transport.

So, any comments?

Sean
=========================
#ISSUES:
#  This doesn't handle partial mirrors.  Should it?  How?

CREATE TABLE items (
   id serial primary key,
   lastUpdated datetime default 'now',

   name text,
   description text,

   type text,     #  module, package
   homepageURL text,
   )

CREATE TABLE packages (
   id serial primary key,
   itemID int4,
   version text,
   md5sum text,
	filesize text,
   insertedOn datetime default 'now',

   description text,       #  description about this specific file
   platformName text,
   platformVersion text,
   architecture text,
   packageFormat text,
   )

#  a collection of URLs that all contain the same set of files (within some
#  precision).  A mirror can be added without having to add all the URLs it
#  contains.  Not useful for handling partial-mirrors though.
CREATE TABLE mirrors (
   id serial primary key,
   name text,
   baseURL text,
   )

CREATE TABLE locations (
   id serial primary key,
   packageID int4,
   mirrorID int4,    #  if found on a mirror, list it
   URL text,            #  append this text to the mirror baseURL
   )

CREATE TABLE depends (
   id serial primary key,
   itemID int4,
   requiresItemID int4
   )

CREATE TABLE users (
   id serial primary key,
   name text,
   password text,
   lastLogin datetime default NULL,
   loginCookie text
   )

CREATE TABLE maintainers (
   userID int4,
   type text,        #  package, item
   piID int4
   )

CREATE TABLE ranking (
   id serial primary key,
   rank int2,
   itemID int4,
   userID int4
   date datetime default 'now',
   )
-- 
 Millions long for immortality who don't know what to do with themselves
 on a rainy Sunday afternoon.                -- Heinlein
Sean Reifschneider, Inimitably Superfluous <jafo at tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python