CPAN functionality for python - requirements

[Bruce Sass]

On Tue, 27 Feb 2001, Doug Hellmann wrote:
> On Tue, 27 Feb 2001, Bruce Sass wrote:
> > On Mon, 26 Feb 2001, Doug Hellmann wrote:
> > > On Mon, 26 Feb 2001, Sean Reifschneider wrote:
> > > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote:
> > <...>
> > There is no way an archive network can resolve dependencies because
>
> No, the network doesn't.  But the network by itself isn't going to do most of
> these things.  The client app would need to resolve the dependencies.
>
> If I tell the client to find and download PmwContribD, it needs to figure out
> that I also need Pmw (and perhaps a minimum version) and either suggest that I
> go ahead and install it or fail with a message telling me to install that first
> depending on the command line arguments given.

You are easily pleased.  :)

> > ther is no way for the archive network to know what is installed on a
> > users machine (at least there had better not be!), or what packages
> > are available to fulfil those dependencies.  The best that can
> > be done is to tell the target system what is required by the pkg being
> > looked at.  The dependencies get resolved when the native system
> > decides on where it should fetch the other stuff from.
>
> It sounds like the knows how to find a particular package, but getting it
> installed is your problem.  We can make it smarter than that.

Not "your", the native package management system's (and they have
been doing it a relatively long time).

> > Dependencies are a big problem.  I've spoken with a local LUG member
> > who was complaining that Debian does not know about the stuff he gets from
> > CPAN, which makes mixing CPAN and debianized CPAN stuff a pain.
> > Debian's solution is a user level program to debianize arbitrary CPAN
> > modules... I want the same solution for Python modules, except I want
> > it done transparently.
>
> I can see that as somewhat desirable, and it could be implemented, but do
> you make the cyphon client download the right package format or convert
> a standard distro into the right format?  Doesn't the user have to specify a
> package type?

I prefer building from a generic package to a system specific package
(QC is reduced to checking the contents of the generic source package
and having a good package making script, hopefully provided by the
same group of people who control the format of the package), but there
is no reason for not supporting both (except maybe room to store it
all), and in some cases it would be desirable.

Sure the user has to specify a package type, _if_ they want a specific
type of package, that info can be included in the name of the file
being requested (built up from the name of the python piece, a client
id, etc.)

> > > 3. Download the appropriate packages to give you what you want.
> > > 4. Install those packages.
> >
> > Doesn't that mean the program needs to know about all the distros, and
> > all the version of those distros, and all the supported ways a user
> > can tweak the system.  If you don't do that, and keep it up, then your
> > installer is going to be breaking often (or always broken, depending
> > on the user or distro).
> >
> > > Do we agree these are the primary features?
> >
> > Both 2. and 4. are the domain of the target system.  2. can be
> > modified (or maybe just reworded).  4...
>
> What I want is a single app I can use that will do all of those things.  I want
> to type one command and have it find and install software on my system that
> will allow me to use some Python package that I don't have yet.  That app would
> have a client side and server side, but as a user I don't care about the server
> side unless I want to specify a specific mirror to use by default.
>
> > > I sense a consensus that the "install" part should be handled by distutils.  Is
> > > that right?
> >
> > As long as distutils does not presume to know how to do the
> > installation.
>
> How can it do the install if it doesn't know how?
>
> It sounds like you do *not* want distutils to do the install, and that you
> cyphon to download/create a platform-specific package to be installed.  Since we
> *have* distutils as an ostensibly standard installation system for Python
> modules, shouldn't we use it?

I would not use it if it means bypassing the native package management
system, and would be willing to wait for "official" packages rather
than use third-party packages.

Best of all scenarios would be if all the Python programmers used the
same system I did and spit out packages in my favorite format, next
best is having a package built by someone who knows my system just as
well or better than I, then their proxy in the form of a scripted
build, followed by someone I don't know building the package, a
tarball or some other package format I can handle.

The first is a dream, the second could happen now and then if I was
lucky, so I'll try for my third choice (and it sounds like distutils
can handle it).

> > > Other requirements we might lay down up front:
> > >
> > > 1. Should run on all platforms where Python runs.
> > > 2. Must support mirrors on the server side.
> > > 3. Need to include documentation along with source for packages.
> > >
> > > The features related to dependencies and downloads could be handled by a
> > > platform-specific packaging system, but integrating with all of the different
> > > options on all of the different platforms where Python runs increases the
> > > complexity of the new tool.
> >
> > Downloading is independent of the native package management system.
>
> No, it isn't.  Even if *all* the tool does is download things I told it to find
> and write them to the current directory, it has to know what format I like or
> it may download (for example) debian packages for my Mac -- and then I won't
> get useful results from the tool.

Build up the name of the download from the name of the package and
system specific information, just like is done with the names of .deb
and .rpm packages.

> > Complexity is only increased if the new tool thinks it should handle
> > platform/OS/distro specific issues.
>
> It *must* handle those issues to be useful.

Isn't that is a lot of duplicated work.

> > > (How do we handle RPMs on MacOS?  How do we handle
> > > HQX files on Win32?  What about for Pippy, where files as such might not even
> > > be appropriate?  Does Jython have any special requirements?)  Do we deal with
> > > this complexity by deciding on a per-platform basis what format to use, or do
> > > we force the user to specify the format they want downloaded?
> >
> > You don't.  You give the data to the target system and let it worry
> > about the complexity of packaging.
>
> What is the "target system," the OS?

The system the python package is being downloaded to.

> > There is no reason for the archive network to know anything except
> > where to send the data!
>
> Right, but we do need a client so that users have a way to interact with the
> network.
>
> Wait -- maybe that's where the disconnect is!  I'm describing features of the
> system from the client-side view, assuming the network would provide services
> to support those features.  It sounds like you are describing features of the
> network.  We should determine the requirements from the user perspective first,
> and design something to meet those requirements.  Designs for the network
> services will follow from the requirements.

I see them as two pieces of the same puzzle.  The third, and most
important, is the data structure used to represent the python package.

> > > If we're modeling what we're doing based on CPAN, maybe we should look at
> > > that design.  What format does CPAN use when downloading Perl packages?  What
> > > features does CPAN have that we want?  What does it not have that we want?
> >
> > Pros:
> > don't know (long strings of punctuation and symbols give me a headache ;).
>
> Heh, tell me about it.
>
> I can't remember now whether CPAN did installations or just downloaded stuff
> and made you figure out how to install it.
>
> > Cons:
> > Dependencies problems when mixed with modules fetched by other methods.
>
> Right, so we need a single way of determining, maintaining, and
> resolving dependencies.

Or a way of enabling the native tools so they can do it.  I would not
want to accept the job of keeping up with changing packaging standards
for <how many?> different systems, and don't expect that anyone else
would want to (or be able to keep it up).


- Bruce