[Catalog-sig] RFC: PEP243: Module Repository Upload Mechanism

Sean Reifschneider jafo@tummy.com
Sat, 24 Mar 2001 16:17:35 -0700

Included below is the version of PEP243 after it's initial round of review.
I welcome any feedback.


PEP: 243
Title: Module Repository Upload Mechanism
Version: $Revision$
Author: jafo-pep@tummy.com (Sean Reifschneider)
Status: Draft
Type: Standards Track
Created: 18-Mar-2001
Python-Version: 2.1
Discussions-To: distutils-sig@python.org


    For a module repository system (such as Perl's CPAN) to be
    successful, it must be as easy as possible for module authors to
    submit their work.  An obvious place for this submit to happen is
    in the Distutils tools after the distribution archive has been
    successfully created.  For example, after a module author has
    tested their software (verifying the results of "setup.py sdist"),
    they might type "setup.py sdist --submit".  This would flag
    Distutils to submit the source distribution to the archive server
    for inclusion and distribution to the mirrors.

    This PEP only deals with the mechanism for submitting the software
    distributions to the archive, and does not deal with the actual
    archive/catalog server.

Upload Process

    The upload will include the Distutils "PKG-INFO" meta-data
    information (as specified in PEP-241 [1]), the actual software
    distribution, and other optional information.  This information
    will be uploaded as a multi-part form encoded the same as a
    regular HTML file upload request.  This form is posted using
    ENCTYPE="multipart/form-data" encoding [RFC1867].

    The upload will be made to the host "modules.python.org" on port
    80/tcp (POST http://modules.python.org:80/swalowpost.cgi).  The form
    will consist of the following fields:

        distribution -- The file containing the module software (for
        example, a .tar.gz or .zip file).

        distmd5sum -- The MD5 hash of the uploaded distribution,
        encoded in ASCII representing the hexadecimal representation
        of the digest ("for byte in digest: s = s + ('%02x' %

        pkginfo (optional) -- The file containing the distribution
        meta-data (as specified in PEP-241 [1]).  Note that if this is not
        included, the distribution file is expected to be in .tar format
        (gzipped and bzipped compreesed are allowed) or .zip format, with a
        "PKG-INFO" file in the top-level directory it extracts

        infomd5sum (required if pkginfo field is present) -- The MD5 hash
        of the uploaded meta-data, encoded in ASCII representing the
        hexadecimal representation of the digest ("for byte in digest:
        s = s + ('%02x' % ord(byte))").

        platform (optional) -- A string representing the target
        platform for this distribution.  This is only for binary
        distributions.  It is encoded as
        "<os_name>-<os_version>-<platform architecture>-<python

        signature (optional) -- A OpenPGP-compatible signature [RFC2440]
        of the uploaded distribution as signed by the author.  This may be
        used by the cataloging system to automate acceptance of uploads.

        protocol_version -- A string indicating the protocol version that
        the client supports.  This document describes protocol version "1".

Return Data

    The status of the upload will be reported using HTTP non-standard
    ("X-*)" headers.  The "X-Swalow-Status" header may have the following

        SUCCESS -- Indicates that the upload has succeeded.

        FAILURE -- The upload is, for some reason, unable to be

        TRYAGAIN -- The server is unable to accept the upload at this
        time, but the client should try again at a later time.
        Potential causes of this are resource shortages on the server,
        administrative down-time, etc...

    Optionally, there may be a "X-Swalow-Reason" header which includes a
    human-readable string which provides more detailed information about
    the "X-Swalow-Status".

    If there is no "X-Swalow-Status" header, or it does not contain one of
    the three strings above, it should be treated as a temporary failure.


        >>> f = urllib.urlopen('http://modules.python.org:80/swalowpost.cgi')
        >>> s = f.headers['x-swalow-status']
        >>> s = s + ': ' + f.headers.get('x-swalow-reason', '<None>')
        >>> print s
        FAILURE: Required field "distribution" missing.

Sample Form

    The upload client must submit the page in the same form as
    Netscape Navigator version 4.76 for Linux produces when presented
    with the following form:

        <H1>Upload file</H1>
        <FORM NAME="fileupload" METHOD="POST" ACTION="swalowpost.cgi"
        <INPUT TYPE="file" NAME="distribution"><BR>
        <INPUT TYPE="text" NAME="distmd5sum"><BR>
        <INPUT TYPE="file" NAME="pkginfo"><BR>
        <INPUT TYPE="text" NAME="infomd5sum"><BR>
        <INPUT TYPE="text" NAME="platform"><BR>
        <INPUT TYPE="text" NAME="signature"><BR>
        <INPUT TYPE="hidden" NAME="protocol_version" VALUE="1"><BR>
        <INPUT TYPE="SUBMIT" VALUE="Upload">


    The following are valid os names:

        aix beos debian dos freebsd hpux mac macos mandrake netbsd
        openbsd qnx redhat solaris suse windows yellowdog

    The above include a number of different types of distributions of
    Linux.  Because of versioning issues these must be split out, and
    it is expected that when it makes sense for one system to use
    distributions made on other similar systems, the download client
    will make the distinction.

    Version is the official version string specified by the vendor for
    the particular release.  For example, "2000" and "nt" (Windows),
    "9.04" (HP-UX), "7.0" (RedHat, Mandrake).

    The following are valid architectures:

        alpha hppa ix86 powerpc sparc ultrasparc


    I currently have a proof-of-concept client and server implemented.
    I plan to have the Distutils patches ready for the 2.1 release.
    Combined with Andrew's PEP-241 [1] for specifying distribution
    meta-data, I hope to have a platform which will allow us to gather
    real-world data for finalizing the catalog system for the 2.2


    [1] Metadata for Python Software Package, Kuchling,

    [RFC1867] Form-based File Upload in HTML

    [RFC2440] OpenPGP Message Format


    This document has been placed in the public domain.

Local Variables:
mode: indented-text
indent-tabs-mode: nil
 A smart terminal is not a smart*ass* terminal, but rather a terminal
 you can educate.  -- Rob Pike
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python