Mailman 3 March 2001 - Distutils-SIG

Patch for PEP 241
by Andrew Kuchling 20 Mar '01

20 Mar '01

I've got a preliminary version of the patch to implement PEP 241: https://sourceforge.net/tracker/?func=detail&aid=407434&group_id=5470&atid=… There are still issues to resolve; I have to figure the right place to convert the keyword and platform options from strings to lists, and implement checking on the license option. A sample package containing a PKG-INFO file is at http://www.amk.ca/files/python/Distutils-1.0.2pre.tar.gz, should you want to test some code on it. --amk

2 1

RFC: PEP 243: Module Repository Upload Mechanism
by Sean Reifschneider 20 Mar '01

20 Mar '01

Here is PEP 243, discussing how to make Distutils submit .tar.gz packages (and the like) to the mythical catalog server. I've got code for this prototyped, but give me a couple of days to make it completely PEP-compliant. Moshe has already commented: a. you don't say which URL to post to (just spell it out: POST http://modules.python.org:80/ (and let upload override that) b. have the codes be HTTP codes and the headers HTTP non-standard (X-) headers. how about using the HTTP headers and error-codes, and have *all* the page human readable. Good comments, I'll look at building them in... I look forward to any comments -- I'd like to start working on distutils patches within the week. Thanks, Sean ========================================================================== PEP: 243 Title: Module Repository Upload Mechanism Version: $Revision$ Author: jafo-pep(a)tummy.com (Sean Reifschneider) Status: Draft Type: Standards Track Created: 18-Mar-2001 Python-Version: 2.1 Post-History: Discussions-To: distutils-sig(a)python.org Abstract For a module repository system (such as Perl's CPAN) to be successful, it must be as easy as possible for module authors to submit their work. An obvious place for this submit to happen is in the Distutils tools after the distribution archive has been successfully created. For example, after a module author has tested their software (verifying the results of "setup.py sdist"), they might type "setup.py sdist --submit". This would flag Distutils to submit the source distribution to the archive server for inclusion and distribution to the mirrors. This PEP only deals with the mechanism for submitting the software distributions to the archive, and does not deal with the actual archive/catalog server. Upload Process The upload will include the Distutils "PKG-INFO" meta-data information (as specified in PEP-241 [1]), the actual software distribution, and other optional information. This information will be uploaded as a multi-part form encoded the same as a regular HTML file upload request. This form is posted using ENCTYPE="multipart/form-data" encoding. The upload will be made to the host "modules.python.org" on port 80/tcp. The form will consist of the following fields: distribution -- The file containing the module software (for example, a .tar.gz or .zip file). distmd5sum -- The MD5 hash of the uploaded distribution, encoded in ASCII representing the hexadecimal representation of the digest ("for byte in digest: s = s + ('%02x' % ord(byte))"). pkginfo -- The file containing the distribution meta-data (as specified in PEP-241 [1]). infomd5sum -- The MD5 hash of the uploaded meta-data, encoded in ASCII representing the hexadecimal representation of the digest ("for byte in digest: s = s + ('%02x' % ord(byte))"). platform (optional) -- A string representing the target platform for this distribution. This is only for binary distributions. It is encoded as "<os_name>-<os_version>-<platform architecture>". signature (optional) -- A GPG signature of the uploaded distribution as signed by the author. This may be used by the cataloging system to automate acceptance of uploads. Return Data The upload will report the status of the upload by sending the string "Upload status:" followed by one of the following: SUCCESS -- Indicates that the upload has succeeded. FAILURE -- The upload is, for some reason, unable to be processed. TRYAGAIN -- The server is unable to accept the upload at this time, but the client should try again at a later time. Potential causes of this are resource shortages on the server, administrative down-time, etc... Following the above string may be a human-readable string providing further information. This message continues to the end of the returned data-stream. Returned data which does not fit the above format should be treated as a temporary failure. Sample Form The upload client must submit the page in the same form as Netscape Navigator version 4.76 for Linux produces when presented with the following form: <H1>Upload file</H1> <FORM NAME="fileupload" METHOD="POST" ACTION="swalowpost.cgi" ENCTYPE="multipart/form-data"> <INPUT TYPE="file" NAME="distribution"><BR> <INPUT TYPE="text" NAME="distmd5sum"><BR> <INPUT TYPE="file" NAME="pkginfo"><BR> <INPUT TYPE="text" NAME="infomd5sum"><BR> <INPUT TYPE="text" NAME="platform"><BR> <INPUT TYPE="text" NAME="signature"><BR> <INPUT TYPE="SUBMIT" VALUE="Upload"> </FORM> Platforms The following are valid os names: debian hpux mandrake redhat solaris suse windows yellowdog The above include a number of different types of distributions of Linux. Because of versioning issues these must be split out, and it is expected that when it makes sense for one system to use distributions made on other similar systems, the download client will make the distinction. Version is the official version string specified by the vendor for the particular release. For example, "nt" (Windows), "9.04" (HP-UX), "7.0" (RedHat, Mandrake). The following are valid architectures: alpha hppa ix86 powerpc sparc ultrasparc Status I currently have a proof-of-concept client and server implemented. I plan to have the Distutils patches ready for the 2.1 release. Combined with Andrew's PEP-241 [1] for specifying distribution meta-data, I hope to have a platform which will allow us to gather real-world data for finalizing the catalog system for the 2.2 release. References [1] Metadata for Python Software Package, Kuchling, http://python.sourceforge.net/peps/pep-0241.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: -- I used to think that the brain was the most wonderful organ in my body. Then I realized who was telling me this. -- Emo Phillips Sean Reifschneider, Inimitably Superfluous <jafo(a)tummy.com> tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python

1 0

PEP 241 posted
by A.M. Kuchling 19 Mar '01

19 Mar '01

I've posted PEP 241 to c.l.py.a, so make any final comments. Time for me to start implementing... --amk

1 0

Finishing PEP 241
by Andrew Kuchling 19 Mar '01

19 Mar '01

I want to get the last XXX removed from PEP 241 so it can be posted to c.l.py.a and python-dev *today*. The missing bit is: Platform (multiple use) A (XXX whitespace? comma?)-separated list of platform specifications. Platform specifications are limited to the following list: XXX copy list from PPD? SourceForge? Example: 'XXX' So, what to do? Some options are: * Specify a short and incomplete list of platform specs: MacOS/Windows/POSIX/BeOS/... State in the PEP that this list is incomplete. Binary distributions will use a different metadata header (Compiled-Platform, Supported-Platform, or something like that) to specify the OS and CPU they were compiled for. * Leave 'Platform' to be filled in for binary distributions. Developers should put platform limitation notes in the 'Description'. I lean toward #1, and if I don't hear any strong reasons to do otherwise, that's what I'll check it and post, sometime after 5PM EST. --amk

2 1

patching filelist.py to allow quoted strings
by Daniel Yoo 17 Mar '01

17 Mar '01

Dmitry Balabanov on python-help had requested information about why the following Manifest.in line didn't work; he used the following line: ### recursive-include 'Max Fray' *.txt ### Because there were spaces in the directory, it didn't quite parse correctly. I've isolated filelist._parse_template_line() to be responsible for parsing out these lines. _parse_template_line() uses a simple string.split() to break a line into tokens. I'd like to know if it's a good idea to allow spaces in directory names like that. I'm attaching a patch that tries to split the line and takes quoted elements into account. I'm in digest mode, and a newbie at distutils, so please be gentle. *grin* --- Original message follows below: --- >From dimonb(a)beep.ru Sat Mar 10 11:36:01 2001 From: dimonb(a)beep.ru (Balabanov Dmitry) Date: Sat, 10 Mar 2001 11:36:01 +0000 Subject: [Python-Help] help with distutils Message-ID: <01031011360100.00515@doors> Hi!! Problem is very simple but i can't solve it... I have such string in MANIFEST.in: recursive-include 'Max Fray' *.txt But `python setup.py sdist --manifest-only` spawns: running sdist warning: sdist: missing required meta-data: url reading manifest template 'MANIFEST.in' warning: no files found matching 'Fray'' under directory ''Max' warning: no files found matching '*.txt' under directory ''Max' writing manifest file 'MANIFEST' problem is rather simple but i don't know how to solve it with simple methods:) do you? Best regards, DimonB

1 0

Is a version number required in the setup script?
by Thomas Heller 17 Mar '01

17 Mar '01

It has been reported by Riaan Booysen that executing 'python setup.py bdist_wininst' with a setup-script not defining a version number dies with a traceback. The reason is that distutils uses '???' as version number, and xxx-???.win32.zip is an invalid filename on windows: c:\util\zip.exe -rq C:\temp\test\build\bdist.win32\wininst\x-???.win32.zip . zip error: Invalid argument zip error: Could not create output file (C:/temp/test/build/bdist.win32/wininst/x-???.win32.zip) creating 'C:\temp\test\build\bdist.win32\wininst\x-???.win32.zip' and adding '.' to it Exception exceptions.AttributeError: "'ZipFile' instance has no attribute 'fp'" in <method ZipFile.__del__ of ZipFile instance at 00885B24> ignored error: C:\temp\test\build\bdist.win32\wininst\x-???.win32.zip: Invalid argument IMO this should be fixed so that either a check is done for a version number - or - a version number of 0.0.0 should be used. Thomas

3 3

PEP 241 draft
by Andrew Kuchling 17 Mar '01

17 Mar '01

[Followups set to distutils-sig(a)python.org, since that's where most of the posters are.] I've checked in a first draft of PEP 241. Here it is; please offer comments and changes. PEP: 241 Title: Metadata for Python Software Packages Version: $Revision: 1.1 $ Author: A.M. Kuchling <amk1(a)bigfoot.com> Type: Standards Track Created: 12-Mar-2001 Status: Draft Post-History: Introduction This PEP specifies the names and semantics of the fields used to store metadata about Python software packages. Including Metadata in Packages The Distutils 'sdist' command will be modified to extract the metadata fields from the arguments and write them to a file in the generated zipfile or tarball. This file will be named METADATA and will be placed in the top directory of the source distribution (where the README, INSTALL, and other files usually go). Authors might expect to include a METADATA file of their own that would be used instead of the generated file, but this will not be permitted. It would be confusing if person B makes a custom release of person A's software, modifies setup.py accordingly, but uses identical metadata because person B didn't delete the METADATA file. When running the 'sdist' command, a user-supplied METADATA file will be ignored and a warning about this action will be printed. XXX are we sure RFC-822 is enough? The METADATA file format is a single set of RFC-822 headers parseable by the rfc822.py module. The field names listed in the following section are used as the header names. Fields This section specifies the names and semantics of each of the supported metadata fields. Name The name of the package. XXX what's the set of legal characters? Example: 'BeagleVote' Version A string containing the package's version number. This field should be parseable by one of the Version classes (StrictVersion or LooseVersion) in the distutils.version module. Example: '1.0a2' Platforms A (XXX whitespace? comma?)-separated list of platform specifications. Platform specifications are limited to the following list: XXX copy list from PPD? SourceForge? Example: 'XXX' Description A one-line summary of what the package does. Example: "A module for collecting votes from beagles." Long-Description (optional) A longer description of the package that can run to several paragraphs. (Software that deals with metadata should not assume any maximum size for this field, though one hopes that people won't include their instruction manual as the long-description.) Example: 'This module collects votes from beagles in order to determine their electoral wishes. Do NOT try to use this module with basset hounds; it makes them grumpy.' Keywords A list of additional keywords to be used to assist searching for this package in a larger catalog. XXX Keywords should probably be constrained to be from a fixed list, but I don't think this can be enforced by the 'sdist' command. (The list might be large, and it could only be updated with new Distutils releases, which seems too inflexible.) Example: 'dog puppy voting election' Home-page (optional) A string containing the URL for the package's home page. Example: 'http://www.example.com/~cschultz/bvote/' Author (optional) A string containing at a minimum the author's name. Contact information can also be added, separating each line with newlines. Example: 'C. Schultz\nUniversal Features Syndicate\nLos Angeles, CA' Author-email A string containing the author's e-mail address. It can contain a name and e-mail address in the legal forms for a RFC-822 'From:' header ("name <email>" or "email (name)"). It's not optional because cataloging systems can use the e-mail portion of this field as a unique key representing the author. A catalog might provide authors the ability to store their GPG key, personal home page, and other additional metadata *about the author*. Author-related metadata fields are not covered by this PEP. (XXX should they be? I think not, since nothing in this PEP will deal with author data at all.) Example: 'C. Schultz <cschultz(a)example.com>' License A string selected from a short list of choices, specifying the license covering the package. Some licenses result in the software being freely redistributable, so packagers and resellers can automatically know that they're free to redistribute the software. Other licenses will require a careful reading by a human to determine the software can be repackaged and resold. The choices are: XXX copy list from SourceForge, + 'Other' Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End:

10 26

Re: [Distutils] Metadata fields
by Jack Jansen 17 Mar '01

17 Mar '01

Recently, "M.-A. Lemburg" <mal(a)lemburg.com> said: > Jack Jansen wrote: > > > > Could we add a field that would allow for the functionality that is > > now in the (very little known) checkversion.py script? > > Doesn't the Version field have this information ? The version field has half the info. The other half needed is an easy way to find the most recent version. With a whole file of metainformation it's probably the easiest to just make the current metainformationfile readily accessible on the net (i.e. not only in an installer archive) and make the URL "permanent". Actually the URL doesn't even have to be permanent: if the version 1.0 metainfofile has a URL www.veryoldsite.com/bla/package/metainfo.xml, and that file (for version 2.0) has www.newersite.com/bla/package/metainfo.xml you could keep a chain of the things. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

3 2

Re: [Distutils] Metadata fields
by Jack Jansen 15 Mar '01

15 Mar '01

Could we add a field that would allow for the functionality that is now in the (very little known) checkversion.py script? I think that it would be good enough if the metadata file contained the URL where the current metadata file is stored. What checkversion would then do is run through your Python installation looking for metadata files, and for each one found it would retrieve the current version of that file over the net. If the version number in the net-based file was higher than the one in the file on your disk it would tell you there's a new version of the package available (and probably offer to download it, etc). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

3 2

Re: Distutils-SIG digest, Vol 1 #478 - 13 msgs
by Alexandre Fayolle 15 Mar '01

15 Mar '01

Andrew Kuchling <akuchlin(a)mems-exchange.org> wrote : > On Wed, Mar 14, 2001 at 12:16:03PM -0800, Paul Prescod wrote: > >I would suggest an algorithmic transformation from metadata to DTD > >(perhaps performed by hand). For instance turn every metadata item into > >an element of the same name with a value attribute or content. Then > >there are no decisions to be made. > > Can you (or some volunteer) please write up a description of this > transformation and post it? I can't really figure out what this would > buy us, but am not quite sure what you're suggesting. You may want to check vcalsax, available at http://www.logilab.org/vcalsax/. This module reads a vcalendar file (used by korganizer and gnomecal, which looks a lot like rfc822), and emits SAX2 events so that it is possible to process the file as if it were XML, and thus, for example build a DOM tree out of it. Alexandre Fayolle -- http://www.logilab.com Narval is the first software agent available as free software (GPL). LOGILAB, Paris (France).

4 5