[Python-checkins] CVS: python/nondist/peps pep-0102.txt,NONE,1.1
Anthony Baxter
anthonybaxter@users.sourceforge.net
Thu, 10 Jan 2002 07:46:30 -0800
Update of /cvsroot/python/python/nondist/peps
In directory usw-pr-cvs1:/tmp/cvs-serv25519
Added Files:
pep-0102.txt
Log Message:
adding PEP-0102, how to do a minor release. This is a trimmed down version
of PEP-0101 to only include the minor release steps.
--- NEW FILE: pep-0102.txt ---
PEP: 102
Title: Doing Python Releases 102 - The Minor Releases
Version: $Revision: 1.1 $
Last-Modified: $Date: 2002/01/10 15:46:28 $
Author: barry@zope.com (Barry A. Warsaw), guido@python.org (Guido van Rossum), anthony@interlink.com.au (Anthony Baxter)
Status: Active
Type: Informational
Created: 22-Aug-2001 (edited down on 9-Jan-2002 to become pep-0102)
Post-History:
Abstract
Making a Python release is an arduous processes that takes a
minimum of half a day's work even for an experienced releaser.
Until recently, most -- if not all -- of that burden was borne by
Guido himself. But several recent releases have been performed by
other folks, so this PEP attempts to collect, in one place, all
the steps needed to make a Python bugfix release. The major Python
release process is covered in PEP 101 - this PEP is just 101, trimmed
down to only include the bits that are relevant for minor releases.
It is organized as a recipe and you can actually print this out and
check items off as you complete them.
How to Make A Release
Here are the steps taken to make a Python release. Some steps are
more fuzzy than others because there's little that can be
automated (e.g. writing the NEWS entries). Where a step is
usually performed by An Expert, the name of that expert is given.
Otherwise, assume the step is done by the Release Manager (RM),
the designated person performing the release. Almost every place
the RM is mentioned below, this step can also be done by the BDFL
of course!
XXX: We should include a dependency graph to illustrate the steps
that can be taken in parallel, or those that depend on other
steps.
We use the following conventions in the examples below. Where a
release number is given, it is of the form X.Y.MaA, e.g. 2.1.2rc1 for
Python 2.1.2 release candidate 1, where "a" == alpha, "b" == beta,
"rc" == release candidate. Final releases are tagged with "releaseXYZ"
in CVS. The minor releases are made from the maintenance branch of the
major release, e.g. Python 2.1.2 is made from the release21-maint
branch.
___ Send an email to python-dev@python.org indicating the release is
about to start.
___ Put a freeze on check ins into the maintenance branch. At this
point, nobody except the RM should make any commits to the branch
(or his duly assigned agents, i.e. Guido the BDFL, Fred Drake for
documentation, or Tim Peters for Windows). If the RM screwed up
and some desperate last minute change to the branch is
necessary, it can mean extra work for Fred and Tim. So try to
avoid this!
___ On the branch, change Include/patchlevel.h in two places, to
reflect the new version number you've just created. You'll want
to change the PY_VERSION macro, and one or several of the
version subpart macros just above PY_VERSION, as appropriate.
___ After starting the process, the most important thing to do next
is to update the Misc/NEWS file. Tim will need this in order to
do the Windows release and he likes to stay up late. This step
can be pretty tedious, so it's best to get to it immediately
after making the branch, or even before you've made the branch.
The sooner the better (but again, watch for new checkins up
until the release is made!)
Add high level items new to this release. E.g. if we're
releasing 2.2a3, there must be a section at the top of the file
explaining "What's new in Python 2.2a3". It will be followed by
a section entitled "What's new in Python 2.2a2".
Note that you /hope/ that as developers add new features to the
trunk, they've updated the NEWS file accordingly. You can't be
positive, so double check. If you're a Unix weenie, it helps to
verify with Tim Peters about changes on Windows, and Jack Jansen
about changes on the Mac.
This command should help you:
% cvs log -rr22a1: | python Tools/scripts/logmerge.py > /tmp/news.txt
IOW, you're printing out all the cvs log entries from the
previous release until now. You can then troll through the
news.txt file looking for interesting things to add to NEWS.
___ Check your NEWS changes into the maintenance branch.
___ Once the release is underway, Fred Drake needs to create the
HTML from the documentation. He does this and uploads the file to
www.python.org. Then he tells Tim Peters where this file is.
This may generate some last minute changes on the branch. Once
Fred is done, there can be no further checkins on the branch in
the Doc/ directory -- not even by the RM. For final releases,
Fred also sends email to Milan Zamazal for conversion to the GNU
Info format.
Note that Fred is responsible both for merging doc changes from
the trunk to the branch AND for merging any branch changes from
the branch to the trunk during the cleaning up phase.
Basically, if it's in Doc/ Fred will take care of it.
___ Tim Peters grabs the HTML and uses this to build the Windows
installer. Tim then creates a new "release" named X.YaZ on the
SourceForge file release manager. (Although, if you get there
first, you should create the new release.)
(Diversion: SF's file manager has "packages" and "releases". We
use packages to name major upcoming releases, e.g. python-2.2 or
python-2.1.1. Inside each package are a number of "releases"
for each new actual release -- i.e. the thing you're building.
An example of a release name is 2.2a3. Once created, packages
and releases are never deleted, but old ones are hidden to
reduce confusion. More on this below.)
If this is the first release for this major Python version, Tim
will create a new package containing the major Python version
number.
___ Tim performs his Windows magic, generating an installer
executable. He uploads this file to SourceForge under the
release he just created. He then sends the RM a notice which
includes the MD5 checksum of the Windows executable.
Note that Tim's creation of the Windows executable may generate
a few more commits on the branch. Tim will be responsible for
merging Windows-specific changes from trunk to branch, and from
branch to trunk.
___ It's Build Time!
Now, you're ready to build the source tarball. First cd to your
working directory for the branch. E.g.
% cd .../python-22a3
___ Do a "cvs update" in this directory. Do NOT include the -A flag!
You should not see any "M" files, but you may see several "P"
files. I.e. you better not have any uncommitted changes in your
working directory, but you may pick up some of Fred's or Tim's
last minute changes.
___ Now tag the branch using a symbolic name like "rXYMaZ",
e.g. r212
% cvs tag r212
___ Change to a neutral directory, i.e. one in which you can do a
fresh, virgin, cvs export of the branch. You will be creating a
new directory at this location, to be named "Python-X.Y.M". Do
a CVS export of the tagged branch.
% cd ~
% cvs -d <cvsroot> export -rr212 -d Python-2.1.2 python/dist/src
___ Generate the tarball. Note that we're not using the `z' option
on the tar command because 1) that's only supported by GNU tar
as far as we know, and 2) we're going to max out the compression
level, which isn't a supported option.
% tar cf - Python-2.1.2 | gzip -9 > Python-2.1.2.tgz
___ Calculate the MD5 checksum of the tgz file you just created
% md5sum Python-2.1.2.tgz
Note that if you don't have the md5sum program, there is a
Python replacement in the Tools/scripts/md5sum.py file.
___ Now you want to perform the very important step of checking the
tarball you just created, to make sure a completely clean,
virgin build passes the regression test. Here are the best
steps to take:
% cd /tmp
% tar zxvf ~/Python-2.1.2.tgz
% cd Python-2.1.2
% ls
(Do things look reasonable?)
% ./configure
(Loads of configure output)
% make test
(Do all the expected tests pass?)
If the tests pass, then you can feel good that the tarball is
fine. If some of the tests fail, or anything else about the
freshly unpacked directory looks weird, you better stop now and
figure out what the problem is.
___ Start your upload to SF. You need to get Python-2.1.2.tgz into
SourceForge. This can take a while both because of the time it
takes to upload such a huge file, /and/ because SF has a 30
minute delay built into the file release process. The next few
steps can be taken in parallel, so it's best to start the upload
now and keep an eye on its progress.
I've found that the `ncftpput' program is a great tool to use if
you have it available. You can execute the following command to
do the upload:
% ncftpput upload.sf.net incoming Python-2.1.2.tgz
If you don't have ncftpput around, you can use whatever ftp
client you're comfortable with. Just be sure that you're
uploading this to the "incoming" directory on upload.sf.net.
___ You also need to upload the tgz file to creosote.python.org.
Usually Tim will have already uploaded the exe file to creosote,
but if not, you'll need to do that too. These steps can take a
long time depending on your network bandwidth. You have two
choices:
1) Upload them to SF first, then wget them from creosote. Pros:
easy to do; much friendlier to your own personal bandwidth.
Cons: can take even longer because you're subject to the 30
minute SF file upload delay, and the upload rate from
SF->creosote never seems to get above 20 KB/sec.
2) scp both files from your own machine to creosote. Pros: you
avoid the 30 minute SF delay. Cons: you don't get much else
done if you're on a small pipe.
I usually opt for #2.
___ While you're waiting, you can start twiddling the web pages to
include the announcement.
___ In the python.org web site CVS tree, cd to the X.Y
subdirectory, and copy index.ht to new-index.ht. Be sure to
do a "cvs update" first!
% cd .../pydotorg
% cvs -q up -P -d
% cd 2.2
% cp index.ht new-index.ht
___ Edit the file for content: usually you can globally replace
X.Ya(Z-1) with X.YaZ. However, you'll need to think about the
"What's New?" section. You also need to watch out about two
specific links in the file, one which references "the full
scoop" on changes -- a shownotes link, and one which
references the downloads on SF -- a showfiles link. You won't
be able to update either link until you've completed the steps
below. For now just note what needs to change.
___ Also, update the MD5 checksums.
___ Preview the web page by doing a "make" -- NOT a "make install".
View the page via a file: url.
___ Similarly, edit the ../index.ht file, i.e. the python.org home
page. In the Big Blue Announcement Block, move the paragraph
for the new version up to the top and boldify the phrase
"Python X.YaZ is out". Edit for content, and preview as
above. Do NOT do a "make install" yet!
___ Now we're waiting for the ncftpput command, and the scp to
creosote to finish. Da de da, da de dum, hmm, hmm, dum de dum.
___ Do the SourceForge file release dance.
___ Go to the Python project and click on "Admin"
___ Click on "Edit/Release Files"
___ Since Tim has usually by now created the package and release
we're going to use, scroll down and click on "Edit Releases"
for the package we're releasing into.
___ Find the release named X.YaZ and click on "Edit This Release"
You should now perform Step 1 of the file release dance...
___ The "Status" field should be "Active" not "Hidden"
___ In the text box that says "Paste The Notes In", paste in all
the "What's New" entries from the Misc/NEWS file that describe
this major version of Python, /not/ just the ones for this
particular release. E.g. If we're releasing Python 2.2a3,
we'd include the "What's New" sections for Python 2.2a3,
2.2a2, and 2.2a1.
___ Leave the "Paste The Change Log In" section blank, but click
on "Preserve my pre-formatted text".
___ Hit the Submit/Refresh button for Step 1.
This will bring you back to the file release page. DO NOT do
the following step until your ftp upload is complete! Once it
is, you can perform Step 2 of the file release dance...
___ Click on the checkbox next to the file Python-X.YaZ.tgz. Be
sure no other box is checked! Then click on the "Add Files
and/or Refresh View" button for Step 2.
And now, Step 3...
___ There should be exactly two files listed here, one is the tgz
file you just added, and the other is the exe file that Tim
added earlier.
___ For the tgz file, be sure that the "Processor" field says
"Any" and the "File Type" field says "Source .gz".
___ Click on "Update/Refresh" for the .tgz file.
___ For the exe file, make sure that the "Processor" field says
"i386" and the "File Type" field says "Other". Tim usually
gets this right <wink>, but if not, make any necessary changes
and click on "Update/Refresh" for the exe file.
Step 4...
DO NOT DO STEP 4 NOW. Wait until after you send out the email
announcement to send the SF email notice.
___ Still on SF, click on the "Files" button at the top of the
page. Find the release you've just made and click on it -- not
on the tgz or exe file, but on the release link under the
package name. E.g. package named python-2.2, click on the
"2.2a3" link.
This should be a page that says "Release Name: X.YaZ" and it
should contain the "What's New" sections you pasted in earlier.
Note the url of this page. Copy and paste it into the
pydotorg/X.Y/new-index.ht file you created above. This is the
"shownotes" link mentioned earlier.
___ Now click on the "Summary" link at the top of the page and
scroll down to the "Latest File Releases" section. Find the
package you just made a release for (the Version should be
X.YaZ, and the Date should be today's date). Click on the
"Download" link.
Your new release should be highlighted in pink. Note the url
for this page. Copy and paste it into the
pydotorg/X.Y/new-index.ht file from above. This is the
"showfiles" link mentioned earlier.
___ Now you need to go to creosote.python.org and move all the files
in place over there. Our policy is that every Python version
gets its own directory, but each directory may contain several
releases. We keep all old releases, moving them into a "prev"
subdirectory when we have a new release.
So, there's a directory called "2.2" which contains
Python-2.2a2.exe and Python-2.2a2.tgz, along with a "prev"
subdirectory containing Python-2.2a1.exe and Python-2.2a1.tgz.
So...
___ On creosote, cd to ~ftp/pub/python/X.Y creating it if
necessary.
___ Move the previous release files to a directory called "prev"
creating the directory if necessary (make sure the directory
has g+ws bits on). If this is the first alpha release of a
new Python version, skip this step.
___ Move the .tgz file and the .exe file to this directory. Make
sure they are world readable. They should also be group
writable, and group-owned by webmaster.
___ md5sum the files and make sure they got uploaded intact.
___ Update the X.Y/bugs.ht file if necessary. It is best to get
BDFL input for this step.
___ Now preview the new-index.ht file once more. IMPORTANT: follow
every link on the page to make sure it goes where you expect it
to go, and that what you expect to be there is there.
___ If everything looks good, move new-index.ht to index.ht and do a
"make install" in this directory. Go up to the parent directory
(i.e. the root of the web page hierarchy) and do a "make
install" there too. You're release is now live!
___ Now it's time to write the announcement for the mailing lists.
This is the fuzzy bit because not much can be automated. You
can use one of Guido's earlier announcements as a template, but
please edit it for content!
Once the announcement is ready, send it to the following
addresses:
python-list@python.org
python-announce@python.org
python-dev@python.org
___ Go back to the file releases page on SF and complete Step 4,
sending out the email notification.
___ Send a SourceForge News Item about the release. From the
project's "menu bar", select the "News" link; once in News,
select the "Submit" link. Type a suitable subject (e.g. "Python
2.2c1 released" :-) in the Subject box, add some text to the
Details box (at the very least including the release URL at
www.python.org and the fact that you're happy with the release)
and click the SUBMIT button.
Feel free to remove any old news items.
Now it's time to do some cleanup. These steps are very important!
___ Go back to SF, Admin->Edit/Release Files. Click on "Edit
Releases" for the package you just added to. For each old
release, click on "Edit This Release" and under Step 1, change
the "Status" to "Hidden". Click on the Step 1 Submit/Refresh
button.
___ Edit the file Include/patchlevel.h so that the PY_VERSION
string says something like "X.YaZ+". Note the trailing `+'
indicating that the trunk is going to be moving forward with
development. E.g. the line should look like:
#define PY_VERSION "2.1.2+"
Make sure that the other PY_ version macros contain the
correct values. Commit this change.
___ For the extra paranoid, do a completely clean test of the
release. This includes downloading the tarball from either
SourceForge or www.python.org.
___ Make sure the md5 checksums match. Then unpack the tarball,
and do a clean make test.
% make distclean
% ./configure
% make test
To ensure that the regression test suite passes. If not, you
screwed up somewhere!
Step 5 ...
Verify! This can be interleaved with Step 4. Pretend you're a
user: download the files from python.org *and* SourceForge, and make
Pythons from them. This step is too easy to overlook, and on
several occasions we've had useless release files. Once a general
server problem caused mysterious corruption of all files; once the
source tarball got built incorrectly; more than once the file upload
process on SF truncated files; and so on.
What Next?
Rejoice. Drink. Be Merry. Write a PEP like this one. Or be
like unto Guido and take A Vacation.
You've just made a Python release!
Actually, there is one more step. You should turn over ownership
of the branch to Jack Jansen. All this means is that now he will
be responsible for making commits to the branch. He's going to
use this to build the MacOS versions. He may send you information
about the Mac release that should be merged into the informational
pages on SourceForge or www.python.org. When he's done, he'll
tag the branch something like "rX.YaZ-mac". He'll also be
responsible for merging any Mac-related changes back into the
trunk.
Final Release Notes
The Final release of any major release, e.g. Python 2.2 final, has
special requirements, specifically because it will be one of the
longest lived releases (i.e. betas don't last more than a couple
of weeks, but final releases can last for years!).
For this reason we want to have a higher coordination between the
three major releases: Windows, Mac, and source. The Windows and
source releases benefit from the close proximity of the respective
release-bots. But the Mac-bot, Jack Jansen, is 6 hours away. So
we add this extra step to the release process for a final
release:
___ Hold up the final release until Jack approves, or until we
lose patience <wink>.
Windows Notes
Windows has a GUI installer, various flavors of Windows have
"special limitations", and the Windows installer also packs
precompiled "foreign" binaries (Tcl/Tk, expat, etc). So Windows
testing is tiresome but very necessary.
Concurrent with uploading the installer, Tim installs Python from
it twice: once into the default directory suggested by the
installer, and later into a directory with embedded spaces in its
name. For each installation, he runs the full regression suite
from a DOS box, and both with and without -0.
He also tries *every* shortcut created under Start -> Menu -> the
Python group. When trying IDLE this way, you need to verify that
Help -> Python Documentation works. When trying pydoc this way
(the "Module Docs" Start menu entry), make sure the "Start
Browser" button works, and make sure you can search for a random
module (Tim uses "random" <wink>) and then that the "go to
selected" button works.
It's amazing how much can go wrong here -- and even more amazing
how often last-second checkins break one of these things. If
you're "the Windows geek", keep in mind that you're likely the
only person routinely testing on Windows, and that Windows is
simply a mess.
Repeat all of the above on at least one flavor of Win9x, and one
of NT/2000. On NT/2000, try both an Admin and a plain User (not
Power User) account.
WRT Step 5 above (verify the release media), since by the time
release files are ready to download Tim has generally run many
Windows tests on the installer he uploaded, he usually doesn't do
anything for Step 5 except a full byte-comparison ("fc /b" if
using a Windows shell) of the downloaded file against the file he
uploaded.
Copyright
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: