SYNOPSIS

This document describes the series of tasks required - some automatic, some manual - to produce a perl release of some description, be that a release candidate, or final, numbered release of maint or blead.

The release process has traditionally been executed by the current pumpking. Blead releases from 5.11.0 forward are made each month on the 20th by a non-pumpking release engineer. The release engineer roster and schedule can be found in Porting/release_schedule.pod.

This document both helps as a check-list for the release engineer and is a base for ideas on how the various tasks could be automated or distributed.

The checklist of a typical release cycle is as follows:

(5.10.1 is released, and post-release actions have been done)
...time passes...
a few weeks before the release, a number of steps are performed,
including bumping the version to 5.10.2
...a few weeks passes...
perl-5.10.2-RC1 is released
perl-5.10.2 is released
post-release actions are performed, including creating new
perldelta.pod
... the cycle continues ...

DETAILS

Some of the tasks described below apply to all four types of release of Perl. (blead, RC, final release of maint, final release of blead). Some of these tasks apply only to a subset of these release types. If a step does not apply to a given type of release, you will see a notation to that effect at the beginning of the step.

Release types

Release Candidate (RC)

A release candidate is an attempt to produce a tarball that is a close as possible to the final release. Indeed, unless critical faults are found during the RC testing, the final release will be identical to the RC barring a few minor fixups (updating the release date in perlhist.pod, removing the RC status from patchlevel.h, etc). If faults are found, then the fixes should be put into a new release candidate, never directly into a final release.

Stable/Maint release (MAINT).

A release with an even version number, and subversion number > 0, such as 5.14.1 or 5.14.2.

At this point you should have a working release candidate with few or no changes since.

It's essentially the same procedure as for making a release candidate, but with a whole bunch of extra post-release steps.

A blead point release (BLEAD-POINT)

A release with an odd version number, such as 5.15.0 or 5.15.1.

This isn't for production, so it has less stability requirements than for other release types, and isn't preceded by RC releases. Other than that, it is similar to a MAINT release.

Blead final release (BLEAD-FINAL)

A release with an even version number, and subversion number == 0, such as 5.14.0. That is to say, it's the big new release once per year.

It's essentially the same procedure as for making a release candidate, but with a whole bunch of extra post-release steps, even more than for MAINT.

Prerequisites

Before you can make an official release of perl, there are a few hoops you need to jump through:

PAUSE account with pumpkin status

Make sure you have a PAUSE account suitable for uploading a perl release. If you don't have a PAUSE account, then request one:

https://pause.perl.org/pause/query?ACTION=request_id

Check that your account is allowed to upload perl distros: go to https://pause.perl.org/pause/authenquery?ACTION=who_pumpkin and check that your PAUSE ID is listed there. If not, ask Andreas König to add your ID to the list of people allowed to upload something called perl. You can find Andreas' email address at:

https://pause.perl.org/pause/query?ACTION=pause_04imprint

search.cpan.org pumpkin status

Make sure that search.cpan.org knows that you're allowed to upload perl distros. Contact Graham Barr to make sure that you're on the right list.

rt.perl.org update access

Make sure you have permission to close tickets on http://rt.perl.org/ so you can respond to bug report as necessary during your stint. If you don't, make an account (if you don't have one) and contact the pumpking with your username to get ticket-closing permission.

git checkout and commit bit

You will need a working git installation, checkout of the perl git repository and perl commit bit. For information about working with perl and git, see pod/perlgit.pod.

If you are not yet a perl committer, you won't be able to make a release. Have a chat with whichever evil perl porter tried to talk you into the idea in the first place to figure out the best way to resolve the issue.

git clone of https://github.com/perlorg/perlweb

For updating the http://dev.perl.org web pages, either a Github account or sweet-talking somebody with a Github account into obedience is needed. This is only needed on the day of the release or shortly afterwards.

Quotation for release announcement epigraph

SKIP this step for RC

For all except an RC release of perl, you will need a quotation to use as an epigraph to your release announcement.

Building a release - advance actions

The work of building a release candidate for an even numbered release (BLEAD-FINAL) of perl generally starts several weeks before the first release candidate. Some of the following steps should be done regularly, but all must be done in the run up to a release.

dual-life CPAN module synchronisation

To see which core distro versions differ from the current CPAN versions:

$ ./perl -Ilib Porting/core-cpan-diff -x -a

Passing -u cpan (and maybe -u undef) will probably be helpful, since those are the only types of distributions that you can actually affect as a perl release manager (as opposed to a CPAN module maintainer).

You can also run an actual diff of the contents of the modules, comparing core to CPAN, to ensure that there were no erroneous/extraneous changes that need to be dealt with. You do this by not passing the -x option:

$ ./perl -Ilib Porting/core-cpan-diff -a -o /tmp/corediffs

then fix the core, or cajole CPAN authors as appropriate. See also the -d and -v options for more detail (and the -u option as mentioned above). You'll probably want to use the -c cachedir option to avoid repeated CPAN downloads and may want to use -m file:///mirror/path if you made a local CPAN mirror. Note that a minicpan mirror won't actually work, but can provide a good first pass to quickly get a list of modules which definitely haven't changed, to avoid having to download absolutely everything.

If you are making a MAINT release, run core-cpan-diff on both blead and maint, then diff the two outputs. Compare this with what you expect, and if necessary, fix things up. For example, you might think that both blead and maint are synchronised with a particular CPAN module, but one might have some extra changes.

How to sync a CPAN module with a cpan/ distro

Fetch the most recent version from CPAN.

Unpack the retrieved tarball. Rename the old directory; rename the new directory to the original name.

Restore any .gitignore file. This can be done by issuing git checkout .gitignore in the cpan/Distro directory.

Remove files we do not need. That is, remove any files that match the entries in @IGNORE in Porting/Maintainer.pl, and anything that matches the EXCLUDED section of the distro's entry in the %Modules hash.

Restore any files mentioned in the CUSTOMIZED section, using git checkout. Make any new customizations if necessary. Also, restore any files that are mentioned in @IGNORE, but were checked in in the repository anyway.

For any new files in the distro, determine whether they are needed. If not, delete them, and list them in either EXCLUDED or @INGORE. Otherwise, add them to MANIFEST, and run git add to add the files to the repository.

For any files that are gone, remove them from MANIFEST, and use git rm to tell git the files will be gone.

If the MANIFEST file was changed in any of the previous steps, run perl Porting/manisort --output MANIFEST.sort; mv MANIFEST.sort MANIFEST.

For any files that have an execute bit set, either remove the execute bit, or edit Porting/exec-bit.txt

Run make (or nmake on Windows), see if perl compiles.

Run the tests for the package.

Run the tests in t/porting.

Update the DISTRIBUTION entry in Porting/Maintainers.pl.

Run a full configure/build/test cycle.

If everything is ok, commit the changes.

For entries with a non-simple FILES section, or with a MAP, you may have to take more steps than listed above.

Porting/sync-with-cpan is a script that automates most of the steps above; but see the comments at the beginning of the file. In particular, it has not yet been exercised on Windows, but will certainly require a set of Unix tools such as Cygwin, and steps that run make will need to run nmake instead.

dual-life CPAN module stability

Ensure dual-life CPAN modules are stable, which comes down to:

for each module that fails its regression tests on $current
did it fail identically on $previous?
if yes, "SEP" (Somebody Else's Problem)
else work out why it failed (a bisect is useful for this)
attempt to group failure causes
for each failure cause
is that a regression?
if yes, figure out how to fix it
(more code? revert the code that broke it)
else
(presumably) it's relying on something un-or-under-documented
should the existing behaviour stay?
yes - goto "regression"
no - note it in perldelta as a significant bugfix
(also, try to inform the module's author)

Similarly, monitor the smoking of perl for compiler warnings, and try to fix.

update perldelta

Get perldelta in a mostly finished state.

Read Porting/how_to_write_a_perldelta.pod, and try to make sure that every section it lists is, if necessary, populated and complete. Copy edit the whole document.

Bump the version number

Do not do this yet for a BLEAD-POINT release! You will do this at the end of the release process.

Increase the version number (e.g. from 5.12.0 to 5.12.1).

For a release candidate for a stable perl, this should happen a week or two before the first release candidate to allow sufficient time for testing and smoking with the target version built into the perl executable. For subsequent release candidates and the final release, it it not necessary to bump the version further.

There is a tool to semi-automate this process:

$ ./perl -Ilib Porting/bump-perl-version -i 5.10.0 5.10.1

Remember that this tool is largely just grepping for '5.10.0' or whatever, so it will generate false positives. Be careful not change text like "this was fixed in 5.10.0"!

Use git status and git diff to select changes you want to keep.

Be particularly careful with INSTALL, which contains a mixture of 5.10.0-type strings, some of which need bumping on every release, and some of which need to be left unchanged. The line in INSTALL about "is binary incompatible with" requires a correct choice of earlier version to declare incompatibility with.

When doing a BLEAD-POINT or BLEAD-FINAL release, also make sure the PERL_API_* constants in patchlevel.h are in sync with the version you're releasing, unless you're absolutely sure the release you're about to make is 100% binary compatible to an earlier release. When releasing a MAINT perl version, the PERL_API_* constants MUST NOT be changed as we aim to guarantee binary compatibility in maint branches.

After editing, regenerate uconfig.h (this must be run on a system with a /bin/sh available):

$ perl regen/uconfig_h.pl

This might not cause any new changes.

Test your changes:

$ git clean -xdf # careful if you don't have local files to keep!
$ ./Configure -des -Dusedevel
$ make
$ make test

re-check earlier actions

create a release branch

For BLEAD-POINT releases, making a release from a release branch avoids the need to freeze blead during the release. This is less important for BLEAD-FINAL, MAINT, and RC releases, since blead will already be frozen in those cases. Create the branch by running

git checkout -b release-5.xx.yy

finalize perldelta

Finalize the perldelta. In particular, fill in the Acknowledgements section, which can be generated with something like:

$ perl Porting/acknowledgements.pl v5.15.0..HEAD

Re-read the perldelta to try to find any embarrassing typos and thinkos; remove any TODO or XXX flags; update the "Known Problems" section with any serious issues for which fixes are not going to happen now; and run through pod and spell checkers, e.g.

remove stale perldeltas

For the first RC release that is ONLY for a BLEAD-FINAL, the perldeltas from the BLEAD-POINT releases since the previous BLEAD_FINAL should have now been consolidated into the current perldelta, and hence are now just useless clutter. They can be removed using:

$ git rm <file1> <file2> ...

For example, for RC0 of 5.16.0:

$ cd pod
$ git rm perldelta515*.pod

All mention to them should also be removed. Edit pod/perl.pod to remove them from its table of contents, then run Porting/pod_rules.pl to propagate your changes there into all the other files that mention them (including MANIFEST). You'll need to git add the files that it changes.

build a clean perl

If you skipped the previous step (removing the stale perldeltas) make sure you have a gitwise-clean perl directory (no modified files, unpushed commits etc):

$ git status
$ git clean -dxf

then configure and build perl so that you have a Makefile and porting tools:

$ ./Configure -Dusedevel -des && make

update Module::CoreList

Update Module::CoreList with module version data for the new release.

Note that if this is a MAINT release, you should run the following actions from the maint branch, but commit the CoreList.pm changes in blead and subsequently cherry-pick any releases since the last maint release and then your recent commit. XXX need a better example

corelist.pl uses ftp.funet.fi to verify information about dual-lived modules on CPAN. It can use a full, local CPAN mirror and/or fall back on HTTP::Tiny to fetch package metadata remotely.

(If you'd prefer to have a full CPAN mirror, see http://www.cpan.org/misc/cpan-faq.html#How_mirror_CPAN)

Then change to your perl checkout, and if necessary,

$ make

If this is not the first update for this version (e.g. if it was updated when the version number was originally bumped), first edit dist/Module-CoreList/lib/Module/CoreList.pm to delete the existing entries for this version from the %released and %version hashes: they will have a key like 5.010001 for 5.10.1.

XXX the edit-in-place functionality of Porting/corelist.pl should be fixed to handle this automatically.

Then, If you have a local CPAN mirror, run:

$ ./perl -Ilib Porting/corelist.pl ~/my-cpan-mirror

Otherwise, run:

$ ./perl -Ilib Porting/corelist.pl cpan

This will chug for a while, possibly reporting various warnings about badly-indexed CPAN modules unrelated to the modules actually in core. Assuming all goes well, it will update dist/Module-CoreList/lib/Module/CoreList.pm.

Check that file over carefully:

$ git diff dist/Module-CoreList/lib/Module/CoreList.pm

Bump $Module::CoreList::VERSION

If necessary, bump $Module::CoreList::VERSION (there's no need to do this for every RC; in RC1, bump the version to a new clean number that will appear in the final release, and leave as-is for the later RCs and final). It may also happen that Module::CoreList has been modified in blead, and hence has a new version number already. (But make sure it is not the same number as a CPAN release.)

Edit the version number in the new 'Module::CoreList' => 'X.YZ' entry, as that is likely to reflect the previous version number.

Bump $Module::CoreList::TieHashDelta::VERSION

$Module::CoreList::TieHashDelta::VERSION should always be equal to $Module::CoreList::VERSION. Make sure the two versions match before proceeding.

Edit the version number in the new 'Module::CoreList::TieHashDelta' => 'X.YZ' entry, as that is likely to reflect the previous version number.

Bump version in Module::CoreList Changes

Also edit Module::CoreList's new version number in its Changes file.

Add Module::CoreList version bump to perldelta

Add a perldelta entry for the new Module::CoreList version.

Update %Module::CoreList::released and CAVEATS

In addition, if this is a final release (rather than a release candidate):

Update this version's entry in the %released hash with today's date.

Make sure that the script has correctly updated the CAVEATS section (Note, the CAVEATS section is in dist/Module-CoreList/lib/Module/CoreList.pod)

Commit Module::CoreList changes

Finally, commit the new version of Module::CoreList: (unless this is for MAINT; in which case commit it to blead first, then cherry-pick it back).

update perlhist.pod

You MUST SKIP this step for a RC release

Add an entry to pod/perlhist.pod with the release date, e.g.:

David 5.10.1 2009-Aug-06

Make sure that the correct pumpking is listed in the left-hand column, and if this is the first release under the stewardship of a new pumpking, make sure that his or her name is listed in the section entitled THE KEEPERS OF THE PUMPKIN.

If you're making a BLEAD-FINAL release, also update the "SELECTED RELEASE SIZES" section with the output of Porting/perlhist_calculate.pl.

Be sure to commit your changes:

$ git commit -m 'add new release to perlhist' pod/perlhist.pod

update patchlevel.h

You MUST SKIP this step for a BLEAD-POINT release

Update patchlevel.h to add a -RC1-or-whatever string; or, if this is a final release, remove it. For example:

Check that the output of /tmp/perl-5.x.y-pretest/bin/perl -v and /tmp/perl-5.x.y-pretest/bin/perl -V are as expected, especially as regards version numbers, patch and/or RC levels, and @INC paths. Note that as they have been been built from a git working directory, they will still identify themselves using git tags and commits. (Note that for an odd-numbered version, perl will install itself as perl5.x.y). perl -v will identify itself as:

This is perl 5, version X, subversion Y (v5.X.Y (v5.X.Z-NNN-gdeadbeef))

where 5.X.Z is the latest tag, NNN the number of commits since this tag, and deadbeef commit of that tag.

Then delete the temporary installation.

push the work so far

Push all your recent commits:

$ git push origin release-5.xx.yy

tag the release

Tag the release (e.g.):

$ git tag v5.11.0 -m "First release of the v5.11 series!"

It is VERY important that from this point forward, you not push your git changes to the Perl master repository. If anything goes wrong before you publish your newly-created tag, you can delete and recreate it. Once you push your tag, we're stuck with it and you'll need to use a new version number for your release.

build the tarball

Before you run the following, you might want to install 7-Zip (the p7zip-full package under Debian or the p7zip port on MacPorts) or the AdvanceCOMP suite (e.g. the advancecomp package under Debian, or the advancecomp port on macports - 7-Zip on Windows is the same code as AdvanceCOMP, so Windows users get the smallest files first time). These compress about 5% smaller than gzip and bzip2. Over the lifetime of your distribution this will save a lot of people a small amount of download time and disk space, which adds up.

Create a tarball. Use the -s option to specify a suitable suffix for the tarball and directory name:

This creates the directory ../perl-x.y.z-RC1 or similar, copies all the MANIFEST files into it, sets the correct permissions on them, adds DOS line endings to some, then tars it up as ../perl-x.y.z-RC1.tar.gz. With -b, it also creates a tar.bz2 file.

If you're getting your tarball suffixed with -uncommitted and you're sure your changes were all committed, you can override the suffix with:

$ perl Porting/makerel -b -s ''

XXX if we go for extra tags and branches stuff, then add the extra details here

Finally, clean up the temporary directory, e.g.

$ rm -rf ../perl-x.y.z-RC1

test the tarball

Once you have a tarball it's time to test the tarball (not the repository).

Copy the tarball to a web server

Copy the tarballs (.gz and possibly .bz2) to a web server somewhere you have access to.

Download the tarball to another machine

Download the tarball to some other machine. For a release candidate, you really want to test your tarball on two or more different platforms and architectures. The #p5p IRC channel on irc.perl.org is a good place to find willing victims.

Check perl -v and perl -V

Check that the output of perl -v and perl -V are as expected, especially as regards version numbers, patch and/or RC levels, and @INC paths.

Note that the results may be different without a .git/ directory, which is why you should test from the tarball.

Run the Installation Verification Procedure utility

$ ./perl utils/perlivp
...
All tests successful.
$

Compare the installed paths to the last release

Compare the pathnames of all installed files with those of the previous release (i.e. against the last installed tarball on this branch which you have previously verified using this same procedure). In particular, look for files in the wrong place, or files no longer included which should be. For example, suppose the about-to-be-released version is 5.10.1 and the previous is 5.10.0:

and carefully examine the output (in perlbug.rep]), especially the "Locally applied patches" section. If everything appears okay, then delete the file, and try it again, this time actually submitting the bug report. Check that it shows up, then remember to close it!

monitor smokes

Wait for the smoke tests to catch up with the commit which this release is based on (or at least the last commit of any consequence).

Then check that the smoke tests pass (particularly on Win32). If not, go back and fix things.

Note that for BLEAD-POINT releases this may not be practical. It takes a long time for the smokers to catch up, especially the Win32 smokers. This is why we have a RC cycle for MAINT and BLEAD-FINAL releases, but for BLEAD-POINT releases sometimes the best you can do is to plead with people on IRC to test stuff on their platforms, fire away, and then hope for the best.

upload to PAUSE

Once smoking is okay, upload it to PAUSE. This is the point of no return. If anything goes wrong after this point, you will need to re-prepare a new release with a new minor version or RC number.

https://pause.perl.org/

(Login, then select 'Upload a file to CPAN')

If your workstation is not connected to a high-bandwidth, high-reliability connection to the Internet, you should probably use the "GET URL" feature (rather than "HTTP UPLOAD") to have PAUSE retrieve the new release from wherever you put it for testers to find it. This will eliminate anxious gnashing of teeth while you wait to see if your 15 megabyte HTTP upload successfully completes across your slow, twitchy cable modem. You can make use of your home directory on dromedary for this purpose: http://users.perl5.git.perl.org/~USERNAME maps to /home/USERNAME/public_html, where USERNAME is your login account on dromedary. Remember: if your upload is partially successful, you may need to contact a PAUSE administrator or even bump the version of perl.

Upload both the .gz and .bz2 versions of the tarball.

Do not proceed any further until you are sure that your tarballs are on CPAN. Check your authors directory on one of the "fast" CPAN mirrors (e.g., cpan.hexten.net or cpan.cpantesters.org) to confirm that your uploads have been successful.

wait for indexing

You MUST SKIP this step for RC

Wait until you receive notification emails from the PAUSE indexer confirming that your uploads have been received. IMPORTANT -- you will probably get an email that indexing has failed, due to module permissions. This is considered normal.

publish tag

Now that you've shipped the new perl release to PAUSE, it's time to publish the tag you created earlier to the public git repo (e.g.):

update epigraphs.pod

Add your quote to Porting/epigraphs.pod and commit it. Your release announcement will probably not have reached the web-visible archives yet, so you won't be able to include the customary link to the release announcement yet.

blog about your epigraph

If you have a blog, please consider writing an entry in your blog explaining why you chose that particular quote for your epigraph.

Module::CoreList nagging

You MUST SKIP this step for RC

Remind the current maintainer of Module::CoreList to push a new release to CPAN.

new perldelta

You MUST SKIP this step for RC

Create a new perldelta.

Confirm that you have a clean checkout with no local changes.

Run Porting/new-perldelta.pl

Run the git add commands it outputs to add new and modified files.

Verify that the build still works, by running ./Configure and make test_porting. (On Win32, run nmake and nmake test TEST_FILES="porting\*.t ..\lib\diagnostics.t".)

If t/porting/podcheck.t spots errors in the new pod/perldelta.pod, run ./perl -MTestInit t/porting/podcheck.t | less for more detail. Skip to the end of its test output to see the options it offers you.

When make test_porting passes, commit the new perldelta.

At this point you may want to compare the commit with a previous bump to see if they look similar. See commit 4eabcf701b for an example of a previous version bump.

bump version

You MUST SKIP this step for RC and MAINT

If this was a BLEAD-FINAL release (i.e. the first release of a new maint series, 5.x.0 where x is even), then bump the version in the blead branch in git, e.g. 5.12.0 to 5.13.0.

First, add a new feature bundle to regen/feature.pl, initially by just copying the exiting entry, and bump the file's $VERSION (after the __END__ marker); e.g.

clean build and test

Run a clean build and test to make sure nothing obvious is broken.

In particular, Porting/perldelta_template.pod is intentionally exempted from podchecker tests, to avoid false positives about placeholder text. However, once it's copied to pod/perldelta.pod the contents can now cause test failures. Problems should resolved by doing one of the following:

Replace placeholder text with correct text.

If the problem is from a broken placeholder link, you can add it to the array @perldelta_ignore_links in t/porting/podcheck.t. Lines containing such links should be marked with XXX so that they get cleaned up before the next release.

Following the instructions output by t/porting/podcheck.t on how to update its exceptions database.

push commits

Finally, push any commits done above.

$ git push origin ....

create maint branch

You MUST SKIP this step for RC, BLEAD-POINT, MAINT

If this was a BLEAD-FINAL release (i.e. the first release of a new maint series, 5.x.0 where x is even), then create a new maint branch based on the commit tagged as the current release.

Assuming you're using git 1.7.x or newer:

$ git checkout -b maint-5.12 v5.12.0
$ git push origin -u maint-5.12

make the maint branch available in the APC

Clone the new branch into /srv/gitcommon/branches on camel so the APC will receive its changes.

update perlhist.pod in other branches

Make sure any recent pod/perlhist.pod entries are copied to perlhist.pod on other branches e.g.

5.8.9 2008-Dec-14

bump RT version number

Log into http://rt.perl.org/ and check whether the new version is in the RT fields Perl Version and Fixed In. The easiest way to determine this is to go to https://rt.perl.org/rt3/Search/Build.html and click on the drop downs next to the Perl Version and Fixed In labels.

If the new version is not listed there, send an email to perlbug-admin at perl.org requesting this.

Relax!

You MUST RETIRE to your preferred PUB, CAFE or SEASIDE VILLA for some much-needed rest and relaxation.

Thanks for releasing perl!

Building a release - the day after

link announcement in epigraphs.pod

Add, to your quote to Porting/epigraphs.pod, a link to the release announcement in the web-visible mailing list archive. Commit it.

check tarball availability

Check various website entries to make sure the that tarball has appeared and is properly indexed:

Check /src on CPAN (on a fast mirror) to ensure that links to the new tarballs have appeared. There should be links in /src/5.0 (which is accumulating all new versions), and an appropriate mention in /src (which describes the latest versions in each branch, with links).

These links should appear automatically, some hours after upload. If they don't, or the /src description is inadequate, ask Ask <ask@perl.org>.

Check http://www.cpan.org/src/ to ensure that the /src updates have been correctly mirrored to the website. If they haven't, ask Ask <ask@perl.org>.

Check http://search.cpan.org to see if it has indexed the distribution. It should be visible at a URL like http://search.cpan.org/dist/perl-5.10.1/.

update dev.perl.org

You MUST SKIP this step for a RC release

In your perlweb repository, link to the new release. For a new latest-maint release, edit docs/shared/tpl/stats.html. Otherwise, edit docs/dev/perl5/index.html.

Then make a pull request to Leo Lapworth. If this fails for some reason and you cannot cajole anybody else into submitting that change, you can mail Leo as last resort.