Update the Core & Solr DOAP RDF files on the unstable and stable branches to reflect the new versions (note that the website {{{.htaccess}}} file redirects from their canonical URLs to their locations in the Lucene/Solr Git source repository - see {{{dev-tools/doap/README.txt}}} for more info): * {{{dev-tools/doap/lucene.rdf}}} * {{{dev-tools/doap/solr/doap.rdf}}}

This page is to help a Lucene/Solr committer create a new release (you need committer rights for some of the steps to create an official release). It does not reflect official release policy - many of the items may be optional, or may be modified as necessary.

Release Phases

prerequisites: making sure you have everything installed to manage a release

preparation: work with the community to decide when the release will happen and what work must be completed before it can happen

branching and feature freeze: creating a branch for this release

add new JIRA versions: add version numbers to JIRA for the release after this one

run tests: confirm that the tests pass within your release branch

build the release artifacts: actually make the release tarballs and jar/pom files for Maven

test the release artifacts: ensure that all tests pass when executed against your release candidate files

initiate a vote: call a vote on the dev list to confirm the Lucene PMC is behind releasing these artifacts

publish artifacts to the ASF mirror system: once a vote is completed, the artifacts can be pushed out to the world via the mirror system

update the website: add references to the new release and publish the latest Javadocs

announce the release: send emails announcing the release

post-release cleanup: various tasks to remove old artifacts and update JIRA tickets, etc

Releases may be of three types:

major: when the major version changes, e.g. 5.x -> 6.0.0 (includes JVM version changes, API breakages and major new features)

minor: such as the transition from 5.3.x to 5.4.0 (includes API compatible updates)

bugfix: such as 5.3.0 -> 5.3.1 (includes only bug fixes so can be a swap in replacement of its previous version)

The steps you will need to follow will, on certain occasions, differ depending upon which type of release you are undertaking.

Prior to every major or minor release (i.e. all except bugfix-only point releases) a feature freeze phase takes place for about 1-2 weeks. At the beginning of the feature freeze the development branch is copied to a release branch, and no commits are allowed to that release branch other than serious bug fixes, documentation or build updates.. This period of time should be used for extensive testing, documentation improvements and for cleaning up old JIRA issues.

Prerequisites

Python 3 (>= 3.4)

The Lucene deployment system makes extensive use of Python 3. You will need to have a copy installed locally to be able to complete any of the following steps. The minimum requirement is 3.4 as the release script uses the Python Enum package, which was only released in 3.4.

Make sure you put your GPG key's fingerprint in the OpenPGP Public Key Primary Fingerprint field in your profile on https://id.apache.org

Optionally, you may want to edit your gnupg/gpg.conf file and add a default-key value with your Code Signing Key (as a HEX value). This will prevent you from having to specify -Dgpg.key=XXXXXX later when signing artifacts.

The tests will complain if your GPG key has not been signed by another Lucene committer - this makes you a part of the GPG "web of trust" (WoT). Ask a committer that you know personally to sign your key for you, providing them with the fingerprint for the key.

Update Version Numbers in the Source Code

Add a new version for the next release using the addVersion.py script. If it is a bugfix release, we will be adding the bugfix version, otherwise we will add the version to come after the release we are producing.

If a bugfix release:

Do the following on the release branch, the stable branch, and on the unstable branch:

python3 -u dev-tools/scripts/addVersion.py X.Y.Z

git add -u .; git commit; git push

Make sure that the backcompat index for the previous release has been added to the release branch. (Note that this will ordinarily not have been done if the current release is X.Y.1, i.e. the first bugfix release off the stable branch.) See the post-release section "Generate Backcompat Indexes" below - remember you'll be generating an index for the previous release.

If a minor release, you'll be adding the next minor version, not the version being released. Do the following on the stable branch and on the unstable branch:

python3 -u dev-tools/scripts/addVersion.py X.Y+1.0

git add -u .; git commit; git push

If a major release, you'll be adding the next major version, not the version being released. Do the following on the unstable branch:

python3 -u dev-tools/scripts/addVersion.py X+1.0.0

The script will print a list of items that need to be done manually for a major release bump.

git add -u .; git commit; git push

Jenkins Release Builds

After the branching is done, add Jenkins task for the release branch so that it runs the test, like other branches using the instructions on the JenkinsReleaseBuilds page.

Inform Devs of the Release Branch

Send a note to dev@ to inform the committers that the branch has been created and the feature freeze phase has started. Include Do's and Don'ts for the feature freeze phase:

No new features may be committed to the branch.

Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all patches you want to commit to Jira first to give others the chance to review and possibly vote against the patch. Keep in mind that it is our main intention to keep the branch as stable as possible.

All patches that are intended for the branch should first be committed to the unstable branch, merged into the stable branch, and then into the current release branch.

Normal unstable and stable branch development may continue as usual. However, if you plan to commit a big change to the unstable branch while the branch feature freeze is in effect, think twice: can't the addition wait a couple more days? Merges of bug fixes into the branch may become more difficult.

Only Jira issues with Fix version "X.Y" and priority "Blocker" will delay a release candidate build.

Run Tests

Build the code and javadocs, and run the unit tests: ant clean test, ant javadocs (in lucene/). Make sure that you are actually using the minimum compiler version supported for the release. For example, 5.x releases are on Java7 so make sure that you use Java7 for the release workflow.

Examine the results. Did it build without errors? Were there Javadoc warnings? Did the tests succeed? Does the demo application work correctly? Does Test2BTerms pass (this takes a lot of memory)?

Remove lucene/benchmark/{work,temp}/ if present

Building the Release Artifacts

If after the last day of the feature freeze phase no blocking issues are in JIRA with "Fix Version" X.Y then it's time to build the release artifacts.

It is recommended to clean your Ivy cache by executing rm -rf ~/.ivy2/cache before building the artifacts. This ensures that all Ivy dependencies are freshly downloaded, so we emulate a user that never used the Lucene build system before (this step ensures downloadability of all artifacts). If you have a ton of stuff in your ivy cache, it's a good idea to move the cache temporarily to another location and restore it after the RC has been built so you wouldn't have to download everything again. This expects you to use ant-1.8.x to build the artifacts. So, ensure that you have ant-1.8.x is installed and that is there in the path when you run the following commands.

Option 1 (recommended): use dev-tools/scripts/buildAndPushRelease.py to build a release candidate and stage the results locally. Run buildAndPushRelease.py with --help to learn the available options. Here's an example of what was done for the 6.0.1 release:

Initiate a Vote

If the smoke test passes against the staged artifacts, send an email to the dev mailing list announcing the release candidate. You can use a subject of the form:

[VOTE] Release Lucene/Solr X.Y.Z RC?

Here is a template you can use:

Please vote for release candidate ? for Lucene/Solr X.Y.Z
The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-X.Y.Z-RC?-rev...
You can run the smoke tester directly with this command:
python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-X.Y.Z-RC?-rev...
Here's my +1
SUCCESS! [0:43:35.208102]

If the key you used to sign the release candidate artifacts has not been signed by other Apache committers, then testers may see the following warning:

verify trust
GPG: gpg: WARNING: This key is not certified with a trusted signature!

Repeat if Needed

Should any issues be discovered, allow fixes to be committed to the release branch, and once complete, rerun tests, produce a new RC and call another vote, as described in the previous sections.

Publishing to the ASF Mirrors and to Maven Central

Once three PMC members have voted for a release, it may be published. You should wait at least 72 hours, not including weekend days. For instance, if you announce the RC on Friday, give until end of day Tuesday or Wednesday morning for the vote.

Announce that the vote has passed on the dev mailing list, ideally with subject beginning [RESULT]

Tag the release from the same revision from which the passing release candidate's was built:

Once you have transferred all maven artifacts to repository.apache.org, you will need to: log in there with your ASF credentials; locate the staging repository containing the release that you just uploaded; "close" the staging repository; wait and wait and keep clicking refresh until it allows you to: and then "release" the staging repository. This will cause them to sync to Maven Central. See here for more details.

Maven central should show the release after a short while, but you need to wait 24 hours to give the Apache mirrors a chance to copy the new release.

If you wish, use this script to continually check the number and percentage of mirrors (and Maven Central) that have the release: dev-tools/scripts/poll-mirrors.py -version 5.5.0.

Update Website

Website += javadocs

The problem: Lucene's and Solr's voluminous per-release javadocs break the standard CMS process for the website (i.e., committer commits to the source tree; the CMS buildbot generates the site, then commits to the staging tree; committer reviews and then publishes to the production tree), because dynamic website updates, currently scheduled at 21 minutes after the hour on the hour, interrupt the extremely long commit times for javadocs being staged by buildbot, resulting in failed commits, caused by conflicts with the dynamic updates: by the time the buildbot-triggered commit has finished, its svn tree has been rendered stale.

The solution: skip committing javadocs to the source tree, then staging, then publishing, and instead commit javadocs directly to the production tree. Ordinarily this would be problematic, because the CMS wants to keep the production tree in sync with the staging tree, so anything it finds in the production tree that's not in the staging tree gets nuked. However, the CMS has a built-in mechanism to allow exceptions to the keep-production-in-sync-with-staging rule: extpaths.txt.

extpaths.txt lists paths in the production tree, relative to the project website's root directory, that are allowed to be out of sync with the staging tree.

Push docs, changes and javadocs to the CMS production tree

Ensure your refridgerator has at least 2 beers - the svn import operation can take a while, depending on your upload bandwidth. (Data point: on a 5Mbps upload connection, it took me 11 minutes to upload the Lucene docs and 6 minutes to upload the Solr docs. - Steve)

untar the lucene and solr release .tgz to create lucene-X.Y.Z and solr-X.Y.Z. Example:

Update the project DOAP files

Update the Core & Solr DOAP RDF files on the unstable and stable branches to reflect the new versions (note that the website .htaccess file redirects from their canonical URLs to their locations in the Lucene/Solr Git source repository - see dev-tools/doap/README.txt for more info):

Announce the Release

For feature releases, your announcement should describe the main features included in the release; typically this is pulled from the wiki.

Mails to the announce@apache.org list must be sent from an @apache.org email address and should contain a signature.

Because you're likely not subscribed to the general, dev, and -user lists with your @apache.org address, sending the announcement email to those list with your @apache.org address will need to be moderated through, which will likely result in delayed transmission. To avoid this, send the emails these lists using your subscribed email address, and then separately send the announcements to announce@apache.org .

Add the new version to Wikipedia (english and maybe your own language)

Post-Release

Synchronize CHANGES.txt

If this is a point release, copy the CHANGES.txt section for this point release back to the stable and unstable branches' CHANGES.txt files, removing any duplicate entries, but only from sections for as-yet unreleased versions; leave intact duplicate entries for already-released versions.

There is a script to generate a regex that will match JIRAs fixed in a release: releasedJirasRegex.py. The following examples will print regexes matching all JIRAs fixed in 5.5.2, which can then be used to find duplicates in unreleased version sections of the corresponding CHANGES.txt files:

Generate Backcompat Indexes

After each version of Lucene is released, compressed CFS and non-CFS indexes created with the newly released version are added to lucene/backwards-codecs/src/test/org/apache/lucene/index/, for use in testing backward index compatibility via org.apache.lucene.index.TestBackwardsCompatibility, which is also located under the backwards-codecs/ module. These indexes are created via methods on TestBackwardsCompatibility itself - see comments in the source for more information.

There is a script (dev-tools/scripts/addBackcompatIndexes.py) that automates most of the process: it downloads the source for the specified release; generates indexes for the current release using TestBackwardsCompatibility; compresses the indexes and places them in the correct place in the source tree; modifies TestBackwardsCompatibility.java to include the generated indexes in the list of indexes to test; and then runs TestBackwardsCompatibility. Run this script on the stable branch and on the unstable branch, and if the current release is a bugfix release, also on the release branch. On each branch, after running the script, git add the generated indexes, then cd lucene/backwards-codecs and run ant test -Dtestcase=TestBackwardsCompatibility and make sure passes before running git commit and git push.

TestBackwardsCompatibility will not test indexes for Version.LATEST, and will fail if it finds indexes from that version, so before you run addBackcompatIndexes.py, make sure dev-tools/scripts/addVersion.py has been run with the next version to be released on each of the branches - see item #3, about adding a new version, in the Branching & Feature Freeze section, above. This will likely only be an issue on the release branch, if the current release is a bugfix release, since addVersion.py will have already been run on the stable branch and on the unstable branch before the release process has gotten to this point.

To print the script's usage, run it with the --help option: python3 -u dev-tools/scripts/addBackcompatIndexes.py --help

The script uses a scratch directory - /tmp/lucenebwc/ by default if you don't specify --temp-dir DIR - to store the source for the current release that it downloads, along with the generated indexes. This directory will be removed by the script unless you specify the --no-cleanup option, which is useful when running the script on multiple branches; in the example below, the --no-cleanup option is included on the non-final script invocations, but not on the final invocation, so that the scratch directory will be cleaned up after it's no longer needed.

Do another JIRA search to find all issues with Unresolved Resolution and fixVersion of the release you just made. Note that Jira can only bulk-change fixVersion if you search only one project at a time. This URL may work - but edit the fixVersion part, and change LUCENE to SOLR to get to Solr's issues separately - https://issues.apache.org/jira/issues/?jql=project+=+LUCENE+AND+resolution=Unresolved+AND+fixVersion=6.0.1, and do a bulk change to the fixVersion to be both the master version and the next version on the branch you just released from. Uncheck the box that says "send an email for these changes".

Add a new Version for the next possible release version on the "Manage Versions" Administration page (https://issues.apache.org/jira/plugins/servlet/project-config/LUCENE/versions). e.g. If the current release is 5.2.1, add 5.2.2 with a description so that contributors can commit to the release branch with the next release version. In case of a minor release e.g. 5.2, this step needs to be done when the new release branch is cut.

Stop mirroring old releases

Shortly after new releases are first mirrored, they are automatically copied to the archives. Only the latest point release from each active branch should be kept under the Lucene PMC svnpubsub area dist/releases/lucene/ and dist/releases/solr/. Older releases can be safely deleted, since they are already backed up in the archives.

Here's an example for the 6.0.1 release (note that the 5.5.1 release is not removed):