Release Checklist
Release checklist
Here's a checklist for the release process.
Leading Up To The Release
- Talk to team about whether there are any changes which MUST go in this release which may cause delay.
- Look through outstanding issues, to identify any problems that might be necessary to fixup before the release. Good candidates are reports of the project not building on different architectures or crashes.
- Identify a good lead for each outstanding issue, and ask them about a fix timeline.
- Create a milestone for the next release on Github, and go though open issues and PRs and mark accordingly.
- Ask (via email) the most significant contributor who has not already named a release to name the release (use
devtools/credit --verbose v<PREVIOUS-VERSION>
to find this contributor). CC previous namers and team.
Preparing for -rc1
- Check that
CHANGELOG.md
is well formatted, ordered in areas, covers all signficant changes, and sub-ordered approximately by user impact & coolness. - Use
devtools/changelog.py
to collect the changelog entries from pull request commit messages and merge them into the manually maintainedCHANGELOG.md
. This does API queries to GitHub, which are severely
ratelimited unless you use an API token: set theGH_TOKEN
environment variable to a Personal Access Token from https://github.com/settings/tokens - Create a new CHANGELOG.md heading to
v<VERSION>rc1
, and create a link at the bottom. Note that you should exactly copy the date and name format from a previous release, as thebuild-release.sh
script relies on this. - Update the package versions:
uv run make update-versions NEW_VERSION=v<VERSION>rc1
- Create a PR with the above.
Releasing -rc1
- Merge the above PR.
- Tag it
git pull && git tag -s v<VERSION>rc1
. Note that you should get a prompt to give this tag a 'message'. Make sure you fill this in. - Confirm that the tag will show up for builds with
git describe
. We don't push it to GitHub yet, just in case the following steps fail, and more fixes are required! - Run
contrib/cl-repro.sh
to generate the requiredcl-repro-<codename>
builder images for the reproducible build environment. - Execute
tools/build-release.sh bin-Ubuntu sign
to locally reproduce the release, generating a matchingSHA256SUMS-v<VERSION>
file and signing it with your GPG key. - Push the tag to trigger the "Release 🚀" CI action, which drafts a new
v<VERSION>rc1
pre-release on GitHub and uploads reproducible builds alongside theSHA256SUMS-v<VERSION>
file and its signature from the[email protected]
key. - Verify your local
SHA256SUMS-v<VERSION>
file matches the one in the draft release, then append your local signatures to the release'sSHA256SUMS-v<VERSION>.asc
file to attest to the build's integrity. - Announce rc1 release on core-lightning's release-chat channel on Discord & Telegram.
- Use
devtools/credit --verbose v<PREVIOUS-VERSION>
to get commits, days and contributors data for release note. - Prepare release notes draft including information from above step, and share with the team for editing.
- Upgrade your personal nodes to the rc1, to help testing.
- Github action
Publish Python 🐍 distributions 📦 to PyPI and TestPyPI
uploads the pyln modules on test PyPI server. Make sure that the action has been triggered with RC tag and that the modules have been published onhttps://test.pypi.org/project/pyln-*/#history
. - Docker image publishing is handled by the GitHub action
Build and push multi-platform docker images
. Ensure that this action is triggered and that the RC image has been successfully uploaded to Docker Hub after the action completes. Alternatively, you can publish Docker images by running thetools/build-release.sh docker
script. The GitHub action takes approximately 3-4 hours, while the script takes about 6-7 hours. It is highly recommended to test your Docker setup if you haven't done so before. Prior to building docker images bytools/build-release.sh
script, ensure thatmultiarch/qemu-user-static
setup is working on your system as described here.
Releasing -rc2, ..., -rcN
- Update CHANGELOG.md by changing rc(N-1) to rcN. Update the changelog list with information from newly merged PRs also.
- Update the package versions:
uv run make update-versions NEW_VERSION=v<VERSION>rcN
- Add a PR with the rcN.
- Tag it
git pull && git tag -s v<VERSION>rcN && git push --tags
. - Pushing the tag automatically starts the "Release 🚀" CI job, creating a draft pre-release and uploading reproducible builds with their
SHA256SUMS
files signed by the project key. - Set up the reproducible build environment by running the script
contrib/cl-repro.sh
to generate the necessary builder images. - Use the command
tools/build-release.sh bin-Ubuntu sign
to locally rebuild the release and generate a personal signature file for the checksums. - After confirming the local and pre-release
SHA256SUMS-v<VERSION>
files match, append your signatures to the pre-release'sSHA256SUMS-v<VERSION>.asc
file to formally attest to the build's validity. - Announce tagged rc release on core-lightning's release-chat channel on Discord & Telegram.
- Upgrade your personal nodes to the rcN.
- Confirm that Github actions for PyPI and Docker publishing are working as expected.
Tagging the Release
- Update the CHANGELOG.md; remove -rcN in both places, update the date and add title and namer.
- Update the contrib/pyln package versions:
uv run make update-versions NEW_VERSION=v<VERSION>
- Add a PR with that release.
- Merge the PR, then:
git pull
VERSION=23.05; git tag -a -s v$VERSION -m v$VERSION
git push --tags
- Pushing the tag will trigger the CI pipeline, which will draft the pre-release and upload the build artifacts with project-signed checksums.
- Prepare the build environments by executing the
contrib/cl-repro.sh
script. - Run
tools/build-release.sh bin-Ubuntu sign
(with--sudo
if you need root to run Docker) to:- Create reproducible zipfile
- Build non-reproducible Fedora image
- Build reproducible Ubuntu-v20.04, Ubuntu-v22.04 and Ubuntu-v24.04 images. Follow link for manually Building Ubuntu Images.
- Build Docker images for amd64 and arm64v8. Follow link for more details on Docker publishing.
- Create and sign checksums. Follow link for manually signing the release.
- If you used
--sudo
, the tarballs may be owned by root, so revert ownership if necessary:
sudo chown ${USER}:${USER} *${VERSION}*
- Verify the checksums match the pre-release
SHA256SUMS-v<VERSION>
, then append your signatures to the official signatureSHA256SUMS-v<VERSION>.asc
file to confirm the build's integrity. - Send
SHA256SUMS-v<VERSION>
&SHA256SUMS-v<VERSION>.asc
files to the rest of the team to check and sign the release. - Team members can verify the release with the help of
build-release.sh
:
- Copy the release captain's
SHA256SUMS-v<VERSION>
andSHA256SUMS-v<VERSION>.asc
into the root folder (lightning
). - Run
tools/build-release.sh --verify
. It will create reproducible images, verify checksums and sign. - Send your signatures from
release/SHA256SUMS-v<VERSION>.asc
to release captain. - Or follow link for manual verification instructions.
- Append signatures shared by the team into the
SHA256SUMS-v<VERSION>.asc
file, verify withgpg --verify SHA256SUMS-v<VERSION>.asc
and include the file in the draft release. - The GitHub action
Publish Python 🐍 distributions 📦 to PyPI and TestPyPI
should upload the pyln modules to pypi.org. However, this can also be done manually by runninguv run make pyln-release
. This process requires keys for each of thepyln-client
,pyln-proto
, andpyln-testing
modules to be accessible to uv. You can set the key as an environment variable and build and publish each pyln release independently:export UV_PUBLISH_TOKEN=<pyln-client token>
uv run make pyln-release-client
- ... repeat for each pyln package with the appropriate token.
- Publish multi-arch Docker images (
elementsproject/lightningd:v${VERSION}
andelementsproject/lightningd:latest
) to Docker Hub either using the GitHub actionBuild and push multi-platform docker images
or by running thetools/build-release.sh docker
script. Prior to building docker images bytools/build-release.sh
script, ensure thatmultiarch/qemu-user-static
setup is working on your system as described here.
Performing the Release
- Edit the GitHub draft and include the
SHA256SUMS-v<VERSION>.asc
file. - Publish the release as not a draft.
- Announce the final release on core-lightning's release-chat channel on Discord & Telegram.
- Send a mail to c-lightning and lightning-dev mailing lists, using the same wording as the Release Notes in GitHub.
- Write release blog, post it on Blockstream and announce the release on Twitter.
Post-release
- Create a PR to update Makefile's CLN_NEXT_VERSION and important dates for the next release on
.github/PULL_REQUEST_TEMPLATE.md
. - Look through PRs which were delayed for release and merge them.
- Close out the Milestone for the now-shipped release.
- Update this file with any missing or changed instructions.
Performing the Point (hotfix) Release
- Create a new branch named
release-<VERSION>.<POINT_VERSION>
, where each new branch is based on the commit from the previous release tag. For example,release-<VERSION>.1
is based onrelease-<VERSION>
,release-<VERSION>.2
is based onrelease-<VERSION>.1
, and so on. - Cherry-pick all necessary commits for the hotfix into the new branch.
- Add entries for changes and fixed issues in
CHANGELOG.md
under a new heading forv<VERSION>.<POINT_VERSION>
. - Update the python package versions by running
uv run make update-versions NEW_VERSION=<VERSION>.<POINT_VERSION>
- Create a new commit that includes the updates from
update-versions
andCHANGELOG.md
. - Tag the release with
git pull && git tag -s v<VERSION>.<POINT_VERSION>
. You will be prompted to enter a tag message, ensure this is filled out. - Confirm that the tag is properly set up for builds by running
git describe
. - Trigger the pre-release by pushing the version tag with
git push --tags
; the CI will handle drafting the release and uploading the initial signed checksums. - Generate the required builder images by running
contrib/cl-repro.sh
. - Sign the release locally by running
tools/build-release.sh bin-Ubuntu sign
which will sign the release contents and createSHA256SUMS-v<VERSION>
andSHA256SUMS-v<VERSION>.asc
in the release folder. - Validate that your local checksums
SHA256SUMS-v<VERSION>
match the Draft release's, then add your signatures to the draft release's signatureSHA256SUMS-v<VERSION>.asc
file. - Share the
SHA256SUMS-v<VERSION>
andSHA256SUMS-v<VERSION>.asc
files with the team for verification and signing. - Append the signatures received from the team to the
SHA256SUMS-v<VERSION>.asc
file. Verify the file usinggpg --verify SHA256SUMS-v<VERSION>.asc
. Then re-upload the file. - Finalize and publish the release (change it from draft to public).
- Ensure that the GitHub Actions for
Publish Python 🐍 distributions 📦 to PyPI and TestPyPI
andBuild and push multi-platform docker images
are functioning correctly. Check that thePyPI
modules published onhttps://pypi.org/project/pyln-*
and that the Docker image has been uploaded to Docker Hub. - Announce the hotfix release in the core-lightning release-chat channel on Discord and on Telegram.
Updated 8 days ago