libjxl

release.md (15075B)

---

 # libjxl release process
 
 This guide documents the release process for the libjxl project.
 
 libjxl follows the [semantic versioning](https://semver.org/spec/v2.0.0.html)
 specification for released versions. Releases are distributed as tags in the git
 repository with the semantic version prefixed by the letter "v". For example,
 release version "0.3.7" will have a git tag "v0.3.7".
 
 The public API is explicitly defined as C headers in the `lib/include`
 directory, normally installed in your include path. All other headers are
 internal API and are not covered by the versioning rules.
 
 ## Development and release workflow
 
 New code development is performed on the `main` branch of the git repository.
 Pre-submit checks enforce minimum build and test requirements for new patches
 that balance impact and test latency, but not all checks are performed before
 pull requests are merged. Several slower checks only run *after* the code has
 been merged to `main`, resulting in some errors being detected hours after the
 code is merged or even days after in the case of fuzzer-detected bugs.
 
 Release tags are cut from *release branches*. Each MAJOR.MINOR version has its
 own release branch, for example releases `0.7.0`, `0.7.1`, `0.7.2`, ... would
 have tags `v0.7.0`, `v0.7.1`, `v0.7.2`, ... on commits from the `v0.7.x` branch.
 `v0.7.x` is a branch name, not a tag name, and doesn't represent a released
 version since semantic versioning requires that the PATCH is a non-negative
 number. Released tags don't each one have their own release branch, all releases
 from the same MAJOR.MINOR version will share the same branch. The first commit
 after the branch-off points between the main branch and the release branch
 should be tagged with the suffix `-snapshot` and the name of the next
 MAJOR.MINOR version, in order to get meaningful output for `git describe`.
 
 The main purpose of the release branch is to stabilize the code before a
 release. This involves including fixes to existing bugs but **not** including
 new features. New features often come with new bugs which take time to fix, so
 having a release branch allows us to cherry-pick *bug fixes* from the `main`
 branch into the release branch without including the new *features* from `main`.
 For this reason it is important to make small commits in `main` and separate bug
 fixes from new features.
 
 After the initial minor release (`MAJOR.MINOR.PATCH`, for example `0.5.0`) the
 release branch is used to continue to cherry-pick fixes to be included in a
 patch release, for example a version `0.5.1` release. Patch fixes are only meant
 to fix security bugs or other critical bugs that can't wait until the next major
 or minor release.
 
 Release branches *may* continue to be maintained even after the next minor or
 major version has been released to support users that can't update to a newer
 minor release. In that case, the same process applies to all the maintained
 release branches.
 
 A release branch with specific cherry-picks from `main` means that the release
 code is actually a version of the code that never existed in the `main` branch,
 so it needs to be tested independently. Pre-submit and post-submit tests run on
 release branches (branches matching `v*.*.x`) but extra manual checks should be
 performed before a release, specially if multiple bug fixes interact with each
 other. Take this into account when selecting which commits to include in a
 release. The objective is to have a stable version that can be used without
 problems for months. Having the latest improvements at the time the release tag
 is created is a non-goal.
 
 ## Creating a release branch
 
 A new release branch is needed before creating a new major or minor release,
 that is, a new release where the MAJOR or MINOR numbers are increased. Patch
 releases, where only the PATCH number is increased, reuse the branch from the
 previous release of the same MAJOR and MINOR numbers.
 
 The following instructions assume that you followed the recommended [libjxl git
 setup](developing_in_github.md) where `origin` points to the upstream
 libjxl/libjxl project, otherwise use the name of your upstream remote repository
 instead of `origin`.
 
 The release branch is normally created from the latest work in `main` at the
 time the branch is created, but it is possible to create the branch from an
 older commit if the current `main` is particularly unstable or includes commits
 that were not intended to be included in the release. The following example
 creates the branch `v0.5.x` from the latest commit in main (`origin/main`), if a
 different commit is to be used then replace `origin/main` with the SHA of that
 commit. Change the `v0.5.x` branch name to the one you are creating.
 
 ```bash
 git fetch origin main
 git push git@github.com:libjxl/libjxl.git origin/main:refs/heads/v0.5.x
 ```
 
 Here we use the SSH URL explicitly since you are pushing to the `libjxl/libjxl`
 project directly to a branch there. If you followed the guide `origin` will have
 the HTTPS URL which wouldn't normally let you push since you wouldn't be
 authenticated. The `v*.*.x` branches are [GitHub protected
 branches](https://docs.github.com/en/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches)
 in our repository, however you can push to a protected branch when *creating* it
 but you can't directly push to it after it is created. To include more changes
 in the release branch see the "Cherry-picking fixes to a release" section below.
 
 ## Creating a merge label
 
 We use GitHub labels in Pull Requests to keep track of the changes that should
 be merged into a given release branch. For this purpose create a new label for
 each new MAJOR.MINOR release branch called `merge-MAJOR.MINOR`, for example,
 `merge-0.5`.
 
 In the [edit labels](https://github.com/libjxl/libjxl/issues/labels) page, click
 on "New label" and create the label. Pick your favorite color.
 
 Labels are a GitHub-only concept and are not represented in git. You can add the
 label to a Pull Request even after it was merged, whenever it is decided that
 the Pull Request should be included in the given release branch. Adding the
 label doesn't automatically merge it to the release branch.
 
 ## Update the versioning number
 
 The version number (as returned by `JxlDecoderVersion`) in the source code in
 `main` must match the semantic versioning of a release. After the release
 branch is created the code in `main` will only be included in the next major
 or minor release. Right after a release branch update the version targeting the
 next release. Artifacts from `main` should include the new (unreleased) version,
 so it is important to update it. For example, after the `v0.5.x` branch is
 created from main, you should update the version on `main` to `0.6.0`.
 
 To help update it, run this helper command (in a Debian-based system):
 
 ```bash
 ./ci.sh bump_version 0.6.0
 ```
 
 This will update the version in the following files:
 
  * `lib/CMakeLists.txt`
  * `lib/lib.gni`, automatically updated with
    `tools/scripts/build_cleaner.py --update`.
  * `debian/changelog` to create the Debian package release with the new version.
    Debian changelog shouldn't repeat the library changelog, instead it should
    include changes to the packaging scripts.
  * `.github/workflows/conformance.yml`
 
 If there were incompatible API/ABI changes, make sure to also adapt the
 corresponding section in
 [CMakeLists.txt](https://github.com/libjxl/libjxl/blob/main/lib/CMakeLists.txt#L12).
 
 ## Cherry-pick fixes to a release
 
 After a Pull Request that should be included in a release branch has been merged
 to `main` it can be cherry-picked to the release branch. Before cherry-picking a
 change to a release branch it is important to check that it doesn't introduce
 more problems, in particular it should run for some time in `main` to make sure
 post-submit tests and the fuzzers run on it. Waiting for a day is a good idea.
 
 Most of the testing is done on the `main` branch, so be careful with what
 commits are cherry-picked to a branch. Refactoring code is often not a good
 candidate to cherry-pick.
 
 To cherry-pick a single commit to a release branch (in this example to `v0.5.x`)
 you can run:
 
 ```bash
 git fetch origin
 git checkout origin/v0.5.x -b merge_to_release
 git cherry-pick -x SHA_OF_MAIN_COMMIT
 # -x will annotate the cherry-pick with the original SHA_OF_MAIN_COMMIT value.
 # If not already mentioned in the original commit, add the original PR number to
 # the commit, for example add "(cherry picked from PR #NNNN)".
 git commit --amend
 ```
 
 The `SHA_OF_MAIN_COMMIT` is the hash of the commit as it landed in main. Use
 `git log origin/main` to list the recent main commits and their hashes.
 
 Making sure that the commit message on the cherry-picked commit contains a
 reference to the original pull request (like `#NNNN`) is important. It creates
 an automatic comment in the original pull request notifying that it was
 mentioned in another commit, helping keep track of the merged pull requests. If
 the original commit was merged with the "Squash and merge" policy it will
 automatically contain the pull request number on the first line, if this is not
 the case you can amend the commit message of the cherry-pick to include a
 reference.
 
 Multiple commits can be cherry-picked and tested at once to save time. Continue
 running `git cherry-pick` and `git commit --amend` multiple times for all the
 commits you need to cherry-pick, ideally in the same order they were merged on
 the `main` branch. At the end you will have a local branch with multiple commits
 on top of the release branch.
 
 To update the version number, for example from v0.8.0 to v0.8.1 run this helper
 command (in a Debian-based system):
 
 ```bash
 ./ci.sh bump_version 0.8.1
 ```
 
 as described above and commit the changes.
 
 Finally, upload your changes to *your fork* like normal, except that when
 creating a pull request select the desired release branch as a target:
 
 ```bash
 git push myfork merge_to_release
 ```
 
 If you used the [guide](developing_in_github.md) `myfork` would be `origin` in
 that example. Click on the URL displayed, which will be something like
 
   `https://github.com/mygithubusername/libjxl/pull/new/merge_to_release`
 
 In the "Open a pull request" page, change the drop-down base branch from
 "base: main" (the default) to the release branch you are targeting.
 
 The pull request approval and pre-submit rules apply as with normal pull
 requests to the `main` branch.
 
 **Important:** When merging multiple cherry-picks use "Rebase and merge" policy,
 not the squash one since otherwise you would discard the individual commit
 message references from the git history in the release branch.
 
 ## Publishing a release
 
 Once a release tag is created it must not be modified, so you need to prepare
 the changes before creating the release. Make sure you checked the following:
 
  * The semantic version number in the release branch (see `lib/CMakeLists.txt`)
    matches the number you intend to release, all three MAJOR, MINOR and PATCH
    should match. Otherwise send a pull request to the release branch to
    update them.
 
  * The GitHub Actions checks pass on the release branch. Look for the green
    tick next to the last commit on the release branch. This should be visible
    on the branch page, for example: https://github.com/libjxl/libjxl/tree/v0.5.x
 
  * There no open fuzzer-found bugs for the release branch. The most effective
    way is to [run the fuzzer](fuzzing.md) on the release branch for a while. You
    can seed the fuzzer with corpus generated by oss-fuzz by [downloading
    it](https://google.github.io/oss-fuzz/advanced-topics/corpora/#downloading-the-corpus),
    for example `djxl_fuzzer` with libFuzzer will use:
    gs://libjxl-corpus.clusterfuzz-external.appspot.com/libFuzzer/libjxl_djxl_fuzzer
 
  * Manually check that images encode/decode ok.
 
  * Manually check that downstream projects compile with our code. Sometimes
    bugs on build scripts are only detected when other projects try to use our
    library. For example, test compiling
    [imagemagick](https://github.com/ImageMagick/ImageMagick) and Chrome.
 
 A [GitHub
 "release"](https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/about-releases)
 consists of two different concepts:
 
  * a git "tag": this is a name (`v` plus the semantic version number) with a
    commit hash associated, defined in the git repository. Most external projects
    will use git tags or HTTP URLs to these tags to fetch the code.
 
  * a GitHub "release": this is a GitHub-only concept and is not represented in
    git other than by having a git tag associated with the release. A GitHub
    release has a given source code commit SHA associated (through the tag) but
    it *also* contains release notes and optional binary files attached to the
    release.
 
 Releases from the older GitLab repository only have a git tag in GitHub, while
 newer releases have both a git tag and a release entry in GitHub.
 
 To publish a release open the [New Release
 page](https://github.com/libjxl/libjxl/releases/new) and follow these
 instructions:
 
  * Set the "Tag version" as "v" plus the semantic version number.
 
  * Select the "Target" as your release branch. For example for a "v0.7.1"
    release tag you should use the "v0.7.x" branch.
 
  * Use the version number as the release title.
 
  * Copy-paste the relevant section of the [CHANGELOG.md](../CHANGELOG.md) to the
    release notes into the release notes. Add any other information pertaining
    the release itself that are not included in the CHANGELOG.md, although prefer
    to include those in the CHANGELOG.md file. You can switch to the Preview tab
    to see the results.
 
  * Finally click "Publish release" and go celebrate with the team. 🎉
 
  * The branch v0.7.x will be pushed to gitlab automatically, but make sure to
    manually push the *tag* of the release also to
    https://gitlab.com/wg1/jpeg-xl, by doing
 
 ```bash
 git push gitlab v0.7.1
 ```
 where `gitlab` is the remote `git@gitlab.com:wg1/jpeg-xl.git`. 
 
 ### How to build downstream projects
 
 ```bash
 docker run -it debian:bookworm /bin/bash
 
 apt update
 apt install -y clang cmake git libbrotli-dev nasm pkg-config ninja-build
 export CC=clang
 export CXX=clang++
 
 mkdir -p /src
 cd /src
 
 git clone --recurse-submodules --depth 1 -b v0.9.x \
   https://github.com/libjxl/libjxl.git
 git clone --recurse-submodules --depth 1 \
   https://github.com/ImageMagick/ImageMagick.git
 git clone --recurse-submodules --depth 1 \
   https://github.com/FFmpeg/FFmpeg.git
 
 cd /src/libjxl
 cmake -B build -G Ninja .
 cmake --build build -j`nproc`
 cmake --install build --prefix="/usr"
 
 cd /src/ImageMagick
 ./configure --with-jxl=yes
 # check for "JPEG XL --with-jxl=yes yes"
 make -j `nproc`
 ./utilities/magick -version
 
 cd /src/FFmpeg
 ./configure --disable-all --disable-debug --enable-avcodec --enable-avfilter \
   --enable-avformat --enable-libjxl --enable-encoder=libjxl \
   --enable-decoder=libjxl --enable-ffmpeg
 # check for libjxl decoder/encoder support
 make -j `nproc`
 ldd ./ffmpeg
 ./ffmpeg -version
 ```