building_and_testing.md (5576B)
1 # Building and Testing 2 3 This file describes the building and testing facilities provided by the `ci.sh` 4 script. It assumes you already have the build environment set up. 5 6 ## Basic building 7 8 To build the JPEG XL software and run its unit tests, run: 9 10 ```bash 11 ./ci.sh release 12 ``` 13 14 ## Testing 15 16 `./ci.sh` build commands including `release`, `opt`, etc. will also run tests. 17 You can set the environment variable `SKIP_TEST=1` to skip this. 18 19 It is possible to manually run all the tests in parallel in all your CPUs with 20 the command: 21 22 ```bash 23 ./ci.sh test 24 ``` 25 26 It is also possible for faster iteration to run a specific test binary directly. 27 Tests are run with the `ctest` command and arguments passed to `ci.sh test` are 28 forwarded to `ctest` with the appropriate environment variables set. For 29 example, to list all the available tests you can run: 30 31 ```bash 32 ./ci.sh test -N 33 ``` 34 35 To run a specific test from the list or actually a set of tests matching a 36 regular expression you can use `ctest`'s parameter `-R`: 37 38 ```bash 39 ./ci.sh test -R ^MyPrefixTe 40 ``` 41 42 That command would run any test whose name that starts with `MyPrefixTe`. For 43 more options run `ctest --help`, for example, you can pass `-j1` if you want 44 to run only one test at a time instead of our default of multiple tests in 45 parallel. 46 47 ## Other commands 48 49 Running `./ci.sh` with no parameters shows a list of available commands. For 50 example, you can run `opt` for optimized developer builds with symbols or 51 `debug` for debug builds which do not have NDEBUG defined and therefore include 52 more runtime debug information. 53 54 ### Cross-compiling 55 56 To compile the code for an architecture different than the one you are running 57 you can pass a 58 [toolchain file](https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html) 59 to cmake if you have one for your target, or you can use the `BUILD_TARGET` 60 environment variable in `./ci.sh`. For some targets such the Windows targets 61 `ci.sh` sets up extra environment variables that are needed for testing. 62 63 This assumes that you already have a cross-compiling environment set up and the 64 library dependencies are already installed for the target architecture as well. 65 66 For example, to compile for the `aarch64-linux-gnu` target triplet you can run: 67 68 ```bash 69 BUILD_TARGET=aarch64-linux-gnu ./ci.sh release 70 ``` 71 72 Whenever using a `BUILD_TARGET` or even a custom `BUILD_DIR` these variables 73 must be set for **every call** to `ci.sh` even calls to `ci.sh test`, for which 74 we recommend exporting them in your shell session, for example: 75 76 ```bash 77 export BUILD_TARGET=x86_64-w64-mingw32 BUILD_DIR=build-foobar 78 ``` 79 80 ### Format checks (lint) 81 82 ```bash 83 ./ci.sh lint 84 ``` 85 86 Linter checks will verify that the format of your patch conforms to the project 87 style. For this, we run clang-format only on the lines that were changed by 88 your commits. 89 90 If your local git branch is tracking `origin/main` and you landed a few 91 commits in your branch, running this lint command will check all the changes 92 made from the common ancestor with `origin/main` to the latest changes, 93 including uncommitted changes. The output of the program will show the patch 94 that should be applied to fix your commits. You can apply these changes with the 95 following command from the base directory of the git checkout: 96 97 ```bash 98 ./ci.sh lint | patch -p1 99 ``` 100 101 ### Programming errors (tidy) 102 103 ```bash 104 ./ci.sh tidy 105 ``` 106 107 clang-tidy is a tool to check common programming errors in C++, and other valid 108 C++ constructions that are discouraged by the style guide or otherwise dangerous 109 and may constitute a bug. 110 111 To run clang-tidy on the files changed by your changes you can run `./ci.sh 112 tidy`. Note that this will report all the problems encountered in any file that 113 was modified by one of your commits, not just on the lines that your commits 114 modified. 115 116 117 ### Address Sanitizer (asan) 118 119 ```bash 120 ./ci.sh asan 121 ``` 122 123 ASan builds allow to check for invalid address usages, such as use-after-free. 124 To perform these checks, as well as other undefined behavior checks we only need 125 to build and run the unittests with ASan enabled which can be easily achieved 126 with the command above. If you want to have the ASan build files separated from 127 your regular `build/` directory to quickly switch between asan and regular 128 builds, you can pass the build directory target as follows: 129 130 ```bash 131 BUILD_DIR=build-asan ./ci.sh asan 132 ``` 133 134 ### Memory Sanitizer (msan) 135 136 MSan allows to check for invalid memory accesses at runtime, such as using an 137 uninitialized value which likely means that there is a bug. To run these checks, 138 a specially compiled version of the project and tests is needed. 139 140 For building with MSan, you need to build a version of libc++ with 141 `-fsanitize=memory` so we can link against it from the MSan build. Also, having 142 an `llvm-symbolizer` installed is very helpful to obtain stack traces that 143 include the symbols (functions and line numbers). To install `llvm-symbolizer` 144 on a Debian-based system run: 145 146 ```bash 147 sudo apt install llvm # or llvm-7, etc for a specific version. 148 ``` 149 150 To install a version of libc++ compiled with `-fsanitize=memory` you can use the 151 `./ci.sh msan_install` command helper. This will download, compile and install 152 libc++ and libc++abi in the `${HOME}/.msan` directory to be used later. 153 154 After this is set up, you can build the project using the following command: 155 156 ```bash 157 ./ci.sh msan 158 ``` 159 160 This command by default uses the `build` directory to store the cmake and object 161 files. If you want to have a separate build directory configured with msan you 162 can for example call: 163 164 ```bash 165 BUILD_DIR=build-msan ./ci.sh msan 166 ```