volver al índice

versión:

Measuring the coverage of your tests

Learn how to measure the test coverage of your application. This guide covers:

Measuring the coverage of your Unit Tests
Measuring the coverage of your Integration Tests
Separating the execution of your Unit Tests and Integration Tests
Consolidating the coverage for all your tests

Please note that code coverage is not supported in native mode.

1. Requisitos previos

To complete this guide, you need:

Roughly 15 minutes
An IDE
JDK 17+ installed with JAVA_HOME configured appropriately
Apache Maven 3.9.6
Optionally the Quarkus CLI if you want to use it
Optionally Mandrel or GraalVM installed and configured appropriately if you want to build a native executable (or Docker if you use a native container build)
Having completed the Testing your application guide

2. Arquitectura

The application built in this guide is just a Jakarta REST endpoint (hello world) that relies on dependency injection to use a service. The service will be tested with JUnit 5 and the endpoint will be annotated via a @QuarkusTest annotation.

3. Solución

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example. Clone the Git repository: git clone https://github.com/quarkusio/quarkus-quickstarts.git, or download an archive.

The solution is located in the tests-with-coverage-quickstart directory.

4. Starting from a simple project and two tests

Let’s start from an empty application created with the Quarkus Maven plugin:

CLI

quarkus create app org.acme:tests-with-coverage-quickstart \
    --extension='rest' \
    --no-code
cd tests-with-coverage-quickstart

To create a Gradle project, add the --gradle or --gradle-kotlin-dsl option.

For more information about how to install and use the Quarkus CLI, see the Quarkus CLI guide.

Maven

mvn io.quarkus.platform:quarkus-maven-plugin:3.9.3:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=tests-with-coverage-quickstart \
    -Dextensions='rest' \
    -DnoCode
cd tests-with-coverage-quickstart

To create a Gradle project, add the -DbuildTool=gradle or -DbuildTool=gradle-kotlin-dsl option.

For Windows users:

If using cmd, (don’t use backward slash \ and put everything on the same line)
If using Powershell, wrap -D parameters in double quotes e.g. "-DprojectArtifactId=tests-with-coverage-quickstart"

Now we’ll be adding all the elements necessary to have an application that is properly covered with tests.

First, a Jakarta REST resource serving a hello endpoint:

package org.acme.testcoverage;

import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;

@Path("/hello")
public class GreetingResource {

    private final GreetingService service;

    @Inject
    public GreetingResource(GreetingService service) {
        this.service = service;
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Path("/greeting/{name}")
    public String greeting(String name) {
        return service.greeting(name);
    }

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        return "hello";
    }
}

This endpoint uses a greeting service:

package org.acme.testcoverage;

import jakarta.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class GreetingService {

    public String greeting(String name) {
        return "hello " + name;
    }

}

The project will also need a test:

package org.acme.testcoverage;

import io.quarkus.test.junit.QuarkusTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Tag;

import java.util.UUID;

import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.is;

@QuarkusTest
public class GreetingResourceTest {

    @Test
    public void testHelloEndpoint() {
        given()
          .when().get("/hello")
          .then()
             .statusCode(200)
             .body(is("hello"));
    }

    @Test
    public void testGreetingEndpoint() {
        String uuid = UUID.randomUUID().toString();
        given()
          .pathParam("name", uuid)
          .when().get("/hello/greeting/{name}")
          .then()
            .statusCode(200)
            .body(is("hello " + uuid));
    }
}

5. Setting up JaCoCo

Now we need to add JaCoCo to our project. To do this we need to add the following to the build file:

pom.xml

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-jacoco</artifactId>
  <scope>test</scope>
</dependency>

build.gradle

testImplementation("io.quarkus:quarkus-jacoco")

This Quarkus extension takes care of everything that would usually be done via the JaCoCo Maven plugin, so no additional config is required.

Using both the extension and the plugin requires special configuration, if you add both you will get lots of errors about classes already being instrumented. The configuration needed is detailed below.

6. Working with multi-module projects

Up until 3.2, data-file and report-location were always relative to the module’s build output directory, which prevented from working with multi-module projects where you want to aggregate all coverages into a single parent directory. Starting in 3.3, specifying a data-file or report-location will assume the path as is. Here is an example on how to set up the surefire plugin:

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <systemPropertyVariables>
      <quarkus.jacoco.data-file>${maven.multiModuleProjectDirectory}/target/jacoco.exec</quarkus.jacoco.data-file>
      <quarkus.jacoco.reuse-data-file>true</quarkus.jacoco.reuse-data-file>
      <quarkus.jacoco.report-location>${maven.multiModuleProjectDirectory}/target/coverage</quarkus.jacoco.report-location>
    </systemPropertyVariables>
  </configuration>
</plugin

7. Running the tests with coverage

Run mvn verify, the tests will be run and the results will end up in target/jacoco-reports. This is all that is needed, the quarkus-jacoco extension allows JaCoCo to just work out of the box.

There are some config options that affect this:

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property	Tipo	Por defecto
`quarkus.jacoco.enabled` Whether or not the jacoco extension is enabled. Environment variable: `QUARKUS_JACOCO_ENABLED` Show more	boolean	`true`
`quarkus.jacoco.data-file` The jacoco data file. The path can be relative (to the module) or absolute. Environment variable: `QUARKUS_JACOCO_DATA_FILE` Show more	string	`target/jacoco-quarkus.exec`
`quarkus.jacoco.reuse-data-file` Whether to reuse (`true`) or delete (`false`) the jacoco data file on each run. Environment variable: `QUARKUS_JACOCO_REUSE_DATA_FILE` Show more	boolean	`false`
`quarkus.jacoco.report` If Quarkus should generate the Jacoco report Environment variable: `QUARKUS_JACOCO_REPORT` Show more	boolean	`true`
`quarkus.jacoco.output-encoding` Encoding of the generated reports. Environment variable: `QUARKUS_JACOCO_OUTPUT_ENCODING` Show more	string	`UTF-8`
`quarkus.jacoco.title` Name of the root node HTML report pages. Environment variable: `QUARKUS_JACOCO_TITLE` Show more	string
`quarkus.jacoco.footer` Footer text used in HTML report pages. Environment variable: `QUARKUS_JACOCO_FOOTER` Show more	string
`quarkus.jacoco.source-encoding` Encoding of the source files. Environment variable: `QUARKUS_JACOCO_SOURCE_ENCODING` Show more	string	`UTF-8`
`quarkus.jacoco.includes` A list of class files to include in the report. May use wildcard characters (* and ?). When not specified everything will be included. For instance: `/fo//` targets all classes under fo and sub packages `/bar/` targets all classes directly under bar `*/BAR*.class` targets classes that contain BAR in their name regardless of path Environment variable: `QUARKUS_JACOCO_INCLUDES` Show more	list of string	`**`
`quarkus.jacoco.excludes` A list of class files to exclude from the report. May use wildcard characters (* and ?). When not specified nothing will be excluded. For instance: `/fo//` targets all classes under fo and sub packages `/bar/` targets all classes directly under bar `*/BAR*.class` targets classes that contain BAR in their name regardless of path Environment variable: `QUARKUS_JACOCO_EXCLUDES` Show more	list of string
`quarkus.jacoco.report-location` The location of the report files. The path can be relative (to the module) or absolute. Environment variable: `QUARKUS_JACOCO_REPORT_LOCATION` Show more	string	`target/jacoco-report`

Configuration property

Tipo

Por defecto

quarkus.jacoco.enabled

Whether or not the jacoco extension is enabled.

Environment variable: QUARKUS_JACOCO_ENABLED

boolean

true

quarkus.jacoco.data-file

The jacoco data file. The path can be relative (to the module) or absolute.

Environment variable: QUARKUS_JACOCO_DATA_FILE

string

target/jacoco-quarkus.exec

quarkus.jacoco.reuse-data-file

Whether to reuse (true) or delete (false) the jacoco data file on each run.

Environment variable: QUARKUS_JACOCO_REUSE_DATA_FILE

boolean

false

quarkus.jacoco.report

If Quarkus should generate the Jacoco report

Environment variable: QUARKUS_JACOCO_REPORT

boolean

true

quarkus.jacoco.output-encoding

Encoding of the generated reports.

Environment variable: QUARKUS_JACOCO_OUTPUT_ENCODING

string

UTF-8

quarkus.jacoco.title

Name of the root node HTML report pages.

Environment variable: QUARKUS_JACOCO_TITLE

string

quarkus.jacoco.footer

Footer text used in HTML report pages.

Environment variable: QUARKUS_JACOCO_FOOTER

string

quarkus.jacoco.source-encoding

Encoding of the source files.

Environment variable: QUARKUS_JACOCO_SOURCE_ENCODING

string

UTF-8

quarkus.jacoco.includes

A list of class files to include in the report. May use wildcard characters (* and ?). When not specified everything will be included.

For instance:

**/fo/**/* targets all classes under fo and sub packages
**/bar/* targets all classes directly under bar
**/*BAR*.class targets classes that contain BAR in their name regardless of path

Environment variable: QUARKUS_JACOCO_INCLUDES

list of string

**

quarkus.jacoco.excludes

A list of class files to exclude from the report. May use wildcard characters (* and ?). When not specified nothing will be excluded.

For instance:

**/fo/**/* targets all classes under fo and sub packages
**/bar/* targets all classes directly under bar
**/*BAR*.class targets classes that contain BAR in their name regardless of path

Environment variable: QUARKUS_JACOCO_EXCLUDES

list of string

quarkus.jacoco.report-location

The location of the report files. The path can be relative (to the module) or absolute.

Environment variable: QUARKUS_JACOCO_REPORT_LOCATION

string

target/jacoco-report

When working with a multi-module project, then for code coverage to work properly, the upstream modules need to be properly indexed.

8. Coverage for tests not using @QuarkusTest

The Quarkus automatic JaCoCo config will only work for tests that are annotated with @QuarkusTest. If you want to check the coverage of other tests as well then you will need to fall back to the JaCoCo maven plugin.

In addition to including the quarkus-jacoco extension in your pom.xml you will need the following config:

pom.xml

<project>
    <build>
        <plugins>
            ...
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>${jacoco.version}</version>
                <executions>
                   <execution>
                      <id>default-prepare-agent</id>
                      <goals>
                           <goal>prepare-agent</goal>
                      </goals>
                      <configuration>
                        <exclClassLoaders>*QuarkusClassLoader</exclClassLoaders>  (1)
                        <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                        <append>true</append>
                      </configuration>
                   </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

1	This config tells it to ignore `@QuarkusTest` related classes, as they are loaded by `QuarkusClassLoader`

build.gradle

plugins {
    id 'jacoco' (1)
}

test {
    finalizedBy jacocoTestReport
    jacoco {
        excludeClassLoaders = ["*QuarkusClassLoader"] (2)
        destinationFile = layout.buildDirectory.file("jacoco-quarkus.exec").get().asFile (2)
    }
    jacocoTestReport.enabled = false (3)
}

1	Add the `jacoco` gradle plugin
2	This config tells it to ignore `@QuarkusTest` related classes, as they are loaded by `QuarkusClassLoader`
3	Set this config to `false` if you are also using the `quarkus-jacoco` extension and have at least one `@QuarkusTest`. The default `jacocoTestReport` task can be skipped since `quarkus-jacoco` will generate the combined report of regular unit tests and `@QuarkusTest` classes since the execution data is recorded in the same file.

This config will only work if at least one @QuarkusTest is being run. If you are not using @QuarkusTest then you can simply use the JaCoCo plugin in the standard manner with no additional config.

8.1. Coverage for Integration Tests

To get code coverage data from integration tests, the following requirements need to be met:

The built artifact is a jar (and not a container or native binary).
JaCoCo needs to be configured in your build tool.
The application must have been built with quarkus.package.write-transformed-bytecode-to-build-output set to true

Setting quarkus.package.write-transformed-bytecode-to-build-output=true should be done with caution and only if subsequent builds are done in a clean environment - i.e. the build tool’s output directory has been completely cleaned.

In the pom.xml, you can add the following plugin configuration for JaCoCo. This will append integration test data into the same destination file as unit tests, re-build the JaCoCo report after the integration tests are complete, and thus produce a comprehensive code-coverage report.

<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>${jacoco.version}</version>
            <executions>
                ... (1)

                <execution>
                    <id>default-prepare-agent-integration</id>
                    <goals>
                        <goal>prepare-agent-integration</goal>
                    </goals>
                    <configuration>
                        <destFile>${project.build.directory}/jacoco-quarkus.exec</destFile>
                        <append>true</append>
                    </configuration>
                </execution>
                <execution>
                    <id>report</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>report</goal>
                    </goals>
                    <configuration>
                        <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>
                        <outputDirectory>${project.build.directory}/jacoco-report</outputDirectory>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>

1	All executions should be in the same `<plugin>` definition so make sure you concatenate all of them.

In order to run the integration tests as a jar with the JaCoCo agent, add the following to your pom.xml.

<build>
    ...
    <plugins>
        ...
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>${surefire-plugin.version}</version>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                    <configuration>
                        <systemPropertyVariables>
                            <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                            <maven.home>${maven.home}</maven.home>
                            <quarkus.test.arg-line>${argLine}</quarkus.test.arg-line>
                        </systemPropertyVariables>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>

Sharing the same value for quarkus.test.arg-line might break integration test runs that test different types of Quarkus artifacts. In such cases, the use of Maven profiles is advised.

9. Setting coverage thresholds

You can set thresholds for code coverage using the JaCoCo Maven plugin. Note the element <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>. You must set it matching your choice for quarkus.jacoco.data-file.

pom.xml

<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>${jacoco.version}</version>
            <executions>
                ... (1)

                <execution>
                    <id>jacoco-check</id>
                    <goals>
                        <goal>check</goal>
                    </goals>
                    <phase>post-integration-test</phase>
                    <configuration>
                        <dataFile>${project.build.directory}/jacoco-quarkus.exec</dataFile>
                        <rules>
                            <rule>
                                <element>BUNDLE</element>
                                <limits>
                                    <limit>
                                        <counter>LINE</counter>
                                        <value>COVEREDRATIO</value>
                                        <minimum>0.8</minimum>
                                    </limit>
                                    <limit>
                                        <counter>BRANCH</counter>
                                        <value>COVEREDRATIO</value>
                                        <minimum>0.72</minimum>
                                    </limit>
                                </limits>
                            </rule>
                        </rules>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        ...
    </plugins>
    ...
</build>

1	All executions should be in the same `<plugin>` definition so make sure you concatenate all of them.

build.gradle

jacocoTestCoverageVerification {
    executionData.setFrom("$project.buildDir/jacoco-quarkus.exec")
    violationRules {
        rule {
            limit {
                counter = 'INSTRUCTION'
                value = 'COVEREDRATIO'
                minimum = 0.80
            }
            limit {
                counter = 'BRANCH'
                value = 'COVEREDRATIO'
                minimum = 0.72
            }
        }
    }
}
check.dependsOn jacocoTestCoverageVerification

Excluding classes from the verification task can be configured as following:

jacocoTestCoverageVerification {
    afterEvaluate { (1)
        classDirectories.setFrom(files(classDirectories.files.collect { (2)
            fileTree(dir: it, exclude: [
                    "org/example/package/**/*" (3)
            ])
        }))
    }
}

1	`classDirectories` needs to be read after evaluation phase in Gradle
2	Currently, there is a bug in Gradle JaCoCo which requires the `excludes` to be specified in this manner - https://github.com/gradle/gradle/issues/14760. Once this issue is fixed, excludes
3	Exclude all classes in `org/example/package` package

10. Conclusion

You now have all the information you need to study the coverage of your tests! But remember, some code that is not covered is certainly not well tested. But some code that is covered is not necessarily well tested. Make sure to write good tests!

1. Requisitos previos
2. Arquitectura
3. Solución
4. Starting from a simple project and two tests
5. Setting up JaCoCo
6. Working with multi-module projects
7. Running the tests with coverage
8. Coverage for tests not using @QuarkusTest
- 8.1. Coverage for Integration Tests
9. Setting coverage thresholds
10. Conclusion