Test

Table of Contents

Properties
Methods
Script blocks
Property details
Method details
Script block details
API Documentation:Test

Executes JUnit (3.8.x, 4.x or 5.x) or TestNG tests. Test are always run in (one or more) separate JVMs.

The sample below shows various configuration options.

plugins {
    id 'java' // adds 'test' task
}

test {
  // discover and execute JUnit4-based tests
  useJUnit()

  // discover and execute TestNG-based tests
  useTestNG()

  // discover and execute JUnit Platform-based tests
  useJUnitPlatform()

  // set a system property for the test JVM(s)
  systemProperty 'some.prop', 'value'

  // explicitly include or exclude tests
  include 'org/foo/**'
  exclude 'org/boo/**'

  // show standard out and standard error of the test JVM(s) on the console
  testLogging.showStandardStreams = true

  // set heap size for the test JVM(s)
  minHeapSize = "128m"
  maxHeapSize = "512m"

  // set JVM arguments for the test JVM(s)
  jvmArgs '-XX:MaxPermSize=256m'

  // listen to events in the test execution lifecycle
  beforeTest { descriptor ->
     logger.lifecycle("Running test: " + descriptor)
  }

  // fail the 'test' task on the first test failure
  failFast = true

  // skip an actual test execution
  dryRun = true

  // listen to standard out and standard error of the test JVM(s)
  onOutput { descriptor, event ->
     logger.lifecycle("Test: " + descriptor + " produced standard out/err: " + event.message )
  }
}

The test process can be started in debug mode (see Test.getDebug()) in an ad-hoc manner by supplying the `--debug-jvm` switch when invoking the build.

gradle someTestTask --debug-jvm

Properties

PropertyDescription
allJvmArgs

The full set of arguments to use to launch the JVM for the process. This includes arguments to define system properties, the minimum/maximum heap size, and the bootstrap classpath.

binaryResultsDirectory

The root directory property for the test results in internal binary format.

bootstrapClasspath

The bootstrap classpath to use for the process. The default bootstrap classpath for the JVM is used when this classpath is empty.

classpath

The classpath to use to execute the tests.

debug

Determines whether debugging is enabled for the test process. When enabled — debug = true — the process is started in a suspended state, listening on port 5005. You should disable parallel test execution when debugging and you will need to reattach the debugger occasionally if you use a non-zero value for Test.getForkEvery().

debugOptions

The Java Debug Wire Protocol properties for the process. If enabled then the -agentlib:jdwp=... will be appended to the JVM arguments with the configuration from the parameter.

dryRun
Incubating

Indicates if this task will skip individual test execution.

enableAssertions

Returns true if assertions are enabled for the process.

environment

The environment variables to use for the process. Defaults to the environment of this process.

excludes

The exclude patterns for test execution.

executable

The name of the executable to use.

failFast

Indicates if this task will fail on the first failed test

forkEvery

The maximum number of test classes to execute in a forked test process. The forked test process will be restarted when this limit is reached.

ignoreFailures

Specifies whether the build should break when the verifications performed by this task fail.

includes

The include patterns for test execution.

javaLauncher

Configures the java executable to be used to run the tests.

jvmArgs

The extra arguments to use to launch the JVM for the process. Does not include system properties and the minimum/maximum heap size.

jvmArgumentProviders

Command line argument providers for the java process to fork.

maxHeapSize

The maximum heap size for the process, if any.

maxParallelForks

The maximum number of test processes to start in parallel.

minHeapSize

The minimum heap size for the process, if any.

modularity

The module path handling of this test task.

options

Returns test framework specific options. Make sure to call Test.useJUnit(), Test.useJUnitPlatform() or Test.useTestNG() before using this method.

reports

The reports that this task potentially produces.

scanForTestClasses

Specifies whether test classes should be detected. When true the classes which match the include and exclude patterns are scanned for test classes, and any found are executed. When false the classes which match the include and exclude patterns are executed.

systemProperties

The system properties which will be used for the process.

testClassesDirs

The directories for the compiled test sources.

testLogging

Allows to set options related to which test events are logged to the console, and on which detail level. For example, to show more information about exceptions use:

workingDir

The working directory for the process. Defaults to the project directory.

Properties added by the jacoco plugin

PropertyDescription
jacoco

The JacocoTaskExtension added by the jacoco plugin.

Methods

MethodDescription
addTestListener(listener)

Registers a test listener with this task. Consider also the following handy methods for quicker hooking into test execution: AbstractTestTask.beforeTest(groovy.lang.Closure), AbstractTestTask.afterTest(groovy.lang.Closure), AbstractTestTask.beforeSuite(groovy.lang.Closure), AbstractTestTask.afterSuite(groovy.lang.Closure)

addTestOutputListener(listener)

Registers a output listener with this task. Quicker way of hooking into output events is using the AbstractTestTask.onOutput(groovy.lang.Closure) method.

afterSuite(closure)

Adds a closure to be notified after a test suite has executed. A TestDescriptor and TestResult instance are passed to the closure as a parameter.

afterTest(closure)

Adds a closure to be notified after a test has executed. A TestDescriptor and TestResult instance are passed to the closure as a parameter.

beforeSuite(closure)

Adds a closure to be notified before a test suite is executed. A TestDescriptor instance is passed to the closure as a parameter.

beforeTest(closure)

Adds a closure to be notified before a test is executed. A TestDescriptor instance is passed to the closure as a parameter.

bootstrapClasspath(classpath)

Adds the given values to the end of the bootstrap classpath for the process.

copyTo(target)

Copies these options to the given options.

copyTo(target)

Copies these options to the given target options.

debugOptions(action)

Configures Java Debug Wire Protocol properties for the process. If JavaForkOptions.setDebug(boolean) is enabled then the -agentlib:jdwp=... will be appended to the JVM arguments with the configuration from the parameter.

environment(name, value)

Adds an environment variable to the environment for this process.

environment(environmentVariables)

Adds some environment variables to the environment for this process.

exclude(excludeSpec)

Adds an exclude spec. This method may be called multiple times to append new specs.The given closure is passed a FileTreeElement as its parameter. The closure should return true or false. Example:

exclude(excludes)

Adds exclude patterns for the files in the test classes directory (e.g. '**/*Test.class')).

exclude(excludes)

Adds exclude patterns for the files in the test classes directory (e.g. '**/*Test.class')).

exclude(excludeSpec)

Adds an exclude spec. This method may be called multiple times to append new specs. If excludes are not provided, then no files will be excluded. If excludes are provided, then files must not match any exclude pattern to be processed.

executable(executable)

Sets the name of the executable to use.

include(includeSpec)

Adds an include spec. This method may be called multiple times to append new specs. The given closure is passed a FileTreeElement as its parameter. If includes are not provided, then all files in this container will be included. If includes are provided, then a file must match at least one of the include patterns or specs to be included.

include(includes)

Adds include patterns for the files in the test classes directory (e.g. '**/*Test.class')).

include(includes)

Adds include patterns for the files in the test classes directory (e.g. '**/*Test.class')).

include(includeSpec)

Adds an include spec. This method may be called multiple times to append new specs. If includes are not provided, then all files in this container will be included. If includes are provided, then a file must match at least one of the include patterns or specs to be included.

jvmArgs(arguments)

Adds some arguments to use to launch the JVM for the process.

jvmArgs(arguments)

Adds some arguments to use to launch the JVM for the process.

onOutput(closure)

Adds a closure to be notified when output from the test received. A TestDescriptor and TestOutputEvent instance are passed to the closure as a parameter.

options(testFrameworkConfigure)

Configures test framework specific options.

removeTestListener(listener)

Unregisters a test listener with this task. This method will only remove listeners that were added by calling AbstractTestTask.addTestListener(org.gradle.api.tasks.testing.TestListener) on this task. If the listener was registered with Gradle using Gradle.addListener(java.lang.Object) this method will not do anything. Instead, use Gradle.removeListener(java.lang.Object).

removeTestOutputListener(listener)

Unregisters a test output listener with this task. This method will only remove listeners that were added by calling AbstractTestTask.addTestOutputListener(org.gradle.api.tasks.testing.TestOutputListener) on this task. If the listener was registered with Gradle using Gradle.addListener(java.lang.Object) this method will not do anything. Instead, use Gradle.removeListener(java.lang.Object).

reports(configureAction)

Configures the reports that this task potentially produces.

setTestNameIncludePatterns(testNamePattern)

Sets the test name patterns to be included in execution. Classes or method names are supported, wildcard '*' is supported. For more information see the user guide chapter on testing. For more information on supported patterns see TestFilter

systemProperties(properties)

Adds some system properties to use for the process.

useJUnit()

Specifies that JUnit4 should be used to discover and execute the tests.

useJUnit(testFrameworkConfigure)

Specifies that JUnit4 should be used to discover and execute the tests with additional configuration.

useJUnit(testFrameworkConfigure)

Specifies that JUnit4 should be used to discover and execute the tests with additional configuration.

useJUnitPlatform()

Specifies that JUnit Platform should be used to discover and execute the tests.

useJUnitPlatform(testFrameworkConfigure)

Specifies that JUnit Platform should be used to discover and execute the tests with additional configuration.

useTestNG()

Specifies that TestNG should be used to discover and execute the tests.

useTestNG(testFrameworkConfigure)

Specifies that TestNG should be used to discover and execute the tests with additional configuration.

useTestNG(testFrameworkConfigure)

Specifies that TestNG should be used to discover and execute the tests with additional configuration.

workingDir(dir)

Sets the working directory for the process. The supplied argument is evaluated as per Project.file(java.lang.Object).

Script blocks

BlockDescription
options

Configures test framework specific options.

Script blocks added by the jacoco plugin

BlockDescription
jacoco

Configures the JacocoTaskExtension added by the jacoco plugin.

Property details

List<String> allJvmArgs

The full set of arguments to use to launch the JVM for the process. This includes arguments to define system properties, the minimum/maximum heap size, and the bootstrap classpath.

DirectoryProperty binaryResultsDirectory

The root directory property for the test results in internal binary format.

Default:
project.testResultsDir/binary/task.name

FileCollection bootstrapClasspath

The bootstrap classpath to use for the process. The default bootstrap classpath for the JVM is used when this classpath is empty.

Default with java plugin:
[]

FileCollection classpath

The classpath to use to execute the tests.

Default with java plugin:
project.sourceSets.test.runtimeClasspath

boolean debug

Determines whether debugging is enabled for the test process. When enabled — debug = true — the process is started in a suspended state, listening on port 5005. You should disable parallel test execution when debugging and you will need to reattach the debugger occasionally if you use a non-zero value for Test.getForkEvery().

Since Gradle 5.6, you can configure the port and other Java debug properties via JavaForkOptions.debugOptions(org.gradle.api.Action).

Default with java plugin:
false

JavaDebugOptions debugOptions (read-only)

The Java Debug Wire Protocol properties for the process. If enabled then the -agentlib:jdwp=... will be appended to the JVM arguments with the configuration from the parameter.

Note: This property is incubating and may change in a future version of Gradle.

Indicates if this task will skip individual test execution.

For JUnit 4 and 5, this will report tests that would have executed as skipped. For TestNG, this will report tests that would have executed as passed.

Only versions of TestNG which support native dry-running are supported, i.e. TestNG 6.14 or later.

boolean enableAssertions

Returns true if assertions are enabled for the process.

Default with java plugin:
true

Map<String, Object> environment

The environment variables to use for the process. Defaults to the environment of this process.

Default with java plugin:
environment of the current process

Set<String> excludes

The exclude patterns for test execution.

Default with java plugin:
[]

String executable

The name of the executable to use.

Default with java plugin:
java command for the current JVM.

boolean failFast

Indicates if this task will fail on the first failed test

Default with java plugin:
false

long forkEvery

The maximum number of test classes to execute in a forked test process. The forked test process will be restarted when this limit is reached.

By default, Gradle automatically uses a separate JVM when executing tests.

  • A value of 0 (no limit) means to reuse the test process for all test classes. This is the default.
  • A value of 1 means that a new test process is started for every test class. This is very expensive.
  • A value of N means that a new test process is started after N test classes.

This property can have a large impact on performance due to the cost of stopping and starting each test process. It is unusual for this property to be changed from the default.

Default with java plugin:
0

boolean ignoreFailures

Specifies whether the build should break when the verifications performed by this task fail.

Set<String> includes

The include patterns for test execution.

Default with java plugin:
[]

Property<JavaLauncher> javaLauncher

Configures the java executable to be used to run the tests.

Default with java plugin:
java.toolchain

List<String> jvmArgs

The extra arguments to use to launch the JVM for the process. Does not include system properties and the minimum/maximum heap size.

Default with java plugin:
[]

List<CommandLineArgumentProvider> jvmArgumentProviders (read-only)

Command line argument providers for the java process to fork.

Default with java plugin:
[]

String maxHeapSize

The maximum heap size for the process, if any.

Default with java plugin:
null

int maxParallelForks

The maximum number of test processes to start in parallel.

By default, Gradle executes a single test class at a time.

  • A value of 1 means to only execute a single test class in a single test process at a time. This is the default.
  • A value of N means that up to N test processes will be started to execute test classes. This can improve test execution time by running multiple test classes in parallel.

This property cannot exceed the value of max-workers for the current build. Gradle will also limit the number of started test processes across all Test tasks.

Default with java plugin:
1

String minHeapSize

The minimum heap size for the process, if any.

Default with java plugin:
null

ModularitySpec modularity (read-only)

The module path handling of this test task.

TestFrameworkOptions options (read-only)

Returns test framework specific options. Make sure to call Test.useJUnit(), Test.useJUnitPlatform() or Test.useTestNG() before using this method.

TestTaskReports reports (read-only)

The reports that this task potentially produces.

boolean scanForTestClasses

Specifies whether test classes should be detected. When true the classes which match the include and exclude patterns are scanned for test classes, and any found are executed. When false the classes which match the include and exclude patterns are executed.

Default with java plugin:
true

Map<String, Object> systemProperties

The system properties which will be used for the process.

Default with java plugin:
[:]

FileCollection testClassesDirs

The directories for the compiled test sources.

Default with java plugin:
project.sourceSets.test.output.classesDirs

TestLoggingContainer testLogging (read-only)

Allows to set options related to which test events are logged to the console, and on which detail level. For example, to show more information about exceptions use:

apply plugin: 'java'

test.testLogging {
    exceptionFormat "full"
}

For further information see TestLoggingContainer.

File workingDir

The working directory for the process. Defaults to the project directory.

Default with java plugin:
project.projectDir

JacocoTaskExtension jacoco (read-only)

The JacocoTaskExtension added by the jacoco plugin.

Method details

void addTestListener(TestListener listener)

Registers a test listener with this task. Consider also the following handy methods for quicker hooking into test execution: AbstractTestTask.beforeTest(groovy.lang.Closure), AbstractTestTask.afterTest(groovy.lang.Closure), AbstractTestTask.beforeSuite(groovy.lang.Closure), AbstractTestTask.afterSuite(groovy.lang.Closure)

This listener will NOT be notified of tests executed by other tasks. To get that behavior, use Gradle.addListener(java.lang.Object).

void addTestOutputListener(TestOutputListener listener)

Registers a output listener with this task. Quicker way of hooking into output events is using the AbstractTestTask.onOutput(groovy.lang.Closure) method.

void afterSuite(Closure closure)

Adds a closure to be notified after a test suite has executed. A TestDescriptor and TestResult instance are passed to the closure as a parameter.

This method is also called after all test suites are executed. The provided descriptor will have a null parent suite.

void afterTest(Closure closure)

Adds a closure to be notified after a test has executed. A TestDescriptor and TestResult instance are passed to the closure as a parameter.

void beforeSuite(Closure closure)

Adds a closure to be notified before a test suite is executed. A TestDescriptor instance is passed to the closure as a parameter.

This method is also called before any test suites are executed. The provided descriptor will have a null parent suite.

void beforeTest(Closure closure)

Adds a closure to be notified before a test is executed. A TestDescriptor instance is passed to the closure as a parameter.

Test bootstrapClasspath(Object... classpath)

Adds the given values to the end of the bootstrap classpath for the process.

Test copyTo(JavaForkOptions target)

Copies these options to the given options.

Test copyTo(ProcessForkOptions target)

Copies these options to the given target options.

void debugOptions(Action<JavaDebugOptions> action)

Configures Java Debug Wire Protocol properties for the process. If JavaForkOptions.setDebug(boolean) is enabled then the -agentlib:jdwp=... will be appended to the JVM arguments with the configuration from the parameter.

Test environment(String name, Object value)

Adds an environment variable to the environment for this process.

Test environment(Map<String, ?> environmentVariables)

Adds some environment variables to the environment for this process.

Test exclude(Closure excludeSpec)

Adds an exclude spec. This method may be called multiple times to append new specs.The given closure is passed a FileTreeElement as its parameter. The closure should return true or false. Example:

copySpec {
  from 'source'
  into 'destination'
  //an example of excluding files from certain configuration:
  exclude { it.file in configurations.someConf.files }
}

If excludes are not provided, then no files will be excluded. If excludes are provided, then files must not match any exclude pattern to be processed.

Test exclude(Iterable<String> excludes)

Adds exclude patterns for the files in the test classes directory (e.g. '**/*Test.class')).

Test exclude(String... excludes)

Adds exclude patterns for the files in the test classes directory (e.g. '**/*Test.class')).

Test exclude(Spec<FileTreeElement> excludeSpec)

Adds an exclude spec. This method may be called multiple times to append new specs. If excludes are not provided, then no files will be excluded. If excludes are provided, then files must not match any exclude pattern to be processed.

Test executable(Object executable)

Sets the name of the executable to use.

Test include(Closure includeSpec)

Adds an include spec. This method may be called multiple times to append new specs. The given closure is passed a FileTreeElement as its parameter. If includes are not provided, then all files in this container will be included. If includes are provided, then a file must match at least one of the include patterns or specs to be included.

Test include(Iterable<String> includes)

Adds include patterns for the files in the test classes directory (e.g. '**/*Test.class')).

Test include(String... includes)

Adds include patterns for the files in the test classes directory (e.g. '**/*Test.class')).

Test include(Spec<FileTreeElement> includeSpec)

Adds an include spec. This method may be called multiple times to append new specs. If includes are not provided, then all files in this container will be included. If includes are provided, then a file must match at least one of the include patterns or specs to be included.

Test jvmArgs(Iterable<?> arguments)

Adds some arguments to use to launch the JVM for the process.

Test jvmArgs(Object... arguments)

Adds some arguments to use to launch the JVM for the process.

void onOutput(Closure closure)

Adds a closure to be notified when output from the test received. A TestDescriptor and TestOutputEvent instance are passed to the closure as a parameter.

apply plugin: 'java'

test {
   onOutput { descriptor, event ->
       if (event.destination == TestOutputEvent.Destination.StdErr) {
           logger.error("Test: " + descriptor + ", error: " + event.message)
       }
   }
}

TestFrameworkOptions options(Action<? super TestFrameworkOptions> testFrameworkConfigure)

Configures test framework specific options.

When a Test task is created outside of Test Suites, you should call Test.useJUnit(), Test.useJUnitPlatform() or Test.useTestNG() before using this method. If no test framework has been set, the task will assume JUnit4.

void removeTestListener(TestListener listener)

Unregisters a test listener with this task. This method will only remove listeners that were added by calling AbstractTestTask.addTestListener(org.gradle.api.tasks.testing.TestListener) on this task. If the listener was registered with Gradle using Gradle.addListener(java.lang.Object) this method will not do anything. Instead, use Gradle.removeListener(java.lang.Object).

void removeTestOutputListener(TestOutputListener listener)

Unregisters a test output listener with this task. This method will only remove listeners that were added by calling AbstractTestTask.addTestOutputListener(org.gradle.api.tasks.testing.TestOutputListener) on this task. If the listener was registered with Gradle using Gradle.addListener(java.lang.Object) this method will not do anything. Instead, use Gradle.removeListener(java.lang.Object).

TestTaskReports reports(Action<? super TestTaskReports> configureAction)

Configures the reports that this task potentially produces.

AbstractTestTask setTestNameIncludePatterns(List<String> testNamePattern)

Sets the test name patterns to be included in execution. Classes or method names are supported, wildcard '*' is supported. For more information see the user guide chapter on testing. For more information on supported patterns see TestFilter

Test systemProperties(Map<String, ?> properties)

Adds some system properties to use for the process.

void useJUnit()

Specifies that JUnit4 should be used to discover and execute the tests.

void useJUnit(Closure testFrameworkConfigure)

Specifies that JUnit4 should be used to discover and execute the tests with additional configuration.

The supplied action configures an instance of JUnitOptions.

void useJUnit(Action<? super JUnitOptions> testFrameworkConfigure)

Specifies that JUnit4 should be used to discover and execute the tests with additional configuration.

The supplied action configures an instance of JUnitOptions.

void useJUnitPlatform()

Specifies that JUnit Platform should be used to discover and execute the tests.

Use this option if your tests use JUnit Jupiter/JUnit5.

JUnit Platform supports multiple test engines, which allows other testing frameworks to be built on top of it. You may need to use this option even if you are not using JUnit directly.

void useJUnitPlatform(Action<? super JUnitPlatformOptions> testFrameworkConfigure)

Specifies that JUnit Platform should be used to discover and execute the tests with additional configuration.

Use this option if your tests use JUnit Jupiter/JUnit5.

JUnit Platform supports multiple test engines, which allows other testing frameworks to be built on top of it. You may need to use this option even if you are not using JUnit directly.

The supplied action configures an instance of JUnitPlatformOptions.

void useTestNG()

Specifies that TestNG should be used to discover and execute the tests.

void useTestNG(Closure testFrameworkConfigure)

Specifies that TestNG should be used to discover and execute the tests with additional configuration.

The supplied action configures an instance of TestNGOptions.

void useTestNG(Action<? super TestNGOptions> testFrameworkConfigure)

Specifies that TestNG should be used to discover and execute the tests with additional configuration.

The supplied action configures an instance of TestNGOptions.

Test workingDir(Object dir)

Sets the working directory for the process. The supplied argument is evaluated as per Project.file(java.lang.Object).

Script block details

options { }

Configures test framework specific options.

When a Test task is created outside of Test Suites, you should call Test.useJUnit(), Test.useJUnitPlatform() or Test.useTestNG() before using this method. If no test framework has been set, the task will assume JUnit4.

Delegates to:
TestFrameworkOptions from options

jacoco { }

Configures the JacocoTaskExtension added by the jacoco plugin.

Delegates to:
JacocoTaskExtension from jacoco