Gr8 makes it easy to shadow, shrink, and minimize your jars.
Gradle has a very powerful plugin system. Unfortunately, Gradle handling of classpath/Classloaders for plugins has some serious limitations. For an example:
- Gradle always forces its bundled version of the Kotlin stdlib in the classpath. This makes it impossible to use Kotlin 1.5 APIs with Gradle 7.1 for an example because Gradle 7.1 uses Kotlin 1.4 (See compatibility matrix for other versions).
buildSrcdependencies leak in the classpath. This causes very weird bugs during execution because a conflicting dependency might be forced in the classpath. This happens especially with popular libraries such asokioorantlrthat are likely to be used with conflicting versions by different plugins in your build.
By shadowing (embedding and relocating) the plugin dependencies, it is possible to ship a plugin and all its dependencies without having to worry about what other dependencies are on the classpath, including the Kotlin stdlib.
To learn more, read the "Use latest Kotlin in your Gradle plugins" blog post.
Gr8 is mostly focused at Gradle plugins but you can use it to relocate/shrink any library/binary. See Shrinking a Kotlin binary by 99.2% for a good illustration.
plugins {
id("org.jetbrains.kotlin.jvm").version("$latestKotlinVersion")
id("com.gradleup.gr8").version("$gr8Version")
}
dependencies {
implementation("com.squareup.okhttp3:okhttp:4.9.0")
// More dependencies here
}
/**
* Create a separate configuration to resolve compileOnly dependencies.
* You can skip this if you have no compileOnly dependencies.
*/
val compileOnlyDependencies: Configuration = configurations.create("compileOnlyDependencies")
compileOnlyDependencies.extendsFrom(configurations.getByName("compileOnly"))
gr8 {
val shadowedJar = create("gr8") {
// program jars are included in the final shadowed jar
addProgramJarsFrom(configurations.getByName("runtimeClasspath"))
addProgramJarsFrom(tasks.getByName("jar"))
// classpath jars are only used by R8 for analysis but are not included in the
// final shadowed jar.
addClassPathJarsFrom(compileOnlyDependencies)
proguardFile("rules.pro")
// Use a version from https://storage.googleapis.com/r8-releases/raw
// Requires a maven("https://storage.googleapis.com/r8-releases/raw") repository
r8Version("8.8.19")
// Or use a commit
// The jar is downloaded on demand
r8Version("887704078a06fc0090e7772c921a30602bf1a49f")
// Or leave it to the default version
}
}Then customize your proguard rules. The below is a non-exhaustive example. If you're using reflection, you might need more rules
# Keep your public API so that it's callable from scripts
-keep class com.example.** { *; }
# Repackage other classes
-repackageclasses com.example.relocated
# Allows more aggressive repackaging
-allowaccessmodification
# We need to keep type arguments for Gradle to be able to instantiate abstract models like `Property`
-keepattributes Signature,Exceptions,*Annotation*,InnerClasses,PermittedSubclasses,EnclosingMethod,Deprecated,SourceFile,LineNumberTable
Using Gr8 to shadow dependencies in Gradle plugin is a typical use case but requires extra care because:
- The
java-gradle-pluginautomatically addsapi(gradleApi())to your dependencies butgradleApi()shouldn't be shadowed. gradleApi()is a multi-release jar file that R8 doesn't support.- Since the plugins are published, the shadowed dependencies must not be exposed in the .pom/.module files.
To work around this, you can use, removeGradleApiFromApi(), registerTransform() and custom configurations:
val shadowedDependencies = configurations.create("shadowedDependencies")
val compileOnlyDependencies: Configuration = configurations.create("compileOnlyDependencies") {
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named<Usage>(Usage.JAVA_API))
}
attributes {
attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, FilterTransform.artifactType)
}
}
compileOnlyDependencies.extendsFrom(configurations.getByName("compileOnly"))
dependencies {
add(shadowedDependencies.name, "com.squareup.okhttp3:okhttp:4.9.0")
add(compileOnlyDependencies.name, gradleApi())
// More dependencies here
}
if (shadow) {
gr8 {
create("default") {
val shadowedJar = create("default") {
addProgramJarsFrom(shadowedDependencies)
addProgramJarsFrom(tasks.getByName("jar"))
proguardFile("rules.pro")
registerFilterTransform(listOf(".*/impldep/META-INF/versions/.*"))
}
removeGradleApiFromApi()
// Optional: replace the regular jar with the shadowed one in the publication
replaceOutgoingJar(shadowedJar)
// Or if you prefer the shadowed jar to be a separate variant in the default publication
// The variant will have `org.gradle.dependency.bundling = shadowed`
addShadowedVariant(shadowedJar)
// Allow to compile the module without exposing the shadowedDependencies downstream
configurations.getByName("compileOnly").extendsFrom(shadowedDependencies)
configurations.getByName("compileOnly").extendsFrom(compileOnlyDependencies)
configurations.getByName("testImplementation").extendsFrom(shadowedDependencies)
}
}
} else {
configurations.named("implementation").extendsFrom(shadowedDependencies)
}By default, R8 removes kotlin.Metadata from the shadowed jar. This means the Kotlin compiler only sees plain Java classes and symbols and Kotlin-only features such as parameters default values, extension function, etc... are lost.
If you want to keep them, you need to keep kotlin.Metadata and kotlin.Unit:
# Keep kotlin metadata so that the Kotlin compiler knows about top level functions
-keep class kotlin.Metadata { *; }
# Keep Unit as it's in the signature of public methods:
-keep class kotlin.Unit { *; }
Note
Stripping kotlin.Metadata acts as a compile-time verification that your API is usable in Groovy as it is in Kotlin and might be beneficial.
You can specify the version of the java runtime to use with systemClassesToolchain:
gr8 {
val shadowedJar = create("gr8") {
proguardFile("rules.pro")
addProgramJarsFrom(configurations.getByName("runtimeClasspath"))
systemClassesToolchain {
languageVersion.set(JavaLanguageVersion.of(11))
}
}
}
Could I use the Shadow plugin instead?
The Gradle Shadow Plugin has been helping plugin authors for years and is a very stable solution. Unfortunately, it doesn't allow very granular configuration and might relocate constant strings that shouldn't be. In practice, any plugin that tries to read the "kotlin" extension is subject to having its behaviour changed:
project.extensions.getByName("kotlin")will be transformed to:
project.extensions.getByName("com.relocated.kotlin")For plugins that generate source code and contain a lot of package names, this might be even more unpredictable and require weird workarounds.
By using R8 and proguard rules, Gr8 makes relocation more predictable and configurable.
Could I use the Gradle Worker API instead?
The Gradle Worker API has a classLoaderIsolation mode that can be used to achieve a similar result with some limitations:
gradle-apiandkotlin-stdlibare still in the worker classpath meaning you need to make sure your Kotlin version is compatible.- classLoaderIsolation leaks memory
- Workers require serializing parameters and writing more boilerplate code.
Are there any drawbacks?
Yes. Because every plugin now relocates its own version of kotlin-stdlib, okio and other dependencies, it means more work for the Classloaders and more Metaspace being used. There's a risk that builds will use more memory, although it hasn't been a big issue so far.