Stop using Gradle buildSrc. Use composite builds instead
You are invalidating the whole project for any change within buildSrc, usually without there being a valid reason. We can keep the sweetness of Kotlin whilst avoiding this.
What is buildSrc
buildSrc is a directory at the Gradle project root, which can contain our build logic. This allows us to use the Kotlin DSL to write our custom build code with very little configuration and share this logic across the whole project.
This has become very popular in recent years and it is a common practice to define project-specific tasks or storing dependencies in buildSrc. We can share a single reference to a library artifact, having one place to modify it in the case of an update. All of this allows us to use IDE auto-complete and have code that is testable. But not everything is that great.
Gradle docs: A change in buildSrc causes the whole project to become out-of-date.
It’s a trap
Gradle builds highly utilise incremental builds and task caching — every time you see UP-TO-DATE or FROM-CACHE, it means some time was saved. Both of these don’t work if any change happens within buildSrc.
The impact depends on your logic within buildSrc, severity of editing it and also on your clean build time — build of the project from scratch.
- Dependency update rebuilds the whole project — Incrementing a version number triggers a clean build even if the dependency is used only within one module.
- Iteration speed is slow — When working on code within
buildSrcIt means you are having a clean build for every single change. - Local build cache is invalidated — It is invalidated for each change in
buildSrcyou perform or pull from your version control system. - Remote Gradle cache is invalidated — If your project uses a shared remote cache, any change in buildSrc basically nukes the whole remote cache both for CI and engineers. A remote cache is typically used in large projects with long clean build times, therefore the impact can be huge.
Example 1 — Dependency update
We will show two identical :app:assembleBuild builds of an example project. Both builds will be updating com.google.firebase:firebase-messaging from version 20.1.2 to 20.1.7. We will be using a great Gradle scan feature to demonstrate the differences in build execution time.
A — Build updating buildSrc. Edit of the version number triggers complete rebuild of the project and takes 1m 11s, avoiding almost no tasks and everything is rebuilt.
B — Build updating build.gradle. Update of this dependency results only in a rebuild of the module using the dependency and build quickly finishes in 26s with 194 avoided tasks.
:feature:push module is recompiled.Whilst modifying buildSrc, the build time is almost 3 times higher:
1m 11s vs. 26s.
The difference would be even more noticeable on larger projects and it is not out of the ordinary to see clean builds of several minutes. Keep in mind that the same clean build is waiting for all your team members after they pull the change and any dependency update invalidates the remote cache for the whole project.
Example 2 — Custom plugin modification
We will be modifying a custom plugin executing instrumented tests in Firebase Test Lab. The modification will only add a simple log message.
A — Modifying plugin in buildSrc. Any modification triggers again a project rebuild and takes 1m 16s this time.
B — Using exactly the same code, but with Composite Builds. There is no need for any rebuilding and only non-cacheable tasks are executed. The whole build takes 12s.
While iterating on a custom Gradle plugin, the impact of buildSrc can be very high. In this case build takes 1m 16s vs. 12s.
Use Composite builds — includeBuild
The solution to the problems stated above might be easy — Gradle Composite builds. We can move our logic from buildSrc into a separate firebasePlugin module and add includeBuild(“firebasePlugin”) within settings.gradle.
For a custom plugin — this is all you need to do.
To achieve feature parity with buildSrc — we still want to reference dependency constants directly in build.gradle. The solution is creating a plugin for dependencies.
When we include this plugin into our /push/build.gradle file, we can again achieve the dependency: api Dependencies.FIREBASE_MESSAGING.
Conclusion
Using buildSrc became very popular and it’s a great way to define your build logic in Kotlin and have this logic testable. It is important to have in mind the tradeoff, which might be surprisingly high.
You may be paying the price in long build time for minor changes. You can still get all the sweetness of Kotlin and auto-complete only by making small tweaks and adding one line of includeBuild(“plugins”) to your settings.gradle.
Happy coding!
Edit: As one reader pointed out, Gradle actually have plans on “Making the implicit
buildSrcproject an included build.” Issue to follow.
