Our AI writing assistant, WriteUp, can assist you in easily writing any text. Click here to experience its capabilities.

Herding Elephants

Summary

Square's Seller organization has a large Gradle project consisting of 3,500 modules, with over two million lines of code in Kotlin and Java spread across multiple applications and libraries. The Mobile Developer Experience Android team embarked on a project to modernize the code responsible for building everything, with the goal of enhancing maintainability and making it easier to upgrade to newer versions of Gradle. In order to achieve this, they created a model of how they build their software and formalized it with convention plugins. This reduced the complexity of the build scripts, improved maintainability, and created a suite of unit and integration tests. Additionally, they wrote a tool to parse and rewrite build scripts in place, making the migration easier.

Q&As

What is the size of Square's Android mega-repo?
Square's Android mega-repo is composed of 2 million lines of Kotlin and 1 million lines of Java spread across more than 3,500 Gradle modules.

What is the goal of the project to modernize the build logic?
The goal of the project to modernize the build logic is to enhance maintainability, make it easier to upgrade to newer versions of Gradle, eliminate cross-project configuration, regularize the build scripts, and create a model of how the software is built.

How do the convention plugins help reduce cognitive load?
The convention plugins help reduce cognitive load by eliminating boilerplate and applying common conventions, such as jvmTarget for compilation, which are baked into the new plugin.

How has the complexity of the root build script changed?
The complexity of the root build script has been reduced from 41 to 16, a 61% reduction.

What is the benefit of publishing build plugins instead of using an included build?
The benefit of publishing build plugins instead of using an included build is that it eliminates a performance regression and improves build engineer productivity.

AI Comments

👍 This article provides an interesting insight into the process of herding elephants - wrangling a 3,500-module Gradle project. The author provides a thorough overview of the process and the many gains that were achieved from it.

👎 This article provides an overwhelming amount of detail about the herding elephants project, making it difficult to gain an understanding of the process.

AI Discussion

Me: It's about Herding Elephants, Wrangling a 3,500-module Gradle project. It talks about the efforts Square's Mobile Developer Experience Android (MDXA) team went through to modernize their build logic. They wanted to improve the maintainability, reduce the dependency of external configuration, and reduce the cognitive load of their feature engineers.

Friend: That's a lot of changes. What were the implications?

Me: Well, they implemented a model of how they build their software, which was formalized in the form of convention plugins. They also created a suite of integration tests to give them confidence that the build logic would work for upcoming releases. They also moved their build logic from buildSrc to an included build and then to binary plugins to reduce the configuration time. Finally, they wrote a tool based on Groovy AST transforms to parse and rewrite their gradle scripts which made it easier to migrate their 3,500 modules.

Friend: Wow, that's a lot of work! I'm sure they're reaping the rewards of their effort. What kind of performance improvements did they see?

Me: They saw a reduction of 61% in the cyclomatic complexity of their root build script and a reduction of 75 to 0 for another very complex script. They also saw a reduction of 30s (about 33%) in the configuration phase for every single build when they published their plugins to an Artifactory instance. Furthermore, they estimated a savings of over $100,000 per year in recovered developer productivity by replacing a Gradle task that took an average of 2 minutes to run with a Kotlin app that took about 300 ms.

Action items

Technical terms

Gradle
A build automation system used to compile and package Java and other programming languages.
LOC
Lines of code.
Point of Sale
A system used to process payments for goods and services.
YAML
A data serialization language used to store and exchange data.
Single Responsibility Principle
A software design principle that states that a class or module should have one, and only one, reason to change.
Intention-Revealing Interfaces
A software design principle that states that the interface of a class should be as clear as possible, so that the purpose of the class is immediately obvious.
Composition Over Inheritance
A software design principle that states that it is better to use composition to create objects than to use inheritance.
Facades
A software design pattern that provides a simplified interface to a complex system.
Adaptors
A software design pattern that allows two incompatible interfaces to work together.
CodeNarc
A static code analysis tool for Groovy.
Spock
A testing and specification framework for Java and Groovy.
Gradle TestKit
A testing framework for Gradle.
JUnit5
A testing framework for Java.
Build-Logic
A collection of Gradle modules.
Artifactory
A software package repository manager.

Similar articles

0.8394928 Non-Traditional Project Planning

0.83910674 Introduction

0.8213895 Manage morale, not metrics, for more effective engineering teams

0.82116896 Software engineers hate code.

0.82098764 Absurd Success

🗳️ Do you like the summary? Please join our survey and vote on new features!