The Firefox for Android build system in 2015

Making it easier to build Firefox for Android the first time

One strong claim made by mhoye — and supported by many others in the room — is that mach bootstrap has significantly reduced the technical accessibility barrier to building Firefox for Desktop. We need to implement mach bootstrap for Firefox for Android.

For those who don’t know, mach bootstrap is a script that prepares the Firefox build environment, including fetching, updating, and installing the pre-requisites needed for building Firefox. It automates the (often difficult!) task of fetching dependencies; ensures that known-good versions of dependencies are installed; and sets development environment defaults. mach bootstrap is the first thing that should be run in a fresh Firefox source tree [2].

Firefox for Android has more complicated dependencies than Firefox for Desktop, including some that cannot be easily distributed or installed: the Java development kit and run-time environment, the Android SDK and NDK; Google’s Play Services libraries, etc. We can save new contributors a long dependency chase before they see a positive result. In addition, seasoned developers spend an unknown-but-large amount of time discovering that the required dependencies have advanced. Pinning the build to known-good versions, failing the build when said versions are not present, and providing mach bootstrap to update to known-good versions will reduce this frustration.

A contributor started writing a shell script that does the work of mach bootstrap. Bug 1108771 tracks building upon this effort. I’ve also filed Bug 1108782 to track pinning the Firefox for Android build requirements to known-good versions.

Reducing the Firefox for Android edit-compile-test cycle time

Firefox for Android is an unusual Android application: a large C++ library backing a medium-sized Java front-end, all plumbed together with a JavaScript-based message passing system. Right now, building the C++ library takes roughly 12 minutes on my system. Building the Java front-end takes roughly 2 minutes, and the JavaScript parts are essentially free. In 2015, glandium has taken a first quarter goal to make it possible to build Firefox (for Desktop and for Android) without building that large C++ library at all [3]. In the future, purely front-end developers (XUL/JavaScript developers on Desktop; Java/JavaScript developers on Android) will download and cache the C++ build artifacts and build the application on top of the cached artifacts. Firefox for Android is really well-suited to this mode of operation because our dependencies are so well-defined. I’ve filed Bug 1093242 to track part of this work.

The previous work will make it faster to build Firefox for Android the first time, because we won’t build C++ libraries. We’re also going to invest in making each incremental build faster, and there’s some low-hanging fruit here. Right now, the most costly parts of our build are compiling individual JAR libraries and DEXing all of the resulting JAR libraries. Every time we split our JAR libraries, we can parallelize a small part of our build and reduce the wall-clock time of our Java compilation. Right now we could split our single third-party JAR library and save ourselves compile time. And we’re very close to being able to split the Background Services (Sync, Firefox Accounts, Firefox Health Report, etc) library out of Fennec proper, which will save even more compile time.

Improving our DEXing time is more difficult. Android’s DEX processor is a byte-code transformation step that turns Java’s byte-code into Dalvik VM byte-code. For historical reasons, we DEX the entirety of Fennec’s byte-code in one DEX invocation, and it’s both a single-process bottleneck and terribly expensive. For some time, it has been possible to DEX each individual library in parallel and to merge the resulting DEX files. All modern Android build system (such as buck or Gradle) support this. We could support this in the Firefox for Android build system as well, but I think we should move to use a more featured build system under the hood instead. Android build systems are very complicated; we don’t want to write our own, and we definitely don’t want to write our own in Make syntax. In 2015, we’ll push to use a full-featured build tool that brings this DEX-time improvement. More on this in a future post.

Making the Firefox for Android development process "standards compliant"

This point is a more philosophical point than the others. Firefox for Android wins when we engage our community. The community of Android developers is large and constantly pushing the boundaries of what’s possible on a device. We want to tap into that well-spring of talent and innovation, and every thing we do that’s non-standard to an Android developer makes it harder for us to do this. Contributor @fedepaol wrote a blog post about how difficult this used to be.

The good news is, we’re getting better: we have rudimentary Gradle support and you can use IntelliJ now. But we still have a long, long way to go. We’ve got lots of easy wins just waiting for us: tickets like Bug 1107811 will go a long way towards making the Fennec "Android standards compliant" IntelliJ experience better. I have a technical plan to land in-source-tree IntelliJ configurations, so developers can open mobile/android directly in IntelliJ and get to a working Firefox for Android APK in the IDE in one step.

At a lower level, Tickets like Bug 1074258 will let us use the IntelliJ design view more easily, and landing Android JUnit 3 Instrumentation test runners in automation (Bug 1064004) will make local testing significantly easier than the convoluted Robocop process we have right now. The list goes on and on.