Build Fennec frontend fast with mach artifact!

Quick start

It’s easy! But there is a pre-requisite: you need to enable Gregory Szorc’s mozext Mercurial extension [1] first. mozext is part of Mozilla’s version-control-tools repository; run mach mercurial-setup to make sure your local copy is up-to-date, and then add the following to the .hg/hgrc file in your source directory:

[extensions]
mozext = /PATH/TO/HOME/.mozbuild/version-control-tools/hgext/mozext

Then, run hg pushlogsync. Mercurial should show a long (and slow) progress bar [2]. From now on, each time you hg pull, you’ll also maintain your local copy of the pushlog.

Now, open your mozconfig file and add:

ac_add_options --disable-compile-environment
mk_add_options MOZ_OBJDIR=./objdir-frontend

(That last line uses a different object directory — it’s worth experimenting with a different directory so you can go back to your old flow if necessary.)

Then mach build and mach build mobile/android as usual. When it’s time to package an APK, use:

mach artifact install && mach package

instead of mach package [3]. Use mach install like normal to deploy to your device!

After running mach artifact install && mach package once, you should find that mach gradle-install, mach gradle app:installDebug, and developing with IntelliJ (or Android Studio) work like normal as well.

Disclaimer

This only works when you are building Fennec (Firefox for Android) and developing JavaScript and/or Fennec frontend Java code! If you’re building Firefox for Desktop, this won’t help you. If you’re building C++ code, this won’t help you.

The integration currently requires Mercurial. Mozilla’s release engineering runs a service mapping git commit hashes to Mercurial commit hashes; mach artifact should be able to use this service to provide automatic binary artifact management for git users.

Discussion

mach artifact install is your main entry point: run this to automatically inspect your local repository, determine good candidate revisions, talk to the Task Cluster index service to identify suitable build artifacts, and download them from Amazon S3. The command caches heavily, so it should be fine to run frequently; and the command avoids touching files except when necessary, so it shouldn’t invalidate builds arbitrarily.

The reduction in build time comes from --disable-compile-environment: this tells the build system to never build C++ libraries (libxul.so and friends) [4]. On my laptop, a clobber build with this configuration completes in about 3 minutes [5]. This configuration isn’t well tested, so please file tickets blocking Bug 1159371.

Troubleshooting

Run mach artifact to see help.

E GeckoConsole(5165)          [JavaScript Error: "NS_ERROR_XPC_GS_RETURNED_FAILURE: Component returned failure code: 0x80570016 (NS_ERROR_XPC_GS_RETURNED_FAILURE) [nsIJSCID.getService]" {file: "resource://gre/modules/Services.jsm" line: 23}]

Acknowledgements

This work builds on the contributions of a huge number of people. First, @indygreg supported this effort from day one and reviewed the code. He also wrote mozext and made it easy to access the pushlog locally. None of this happens without Greg. Second, the Task Cluster Index team deserves kudos for making it easy to download artifacts built in automation. Anyone who’s written a TBPL scraper knows how much better the new system is. Third, I’d like to thank @liucheia for testing this with me in Whistler, and /u/vivek for proof-reading this blog post.

Conclusion

In my blog post The Firefox for Android build system in 2015, the first priority was making it easier to build Firefox for Android the first time. The second priority was reducing the edit-compile-test cycle time. The mach artifact work described here drastically reduces the first compile-test cycle time, and subsequent compile-test cycles after pulling from the upstream repository. It’s hitting part of the first priority, and part of the second priority. Baby steps.

The Firefox for Android team is always making things better for contributors! Get involved with Firefox for Android.

Discussion is best conducted on the mobile-firefox-dev mailing list and I’m nalexander on irc.mozilla.org/#mobile and @ncalexander on Twitter.

Changes

• Wed 1 July 2015: Initial version.
• Mon 6 July 2015: fix typo in link to Vivek. Thanks, sfink!

Notes