Bitrise vs. CircleCI for Android in a Head-to-Head Battle

In mobile development, usually, developers are the ones who will set up and maintain CI/CD. In that sense, the best CI is the one they waste the least time on so they can focus on development more.

The Android team here at Infinum have been using CircleCI for a few years and recently migrated the majority of projects to Bitrise—a newer continuous integration and delivery (CI/CD) service made for mobile apps. This means that it’s focused just on that and provides great service in that field.

On the other hand, while CircleCI provides a more generic solution suitable for any type of project, it might require just a bit more configuration.

In CircleCI’s free option you get one container with 1000 build minutes per month. Meanwhile, the free option in Bitrise features somewhat limiting 10 minutes per build.

For concurrencies, both Bitrise and Circle have similar pricing:

If you need 2 or 3 concurrencies, you might want to consider the difference in the pricing, as Circle can be cheaper. Otherwise, both are in the similar price range so that shouldn’t be the deciding factor.

There’s no point in comparing the pricing without looking at what you get for the money.

Both machines have similar Intel Xeon and you’ll get 2 vCPU. The only difference is that on Bitrise you’re getting 8GB of RAM compared to Circle 4GB, which can be a big deal. These are the standard options and you can pay for additional ones.

Need more power? Bitrise standard option with 8GB of RAM should be enough for most projects. If you need more power, you can get elite machines which are pretty much double the resources (4 vCPU, 16GB RAM) for double the money.

This doesn’t mean that your build times will be two times as fast. Gradle tasks don’t fully utilize all the resources and multithreading so from our experience you can expect about 20% faster builds.

On CircleCI, the standard option with 4GB might just not be enough for some bigger projects. Unlike Bitrise that has only 2 options (standard and elite), CircleCI has 5 resource classes:

Something you should be aware of is that there is no public pricing for CircleCI resource classes. You need to personally contact them to gain access. In our case, we got them for free for a limited time but then got unpleasantly surprised with the full price of the extra resources.

The way CI was setup usually involved writing configuration files in JSON or YAML notation. For bigger projects, this config file would grow to levels that are hard to maintain.

Bitrise has a somewhat different approach to configuration. To start with, you don’t need to have a configuration file in your repo—you can just hold it in their web app. But Bitrise’s edge is the visual configuration editor which makes the whole setup much easier.

Each step is represented as one block in the editor. And with this, you don’t need to edit configuration files directly, you can just click through the visual editor. This is also much easier for developers who have never worked with a CI before. It takes time to learn how to read and write configuration file, and the editor is self-explanatory. Some of my colleagues didn’t like this editor as they were used to handwriting YAML files and didn’t want to switch. They can still continue to write files without using the visual editor.

Bitrise comes with the library of steps which contains a lot of predefined steps you might need, like Install missing Android dependencies, Android Build, Android Lint, etc. You can write your own step and open a request to merge it into the library.

Similar to the Bitrise library of steps are the CircleCI orbs. You have a public registry of orbs. Orbs are part of CircleCI 2.1, so they are a pretty recent thing. The problem I see here is that users don’t really need orbs and don’t want to bother updating to 2.1, so the adoption of orbs is low. Currently, there is just one useful orb for Android while on Bitrise there already are a lot of useful steps, with new ones coming regularly.

One thing I would like to see is the possibility to write and share a step internally within the organization, which currently isn’t possible on either Bitrise or CircleCI.

For some projects, we need to build several flavors of application. For these, we use parallel builds. Both Bitrise and CircleCI support parallel builds and are easy to set up. For Bitrise, you would have one workflow that initiates all the parallel workflows and waits for them to finish. This way, one container will always be used just to wait for others, which is not the case on CircleCI. That’s a plus for Circle!

UI testing has always been one of the trickier things to set up on your CI/CD servers. A few years back, you had to create emulator image and start it, run UI tests, and then read text reports and somehow fetch that result back into the CI report.

There was a lot of room for this process to fail just because something went wrong while starting an emulator, and then your tests would be reported as failed. Plus, running emulator required a lot of resources on the build server.

In 2016, Firebase Test Lab was announced, which made things easier because it eliminated the need to work with an emulator. Still, it continued to be a pretty lengthy setup. There were multiple requirements—a Firebase project, hooking up the Google Cloud keys (for remote access to your project), and then running a few commands to send apk and run tests. This was the standard procedure in CircleCI, and this alone would add 30 lines to your .yml configuration.

With Bitrise, setting up UI testing is simple as it gets. There are two steps ready: “Android Build for UI Testing” and “[BETA] Virtual Device Testing”. You add these and that’s it.

If you want to fully automate the whole process, you will probably want to deploy directly to Play Store from your build server. Using CircleCI, the best approach would be either to use fastlane or some bash script you can find online that does all the communication with Play Store to publish the .apk.

This is simplified on Bitrise with Google Play Deploy step. It accepts .apk or .aab and Play Store Service Account JSON (which is required for any remote access to the Play Store). So there’s another plus for Bitrise!

Bitrise workflow editor is a really innovative approach that saves us from .yml files and brings a simple interface that I enjoy using. The focus on mobile development is a big advantage and I can see this paying off even more in the future.

CircleCI looks to be taking a different approach. With 2.x, there are some changes that you really don’t need but can be very helpful in complex environments. Finally, if you don’t have a CI/CD and you are doing mobile development, Bitrise should be your first choice.