How to do Android Automated Testing

Verifying the behaviour of our applications is an essential part of our day-to-day development. This blog will discuss two distinct stages of automated testing. Unit tests and acceptance tests.

If you are using Unit Testing as part of Test Driven Development (TDD) process, then it is important that the Unit Test framework runs in a matter of seconds. Unfortunately, the standard Android Testing Framework can be a bit of a disappointment in that respect. It requires a running device/emulator to invoke any code that uses Android SDK methods.

Therefore, my preference is to use Robolectric. Robolectric is an Open Source Unit Test framework which allows you to run tests which use the Android API directly on the JVM within your IDE. Therefore, it enables as much real behaviour as possible whilst being fast enough for TDD.

For acceptance testing I am a big fan of Calaba.sh. Calaba.sh is an opensource automated acceptance testing tool for Android and iOS that supports Cucumber style tests.

Cucumber allows us to create tests using a natural language that can be understood by non-technical QA staff as well as developers. Tests typically follow the Behavior-Driven Development (BDD) style of Given, When, Then, which any layperson can understand.

To use calaba.sh you create a feature file (using this natural language) that essentially becomes an executable test use case of your app. Each feature file will contain one or more scenarios that correspond to a possible outcome of the given use case. Within the use case, you can specify various end-user actions (such as ouch, swipe, scroll, etc) or make an assertion (such as “look for the following text…”).

Calabash can be further extended and used alongside a cloud based mobile device testing service, such as Appthwack, to create a truly powerful testing setup. This means you can run your acceptance test on a wide range of the most common Android handsets at the touch of a button. Adding a take screenshot step will provide a snapshot of how your app looks at a given point in the test, meaning it can also be visually inspected for any graphical errors. This is highly useful for seeing how your app looks across numerous devices without having to manually run it on each one.