Automated mobile application testing has always been problematic for several reasons: tests are flaky, we get different results depending on the device we test the app on, we need to deal with system alerts and pop-ups informing about system update or any permissions. These all lead to the conclusion that test maintenance is more expensive than writing them, which doesn’t make any sense. When starting a new project I thought of automating the tests. At this time, I was working on a mobile app written in React Native, and the first thing I did was checking what other specialists use to test these types of apps. Detox appeared to be quite popular, recommended by many people, thus I decided to try this tool out.
In the following article, I would like to share everything I learned about the tool. In particular, I will focus on describing how detox works, how to configure it, how to write tests and how to make them easily readable and maintainable.
Detox is a gray-box end-to-end testing and automation library for mobile apps (specifically dedicated for apps written in React Native). It’s important to underline and highlight that detox is a gray-box, not black-box, which means:
- it allows the test framework to monitor the app from the inside and actually to synchronize with it,
- it runs in the same process, has access to memory, and can monitor the execution.
Detox uses native gray-box drivers developed by Google — EarlGrey for iOS and Espresso for Android. These frameworks synchronize with the application and interact with the application only when it’s idle.
Getting started
It appears to be quite easy to start writing tests in Detox. Nevertheless, you need to make sure you have all required dependencies installed:
- Mac with macOS (at least macOS El Capitan 10.11)
- Xcode 8.3+ with Xcode command line tools
- A working React Native app you want to test
- Node >= 7.6.0
- applesimutils
- detox-cli
- Mocha/Jest as a test runner
After that, you can install detox using npm. In the next step, in a package.json file, you need to add configurations you are going to run your tests on. In my case, there are two configurations: dev (called ios.sim.dev) and test (ios.sim.test), which represent the environment the app is launched on. We use 3 different environments in the project, 2 of which are used in the development:
- DEV — all reviewed changes are merged into this branch. Tests are automatically run after each merge.
- TEST — all changes and features completed within sprint timeframe are merged into this branch. Tests are run automatically after the merge. The build might be distributed to beta testers.
- PROD — production environment.
GIST: https://gist.github.com/lkosiorowski/141ac65ccd63393056b7bc4ec688442b.js
You can open a terminal now, go to the folder with project repository cloned and type ‘detox init -r mocha’ (if you prefer mocha) or ‘detox init -r jest’ (if you prefer jest). It automatically creates:
- ‘e2e’ folder where tests will be stored,
- ‘init.js’ file where you can set the timeout for a test,
- ‘mocha.opts’ config file for mocha runner (‘config.js’ for jest),
- ‘firstTest.spec.js’ file with the exmaple test (‘firstTest.test.js’ for jest).
Writing tests
If you have followed the above suggestions, it seems you are ready to write tests for your application. However, I would recommend you to spend some time beforehand in order to identify test cases you would like to be automated (it is good to verify the ones which will not be automated, as well as keep them documented). It will assure you that all the critical paths of the application are covered and finally your tests bring the biggest value. Writing tests flow is very simple. Once you have scenarios to be implemented, you can identify elements (in React Native components are usually elements that are used) which will be used in your test. Use a proper matcher to match the element, perform an action on that element or on a group of elements and set expectation — the expected result of your test.
GIST: https://gist.github.com/lkosiorowski/a7b9a5d689099f6f11573c73e823b5c9.js
Detox gives us a lot of options to interact with the elements specific for a mobile application, like multi-touch, swipe and others. It also gives us an opportunity to interact with the device by using a device object. Thanks to that, we can, for example, launch the app with certain parameters (e.g. grant access to the camera), reload the app, terminate the app, etc.
During work on my test project, I was writing tests keeping in mind the following rules:
- start each scenario from a predictable application state. Using beforeAll, beforeEach, await device.reloadReactNative() methods can help with that,
-
GIST: https://gist.github.com/lkosiorowski/2fae5671134d4d3d4b3e94ab63759d35.js
- add testIDs to the elements you are going to use in the tests,
GIST: https://gist.github.com/lkosiorowski/559834dce9b9643afea6090080ca9ab6.js
- keep all locators in one place (changing them in the future will not cost too much time),
- create helpers method to make your code more legible and easily maintainable.
Screen object pattern
As mentioned above, I recommend keeping all locators in one place. It is easier to maintain the code and to write the tests when you group locators for particular screens. Being inspired by Page Object Pattern from web applications testing, I implemented some kind of Screen Object Pattern and stored all locators in one file. For each screen, I created a separate dictionary with the same name as the screen.
GIST: https://gist.github.com/lkosiorowski/45e7f73e3501e7e1321ec110bd22d158.js
It not only helped me to write the tests but also helped other people to understand what is being actually tested.
GIST: https://gist.github.com/lkosiorowski/50e3da017ac24ed9d6ec621cf93f7372.js
Running the tests
Tests can be run locally or on the CI server. Before executing the tests you need to build the application using ‘detox build’ method or just use binary already built in the build process on the CI server. After that, you can run all the tests using ‘detox test’ method with parameters. There you can specify what configuration tests should be run on.
Even if tests written in detox are stable and reliable, some of them may fail without obvious reason. Detox and React Native give us a few options to narrow down the problem and to fix failing tests quickly and easily:
- React Native Debugger — debugging tool helping to inspect elements in the application and to navigate through the views hierarchy. It also has an option to access application local storage, to check memory allocation or to analyse network communication.
- We can get more information about what is going on in our tests by increasing the level of details printed in the logs (e.g. ‘detox test -l trace’). During execution, there will be a lot of useful information printed in the console.
Conclusion
Detox has started a new era in testing mobile applications — the era of gray-box testing. The one where we do not need to worry about delays, freezes or any other unexpected behaviours occurring quite often when using mobile apps. Thanks to that, we can avoid tests flakiness and make our test more reliable, stable and trustworthy. Configuration is less complex than in case of any other mobile TA framework, thus it makes automated testing a lot of easier and, what is very important, it leverages cooperation between testers and developers.
Hero Image from https://marketingland.com/optimize-customer-mobile-experiences-strategy-execution-234435
