Skip to main content
The Incredible Necessity of Android Automated Testing

This post is from the Experitest blog and has not been updated since the original publish date.

Last Updated Mar 24, 2018 — Continuous Testing Expert

The Incredible Necessity of Android Automated Testing

Continuous Testing

In the digital world Android automated testing is as important as ever. Android testing tools are abundant with Appium, gaining rapid popularity. Enter Appium Studio by Experitest, an Android app testing tool that takes the capabilities of Appium to the next level.


The global dominance of Android can be a bit of a mixed blessing for Android App Developers. The numbers all point to Android as the dominant force in global smartphone operating systems. IDC, in their Smartphone OS report from May 2017 1 puts Android's worldwide market share at 85.1% compared to Apple at just 14.7%. This gives developers access to an incredible number of potential users.


StatCounter, in April of 2017 2 went so far as to say that Android had even surpassed Microsoft Windows as the leading global operating system. Much of that global success comes from the growth of Android in emerging markets where price is an obvious factor and people are forgoing the purchase of personal computers in favor of smartphones.


Android App Testing Android Fragmentation


Testing Android apps is a challenge


There are two billion Android users worldwide, more than 4,000 different Android devices on the market being sold by over 400 different device manufacturers. The OSes are not standard either as the device manufacturers often release devices with customized versions of Android that are updated infrequently. This fragmentation of device models, OS versions, and different network conditions makes compatibility testing a big challenge.


This is made even more complicated as the cycle of new device purchases slows. Dan Luu, a senior engineer at Microsoft reported in 20173 that more than half of the Android devices in use worldwide are more than two years old which makes their OSes out of date.


He also mentioned that there are as many as 8 major versions of Android in circulation worldwide. But when looking at the billions of Android users, the long tail of users that are using rare or under supported OS versions may still represent a large number of users. They are also running on OSes and devices that may have potential security risks due to either the uniqueness of the OS or the age of the device. These users that have not adopted your app to their platform could potentially affect your rating in the app store.


These challenges for Android app developers have as much to do with device fragmentation as it does market trends. The version cycle for apps is as short as two weeks in some cases and so those apps and updates need to be ready to be released to the market on time. They also need to operate properly on any number of device and OS combinations. It is that fact that can lead to delays if testing is not carried out quickly and effectively.


As mentioned above, if your app or update is not released in time and/or does not work on certain device and OS combinations your app will be rated poorly and abandoned in favor of competing apps that work properly.


There are many ways that developers and testers can mitigate these risks, and even come out ahead of the game when it comes to releasing new apps and updates in a fragmented Android world.


Android App Testing Android Versions


Android automated testing is the key


Apps must be tested using robust Android automated testing to ensure they are compatible with any device/OS combination in a short period of time.


Developers or test engineers must be able to use Android testing tools to develop an automated test suite that covers all of the scenarios related to their Android application. The purpose is to reduce their time-consuming dependence on manual testing. Test automation development tools must be able to support the most complex scenarios such as barcode/check scanning, Touch ID, and GPS simulation.


More than that, in order to achieve short testing cycles and increase testing coverage automated test suites must be executed on a large number of mobile devices simultaneously. And not just emulators, but real physical Android devices. Only this type of execution can accurately expand coverage and increase speed.


Executed tests must run continuously to make sure all scenarios are tested in a short period of time. This includes regression testing. To achieve a short testing cycle DevOps teams usually use and Android testing framework and implement CI/CD methodology in order to trigger test execution directly from their Continuous Integration environment (e.g. Jenkins and TeamCity) to create a 24/7 testing process.


Appium open Source


One of the most popular open source automation tools is Appium. It allows developers and testers to create Android automated testing projects on both Android and iOS devices just not at the same time. Based on the Selenium Webdriver its operation is something that is familiar to many who already use Selenium. With Appium, testers can install applications on devices, automate and simulate many different use cases. In addition to testing on native and web apps, you can test on hybrid apps and browsers. The devices used in Appium testing can be connected locally, remotely, or even emulated.


Appium is far from the perfect Android app testing tool for test automation, however.


Appium limitations


Being an open source software with no dedicated support there is a lot of functionality and day-to-day convenience that can make Android automated testing an unwieldy process.


Only the highest skilled programmers and developers are able to work with Appium, due to its limitations. They need to be true Appium experts.


For starters installation is difficult and time-consuming. In addition to that, in order to increase Appium's core functionality, you must connect different systems and third-party apps which makes the entire process prone to error


There is also an insufficient GUI which makes test development reliant on detailed test scripting which slows the process and leads to buggy tests. Debugging a test is also near impossible because of a lack of visual tools like a device reflection and test reports. The lack of object identification tools like the object spy or XPath identifiers makes scripting that much more difficult.


The final and perhaps most challenging aspect of running automated testing with Appium is the fact that since it is an open source software, there is no telling when new operating systems will be compatible with Appium. Sometimes users need to wait for weeks or even months to be able to test their apps' compatibility with a new OS and in the digital age, that kind of a delay cannot be tolerated.


Appium Studio


To meet these challenges, an enhanced version of Appium open source, Appium Studio was created to help organizations release high-quality mobile apps more quickly, with effective Android automated testing.


Android App Testing Appium Studio UI


Appium Studio can be downloaded as a free community edition from the Experitest website and installed with one click. The entire process takes just a few minutes, and once installed users will be able to make use of free support through the Appium Studio Forum.


Appium Studio's functionality is broken down into three main categories.


The first is a visual test development tool that provides its users with a test recorder, device reflection, and object spy. This helps testers write robust tests in a fraction of the time it once took.


Android App Testing Appium Studio Visual Test Development


Second is the quick and easy setup we discussed above.


The third category is the Appium Execution Engine which is an execution engine that is designed for mobile. The engine enables test execution, out of app testing, and parallel execution.


Android App Testing Appium Studio Execution Engine


This combination helps Appium Studio mitigate if not eliminate many of Appium's limitations. It allows users to start testing within minutes, reduces costs and effort by enabling testing of iOS apps on Windows machines, and helps improve coverage with out-of-app testing. Appium Studio has a simple and easy to use UI to make test development simpler, and developers can test any use case, including testing outside of an app.


Tests, and executions, are much more stable and results are presented in clear reports. Lastly, Appium Studio supports large-scale parallel execution and comes with guaranteed OS version support.


The functionality of Appium Studio is pushed to the next level when you connect to the SeeTest Digital Assurance Lab. This gives testers access to a grid of real mobile devices that can be accessed remotely and tested on. Testing using the lab gives you the ability to carry out your tests on many devices in parallel. This works fantastically for anyone who is running a continuous testing environment. Another added capability is that while tests are being executed you can log into the lab and see the tests being carried out in real time on the devices you chose.


Android App Testing Digital Assurane Lab


Once testing is complete. Appium Studio automatically generates reports that contain step-by-step results including screenshots, device logs and non-functional data like device vitals consumption. Appium Studio reports are fully shareable between DevOps team members and lead to faster bug investigation and root cause analysis which cuts down on resolution times dramatically.


Final Thoughts


Android testing is an extremely important aspect of releasing or updating a mobile app. The fragmentation of the Android market has made keeping track of devices models and OSes while releasing high-quality apps on time a difficult task. While Appium is a solid piece of testing software that can help Android testers, but its lack of functionality led to the creation of Appium Studio, the free, easily downloaded automation software from Experitest that can truly take your automation project to new heights.

1. [IDC Smartphone OS Report]2. [Statcounter Android overtakes Windows]3. [Dan Luu Android Updates]

Take a free trial of our Mobile Application Testing Tools:Free trial

Follow our Digital App Testing page on LinkedIn


The Incredible Necessity of Android Automated Testing

Jonny Steiner - Content Manager


More from the Blog

View more
May 13, 2022

What tips would you give testers on how to ask better questions?

Continuous Testing
The nature of testing is not about confirmation. It is about questioni ...
Read More
Mar 23, 2022 Continuous Testing, First in Market to Support Android 13 DP

Continuous Testing is proud to announce that our Continuous Testing solution i ...
Read More
Mar 21, 2022

Eliminate inefficiencies in your enterprise with codeless and continuous automated testing

Continuous Testing
Today, terms like codeless, continuous automated testing and shift lef ...
Read More
Feb 07, 2022

These key factors will help you choose an automation tool

Continuous Testing
The year 2022 looks to be another banner year of growth in the softwar ...
Read More
Contact Us