This post is from the Experitest blog and has not been updated since the original publish date.
How to identify you're testing on a real device, and why is it even important?
Superb UX comes with flawless testing
In today's world, end users have become accustomed to a superb user experience when using their mobile applications. They expect everything to run smoothly, quickly and usefully, on every device they have and under almost every network condition.
This has made mobile and web application testing a critical phase in the app-release lifecycle. Functional problems that were not detected while testing, may result in loss of customers, reputation and revenue to the organization. Therefore, companies spend a significant amount of time and effort on testing applications on all types of devices, operating systems and browsers, assuring nothing gets broken.
The testing challenge of the digital era
In the digital era we live in, the speed with which new mobile devices and operating systems are released is hard to follow. Testers need to have access to numerous devices and operating systems, and the different combinations they might have. So instead of acquiring and maintaining such a large array of devices, more and more organizations are transforming to testing on devices hosted in the cloud – either real, emulated or simulated. The problem is, that when connecting to a cloud-based device, it is hard to determine whether the allocated device is a real one, an emulator or a simulator.
The importance of testing on a real device
Although emulators and simulators might seem like a good enough option for testing, there is actually great importance given to testing applications on real devices. The problems with emulator/simulator testing are a bit different between iOS and Android-based devices:
- Testing apps on Android emulators is tricky since every device manufacturer performs some changes to the operating system - what is known as "Android fragmentation". However, all Android emulators run the original Android OS, and therefore cannot emulate anything changed by the device manufacturer, which can, of course, affect the application.
- Another issue is that actual physical components on the device make it behave differently than the emulator. The emulator's software cannot accurately imitate the behavior of real hardware such as GPS, camera, CPU, USB connection, etc. so every test that interacts with hardware will not exactly reflect the real-life behavior.
- Real device vitals (CPU usage, temperature, battery consumption, and memory) cannot be measured and monitored using an emulator and might be affected in real life. Measuring these parameters is crucial for testers, as end users will not accept applications that affect their device's overall performance.
- Emulators are jailbroken by nature. This means that any application security features (such as jailbreak detection, fingerprint authentication etc.) will not work on an emulator.
- Another surprising fact is that despite what people tend to think - emulators are actually slower in their interaction speed and in the initialization of a new session. The reason is that they're using virtualized hardware, whereas today real device hardware is very strong.
iOS devices (or Apple devices):
- iOS does not have any emulators, only simulators, which simulate software only and not hardware. The iOS simulators don't run the actual iOS operating system, but rather a simulation of it that mimics the responses expected from the OS. This means that the application is not even tested on the real operating system. Additionally, the actual application that will be installed on the real device cannot be installed on the simulator. It has to go through a different compilation in order to run on the simulator, and the result of this is that the application's behavior on the simulator and on the real device can eventually be significantly different. This makes simulator-based tests untrustworthy.
- Similar to Android, device vitals are extremely important to test, and cannot be measured and monitored using a simulator.
- iOS simulators, similar to Android emulators, are actually slower to interact and initialize compared to real devices.
A real, legitimate device, not jailbroken or rooted
So at this point, it seems quite clear that performing tests on real devices is a must. Another issue to consider is testing on real licensed devices, and not jailbroken or rooted ones. The issue with breaking/rooting the device is that such an activity changes the operating system. Some processes will work well on a jailbroken/rooted device, but not on a real one. Another issue is security – devices that were jailbroken or rooted are not secure and enable protocol cracking and reverse engineering. Therefore, organizations that want to stay on the safe side should not enable their app's installation on jailbroken or rooted devices, let alone test their applications them.
How do you identify a simulator, emulator or a jailbroken/rooted device?
Apple, who like to be in full control of their products, have applications that cannot be deleted from the device. If you find a device that has no "Settings" or "Clock" applications – the device is either not real, or jailbroken.
|Photo application found on a phone||Clock application not found on a phone indicating this is a simulator or a jailbroken device|
Trying to use the phone's camera and checking whether you can take an actual image is another way of knowing it's a real device with a real physical camera.
Another hint that the device might be jailbroken is the iOS version. Apple continuously releases iOS versions and updates to overcome security issues that may be exploited to jailbreak the device. If the device is on an early version of iOS (e.g. iOS version 10.3 while 10.4.1 has already been released), and the software update is blocked, there's a high probability the device is jailbroken.
Android is a bit trickier, as Google allows performing changes to the operating system. In order to assure the device is real you must go into:
Within that path, you must verify there's a utility called: "Developer options". If you cannot find this utility, you're running on an emulator.
|Real Android device with Settings->System-> Developer options utility||Emulator – with no "Developer options" under Settings->System|
Another option for identifying an emulator is opening the camera application, and checking whether it's possible to take an actual image. This way you'll know whether the device has a real physical camera.
Identifying a rooted Android device is a little more difficult, but there are a couple of tools available for installation that can perform that check for you, see links at the bottom of this post.
Another option for checking if it's a real device, for both Android and iOS, is verifying the device has a serial number, which assures it is real.
|Android serial number||iOS serial number|
Always check before testing
To achieve the best and most reliable results in testing, be sure that you test your application on real devices. When testing on a cloud platform, use the techniques provided above to verify the device allocated to you is real, so you can send your results and recommendations with peace of mind and full confidence. Check out the links below for additional information and tools to tell a simulator/emulator-jailbroken/rooted-real trusted device apart.
Identify an Android emulator:
Identify an iOS jailbroken device:
Identify an Android rooted device:
Test your application on hundreds of real mobile devices.Start your free trial now.