Glimpse into the Future of Mobile and Web Application Testing

Progressive web apps (PWAs) are becoming increasingly popular. PWAs offer the advantages of both mobile and web apps. However, testing of PWAs is markedly different. In this article, I will explain the future of mobile and web application testing in view of the emergence of PWAs. To that end, we will take a look at some of the more popular testing software out there for PWAs. I will also describe how we facilitate PWA testing with the SeeTest device and browser lab.

Progressive web apps (PWAs): An introduction

PWAs “combine the best of web and the best of apps”, as this Google Developers definition states. These are web apps that feel and function like mobile apps. We have mentioned PWAs in previous blog posts, but let’s recap for a moment. A PWA needs to meet the following criteria:

• Responsive.
• Speedy, even when the Internet connection is slow.
• Engaging, just like a native mobile app.
• Secure, i.e., running under HTTPS.
• Discoverable on the web, and linkable.
• Installable on the home screen.
• Always up-to-date.
• Works offline.
• Includes push notifications.
• Progressive: Every user should be able to use it no matter what browser they use.

Their higher speed, offline features, higher security, ease of fine-tuning combine with the native app-like experience will make PWAs even more important in the future. Starbucks, Housing.com, Flipboard are among popular examples of PWAs.

PWAs Replacing Native Apps

The main question facing developers and testers is if the growth of PWAs means the end of Native apps. In short yes, but in long, only sort of. What I mean is that native apps do provide a mostly superior experience due to the advanced nature of their development. The truth, however, is that the amount of time money and energy it takes to create and manage one is simply not worth it these days. Unless of course, your app is in the top 20 on the app store.

It is clear that users prefer to interact with digital properties through an app interface because let’s face it mobile browser speeds are not fast enough to provide the best browsing experience.

So the PWA then takes everything we love about a native app and packages it in an easier to develop a way that still provides the same functionality.

Add to that the fact that native app stores are overcrowded and that PWAs rank on google and you have a combination that is going to give native apps fits for years to come.

Mobile and web application testing in the context of PWAs

Testing PWAs is also becoming a simple process. Let’s take a look at the elements that make up a PWA.

The key technical building blocks of PWAs are as follows:

• The “Web App Manifest”: This JSON file influences the display, and has important fields like short name, name, description, start URL, etc.
• The “Service Worker”: A JavaScript file executing independently, which facilitates the offline usage of the app.
• PWAs run under HTTPS.

These building blocks influence mobile and web application testing. In addition to testing the functionalities, testers should validate the following, when testing PWAs:

• The Manifest file, by checking the presence of the following:
  • A “Name” or “short name” property.
  • A “Start URL” property.
  • An “icon” property with 192px and 512px icons.
  • The “Display” property set to full-screen, standalone, and minimal-UI.
• The “Service Workers”, by checking the registration of the service workers with the fetch event-handlers.
• The website must fully be on HTTPS.
• Cross-browser compatibility.
• Responsive design.
• Offline availability of the key features.
• Required metadata for installing on the home screen.
• Smooth page transition even when the Internet connection is slow.
• Valid and unique URL for each page.
• Availability of “Schema.org”.
• Whether Push notifications work and whether they ask for users’ permission.

Challenges with PWA testing

Open source testing tools like Selenium are quite popular for web app testing. However, they don’t fully address some of the challenges with PWA testing. These challenges are as follows:

•  The progressive nature of PWAs means that users will have access to more features if they use a more modern browser. On the contrary, users with older browsers can only use the features their browser supports. This requires testing with a wide variety of browsers. Selenium can’t address this adequately.
• Selenium WebDriver can’t run stand-alone on mobile operating systems.
• If the PWA uses AngularJS, then it’s hard to capture the web elements using Selenium WebDriver. AngularJS apps have a few HTML attributes like “ng-repeater”, “ng-controller”, “ng-model”, etc., and these aren’t included in the Selenium locators.

Tracking the future of PWAs with Open source testing tools

Following are a few examples of open source testing solutions for PWAs that will be leading the trends of the future:

Protractor

Protractor is an end-to-end testing framework for Angular and Angular JS apps. It’s a Node.js program built on top of WebDriver.js. It uses native events and browser-specific drivers to interact with the app. Testers can use it to interact with the app running on a browser.

Protractor can integrate with Selenium and other popular open-source testing frameworks. It uses WebDriver for controlling the browser with user actions. Testers can write automated regression tests for web apps.

Testers need not manually add waits and sleeps. Protractor automatically executes the next step in the test as soon as the webpage completes the pending tasks.

Protractor addresses the AngularJS-specific limitations of testing with Selenium that I mentioned earlier. However, it goes beyond just unit testing of Angular JS apps and focuses on testing the actual functionality.

WebdriverIO

WebdriverIO is a test framework for Node.js. It’s easy to set up, thanks to its’ “Command Line Interface” (CLI). The CLI includes a configuration utility. WebdriverIO includes other 3rd party packages. These include framework adaptions, reporting tools, and other services. The set-up process installs all of these.

There’s a simple interface to write tests and share them. The UI facilitates debugging. WebdriverIO is an extendable tool. Testers can easily add helper functions and complex sets and combinations of commands.

WebdriverIO is compatible with WebDriver and Appium, which helps in mobile and web application testing. It implements all WebDriver protocol commands. There is also integration with other tools. For example, it has integration support for Applitools Eyes, an AI-powered visual testing tool. This helps testers write visual regression tests.

Mocha

Mocha is a JavaScript testing framework. It runs on Node.js and on the browser. Its’ simple UI makes both synchronous and asynchronous testing easy. There is a CLI that supports all the key features. There are also editor plugins like JetBrains, and testers can use them to write tests quicker.

Testing teams can run their tests in a sequence. Mocha.js maps uncaught exceptions to the correct test cases. There are supports for async test timeout, test-retry, test-specific timeouts, Node debugger, etc. This framework can highlight slow tests.

Mocha provides flexible and accurate reports. It can report test coverage, test duration, and other important metrics. The reports are available on different formats including the default format, lists, progress bar, summary-only reports, etc.

Jasmine

Jasmine is a testing framework for JavaScript. Testers can run it on JavaScript-enabled platforms. It’s non-intrusive as far as the app and IDE are concerned. It’s a low-overhead framework since Jasmine-core has no dependencies.

Testing teams can run browser tests and Node.js tests using the same framework. Apart from Node.js, Jasmine works for Python-based web projects (Django, Flask, etc.), and Ruby projects (with or without Rails).

Jasmine supports asynchronous testing. It has an extension called “Jasmine-jQuery”, which makes front-end testing easy. Jasmine has built-in reporters, however, testers can create their custom reporters.

Another advantage of Jasmine is its’ set of built-in matchers. The set includes “toBe”, “toBeTruthy”, “toBeFalsy”, “toContain”, “toBeDefined”, “toBeUndefined”, “toBeNull”, “toBeGreaterThan”, “toBeLessThan”, and more. These matchers return a Boolean value. The value is “True” if the expectation is matched, i.e., the test has passed. A “False” value indicates that the test has failed.

SeeTest “Mobile Device & Browser Lab”: Making PWA testing easier

While the above open-source frameworks aid in PWA testing, you still need to access a wide variety of browsers and mobile devices. This is where SeeTest “Mobile Device & Browser Lab” makes mobile and web application testing easier.

The lab makes real devices and browsers easily available to testing teams. Testing of PWAs needs to validate whether users can progressively access features supported by their browsers. The wide variety of devices and browsers makes it easier. It’s also easy to add new devices and operating systems.

You can choose from any of the following deployment models:

• On-premise lab: This enables you to host the lab in your own data center. The set-up is easy, and you get advanced management capabilities.
• Hosted private lab: In this model, you have a private lab for yourself, while we host it in our data center. We have data centers spread across the world, therefore, you experience no latency. The security is high, and you get advanced management capabilities.
• Public lab: You get access to over 1,000 devices and browsers. The public lab offers high security, and there is no latency.

You can easily integrate our mobile device and browser labs with your choice of open-source solutions. This substantially benefits your PWA testing.

So what does it all mean

We live in a world of quality, speed, and ease. Businesses want to release apps at a fast pace to meet the demands of their users, and users want simple and effective apps that make their lives easier. It seems too simple and yet in the overcrowded world of the native app, it’s not. First, you must develop a nearly flawless app then you need to release it. Once you accomplish that you must promote your app in a way that distinguishes it from the hundreds if not thousands of other apps in its segment of the market.

It’s all such a pain.

Otherwise, you can develop a PWA that takes much less time to release and even ranks on Google. Seems like a no brainer and many companies are hopping on board and providing PWA experiences for their users. Add to that the plethora of testing software some of which we mentioned above and you have a significant recipe that may not signify the death knell of the native app but may hasten its demise all the same.

To conclude, mobile and web application testing teams use open source solutions extensively. These solutions are helpful, as the above examples of PWA testing tools show. However, open source solutions have some limitations. PWA testing in an enterprise environment requires a wide variety of devices and browsers. SeeTest mobile device & browser lab integrations with open-source solutions address this.

Jonny Steiner – Content Manager