Continuous Testing data sheet

Parallel execution

Increased coverage  

Desktop browsers:

• Chrome (all versions)

• Safari (all versions)

• IE (all versions)

• Firefox etc…

Run your tests in parallel across any number of desktop browsers

Developing and debugging web apps

Interact with your web application on physical devices 

• Numerous devices

• Any OS version

• Browser versions

Debug web and hybrid applications directly from the development environment 

• Safari for Developers

• Chrome DevTools

• Device

• Operating system

• Configurations

• Network setting

Manual testing

Web based interface 

View, control device screens, and manually test your application using a web interface — no installation required

Fast device interaction

View and control device screens without latency, as if the device is connected locally. With minimal bandwidth requirements

Full device control 

• Support all gestures including: Multi-touch, swipe, flick, drag & drop, zoom in and out, scroll including conditional scrolling, shake, pinch, force touch (iOS), tap, and custom generated gestures

• Full power cycle: Reboot your device, reset, or disconnect the USB connection

• Control all physical and virtual device buttons: Home, volume control, power, recent apps, portrait/landscape mode, send calls/SMS to the device

• Quickly switch between different versions of their applications

Test complex use cases

• Test all the functionality of your applications: Location-based feature testing – set any location of a device as necessary for testing (GPS simulation), test login authentication with TouchID, password, pin, pattern

• Test audio features such as Siri integration, voice to text, and more test image scanning (e.g barcode and check scanning)

Performance testing

Network virtualization

• Simulate different network profiles (different bandwidth latency and packet loss)

• Simulate different on-the-move scenarios (driving through a tunnel, riding an elevator, etc.)

Device vital online monitoring

• Monitor device and application’s CPU, memory, network, and battery

• Analyze device resource usage over time

Complex use case testing capabilities

Location-based feature testing 

Test geolocation-based applications by allowing users to set any location of a device as necessary for testing

Authentication feature testing

Test login authentication with TouchID, password, pin, pattern

Audio feature testing  

Test audio features such as call, voice to text, and more

Trigger incoming SMS/calls

 Send calls/SMS to the device

Non-functional testing capabilities

Device conditions 

Monitor different device conditions: device CPU, device memory, device battery consumption including a detailed breakdown of energy consumption causes

Network conditions 

Simulate any network condition like server location, user location, network signal, tech parameters bandwidth, jitter, latency, mobility aspects – user hopping cells, losing reception in tunnel or deteriorated reception in elevator

Run your code on a physical remote device

• Numerous devices

• Any OS version

Debug native mobile applications directly from the development environment

• XCode, including inserting breakpoints

• Android Studio

Perform debugging on the device and conditions where errors occurred

• Device

• Operating system

• Configurations

• Network setting

• Android Studio

Simple test development

Test recorder    

Record and play your tests, then automatically export your recorded test to code; use the same test script for different mobile OS

Object spy 

View your application’s element structure and generate identifiers with ease

XPath query

Support XPath identifiers, along with unique XPath generators and functions

Execution command and element highlighting 

Mark the targeted element of each command with gesture annotations and bounds

Object repository as well as dynamic identification

Create common objects or descriptively identify your elements in run-time

Offline analysis for element identification failure 

Open failed reports within the debug window to identify root causes

Test code generation

Generate test code for all test executions

Advanced object identification

• Identify objects by their properties

• Identify customized UI elements

• Identify by image or text recognition

Increased test coverage

All app types 

Test on native, web and hybrid applications, system, and MDM apps (device settings, SMS verification), and non-packaged apps (Facebook, LinkedIn etc.)

Location-based feature testing  

Test geolocation-based applications by allowing setting any location of a device as necessary for testing (GPS simulation)

Authentication feature testing

Test login authentication with TouchID, password, pin, pattern

Image scanning testing  

Test image validation stories (e.g barcode and check scanning)

OS support 

• Immediate support for all general availability (GA) mobile OS released to the market:

• iOS

• AndroidSupport for all beta versions of mobile OS within two weeks from release to the market

Powerful test & automation development tools

Providing immediate visual feedback on all actions

Powerful object identification and management  

• Object spy – Identify objects by their native/web properties (also image/OCR)

• Identify customized UI elements

• XPath query

• Object repository and dynamic identification

• Drag and drop script editor

Appium test capabilities management  

• Automatically generated test capability requirements

• Device, application, version

Verify and analyze scripts  

• Script “story” view – visualize the impact of every line of code

• Command and element highlighting during test run

• Offline analysis for element identification failure

Leverage pre-configured Eclipse project framework assets for mobile including

Code snippets, properties, and configuration files

Simple development and execution of robust tests

Understand application structure and write robust, maintainable tests scripts

• Object spy – View your application's element structure, and generate identifiers with ease

• Test recorder – Record and play your tests, then automatically export your recorded test to code

• Documentation, videos, free online training

Develop tests using an intuitive user interface  

• See a device reflection

• Simple test capabilities management (device, application)

Powerful IDE integration for automation projects

• Code export

• Automation project boilerplates

• Real time visibility of automation run – including execution commands and element highlighting

Integrations

Execute tests in any language, test framework, and IDE

Any programming language:

• Java, C#, Javascript, Perl, Ruby, Python

Any test framework:

• Appium, Selenium, XCUITest, Espresso

Any IDE:

• Eclipse, IntelliJ

Execute Appium/Selenium/Digital.ai Continuous Testing, formerly SeeTest, mobile tests  

Develop and run large Appium/Selenium/Protractor/WebdriverIO automation projects on multiple real iOS and Android devices.

Integrate with native mobile testing frameworks  

Execute XCtest and Espresso tests on multiple iOS and Android devices in parallel

Parallel execution

Increased coverage 

Run your tests in parallel across any number of desktop browsers, real mobile devices, simulators, and emulators

Supported platforms  

• Mobile – iOS (all versions), Android (all versions)

• Desktop browsers: Chrome (all versions), Safari (all versions), IE (all versions), Firefox, etc…

Execution management

Leverage grid intelligent execution management

• Dynamic test scheduling, execution, and distribution according to specific attributes such as device manufacturer, model, OS version

• Mass deployment of app on targeted devices (RESTful API)

• Automated provisioning process for apps and services

Continuous integration

Integrate with any CI environment 

Integrate with:
Jenkins, Bamboo, TeamCity, Hudson

Advanced grid capabilities

Grid view 

Provide a grid view of all executions in real time

Monitoring capabilities

Network statistics 

• Runs your tests under different network conditions

• Simulate server and user location, network signal, etc.

Monitor device vitals  

Monitors the device vitals during test runs (CPU, memory, battery)

CI/CD integration & parallel execution

Out-of-the-box mobile device grid for large scale execution

Pre-configured out of the box; no need for additional installation and configuration

Intelligent grid execution management  

• Real-time view of all executions

• Dynamic test scheduling, execution, and distribution.

• Ability run multiple CI projects in parallel (multiple applications).

Seamless integration into CI environments

Providing quality status at a glance, reducing environmental noise and highlighting real issues

Quality reporting and analytics and insight 

Immediate support for all general availability (GA) mobile OS released to the market:

• iOS and Android Support for all beta versions of mobile OS within two weeks from release to the market

Quality dashboard

Single snapshot

• View key information on all of your applications

• Test execution status

• Test coverage rate

Root-cause heat map

Identify errors based on platform, manufacturer, OS, screen size, test, etc.

Alerts

Trigger key failure alerts

Test coverage analysis

Test coverage 

Define test coverage matrix dimensions to suit your specific needs

Multi-execution analytics 

• Identify gaps in your coverage matrix

• Receive trend reports

Execution specific analytics

• Execution progress

• Compare test execution coverage against baseline for better context

Root-cause analysis

Failure grouping

Group passing tests and omit failures within tolerated levels

Repeat failures 

Weed out repeat failures (e.g. environmental issues, functional aspects, components) and suggest root cause

Failure correlation analysis

Application sensitivities 

Automatically identify sensitive areas in your application (stability, performance, functionality)

Causes of failure

Highlight key factors that cause your test to fail

Eliminate test fluctuations

Redundant tests 

Filter out redundant runs and repetitions

Environmental noise 

• Define rules to reduce environmental noise

• Define minimal pass requirements for functional testing (single pass may indicate functional success)

• Create thresholds for unstable results

Test grouping  

Group similar test runs for clarity

Eliminate test fluctuations

Central management

Centrally manage the fault status, including investigation results and action items

Summary reports 

Consolidated management summary reports including fault categorization and resolution

Investigation workflow

• Assign groups of tests to organizational units

• Receive updates and feedback

• Manage status

• Integration with defect management systems

Data sharing  

Enable data sharing with web access to central test result repository

Customizable reports

Focus areas

Create new reports based on your key focus areas

Report types    

• Test status

• Coverage reports

• Heat maps

• Trend reports – historic analysis of your tests to evaluate progress and identify issues

Reporting module  

 Pluggable reporting module

Support for all test platforms

Testing frameworks and defect Management systems 

All testing frameworks and defect management systems are supported

Test types

All test types supported:

• Unit, smoke, sanity, functional, non-functional, UI functional, persistence, accessibility

QA view  

 End-to-end QA view including web, mobile and API

Enterprise-level security

On-premise set up  

• Retain your company security level, as your devices will reside inside your company’s VPN

• Test on an independent physical, and network environment including rack, servers, network equipment, and devices the racks are Wi-Fi shielded for full network security

• Support both TLS and SSL security protocols

• Use LDAP information to access the cloud

No USB needed  

Eliminate the need to unlock USB ports in your organization

Physical device security  

Secure your device inventory in a datacenter to prevent lost or missing devices

Malicious software security  

Prevent unwanted device upgrades and block the device settings including the Wi-Fi network

ISO security 

• ISO 27001 & SOC 2 certified

• TLS and SSL protocol support

• Multi-factor authentication/single sign-on (SSO)

• Customizable cleanup processes

• Access restriction

• VPN connections

• LDAP integration

Others  

• Eliminate the need for active x or flash; operate in a purely HTML coding environment

• Secure user portal

• Control access based on role

• Create strong passwords with password complexity enforcement, and a password aging policy