Test Management in TeamForge
CollabNet TeamForge ALM covers all phases of the life cycle from plan, code, build and test to deploy. With the recent addition of TestLink, a widely adopted open source product for test management, TeamForge now has the added capability to create test case trackers and associate them with requirements. With TeamForge, users have the ability to execute test cases and store test results. TestLink utilizes a tracker to store test cases and also tie Test Plans into build. By integrating TeamForge with TestLink the test management features are available in TeamForge as part of end-to-end ALM solution.
The main idea behind this integration is to associate Test Suite to the TeamForge requirements tracker epics, stories and tasks. The execution of Test Plan creates defects in TeamForge with associations to other TeamForge tools and data as shown in the following illustration.
Implementing Enterprise Agility
TeamForge’s open architecture allows customers to integrate their favorite open source tools, commercial and home grown tools into the software lifecycle. TestLink leverages the ‘Open ALM Platform’ nature of TeamForge to provide a seamless, fully packaged and fully supported set of test capabilities for all users who want to participate in collaborative test management for local and distributed teams alike. This integration ensures complete traceability of all artifacts in the software lifecycle from requirements to test plan through test execution and reporting.
TeamForge-TestLink integration results in extended capabilities both at a project level and at a company-wide community level – i.e., the organizational “Community Architecture”. The TestLink integration lets managers and stakeholders access continuous Test Management reports in TeamForge project pages. These real time reports bring a consolidated view of the TestLink within TeamForge, which is seamlessly viewed by the company wide stakeholders who have appropriate permissions. In addition, the workflow between TeamForge and TestLink can be rolled out to all projects and have a dramatic impact on improving overall release cycle velocity, software quality and reducing costs.
The TeamForge-TestLink integration is based on CollabNet’s Integrated Application Framework (IAF) similar to how Review Board was integrated with TeamForge. Using the IAF framework, single sign-on between TeamForge and TestLink is managed, with TeamForge project and user creation replicated to TestLink. In addition, the TestLink permissions are displayed in TeamForge during user creation.
- TestLink has a requirement management tracker that can be linked to multiple bug trackers. As part of this integration, it is assumed that requirements and defects are managed in TeamForge.
- The TeamForge-TestLink integration was created with multiple use cases gathered via interviews with current and potential customers. Enhancements and feature request will evolve as user experiences evolve.
- Test Case: A set of inputs, execution preconditions, and expected results (outcomes) developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.
- Test Suite: Organizes Test Cases to units and structures Test Specification into logical parts.
- Test Plan: Created to execute Test Cases. Test Plans can be made up of Test Cases from the current Test Project. Test Plan includes Builds, Milestones, user assignment and Test Results.
- Builds: A specific release of software. Each project in an organization is most likely made up of many Builds. If there are no Builds created for a project then the execution of test cases will not allowed.
The above capabilities are hierarchical, facilitating complex test case management. For instance, the Test Suite is a container that can hold multiple test cases. Test cases can be used in multiple Test Plans. The same Test Plan can be executed against multiple builds.
While creating requirements in TeamForge users can create Test Suites as a placeholder in TestLink. For every requirement, the user can create a Test Suite in TestLink. Later, the user can get into TestLink and create test cases for the requirements. Once test cases are created, a Test Plan that bundles the test cases to be executed is created. Test Cases are reused in multiple Test Plans. For example, one would like to run all tests of a feature ‘A’ that has four test cases in Sprint 1. Later, a test plan, for example TestPlan1, can be created and all the four test cases can be associated to this Test Plan1. The same test cases can be reused in TestPlan2 as shown above. Once the Test Plan is established, it can be executed and the user can update the test result as Pass/Failed/Blocked. For failed test cases a defect is automatically created in TeamForge defect tracker and association is created between the defect and the test case to maintain traceability.
Integration Out of box features
TestLink has the following features that can be leveraged by users:
- Import/Export – Test Suites and Test cases can be imported and exported between projects in XML format.
- Email notification – Notification about test case assignments and test reports (once the report is generated in TestLink) can be sent to users via email.
- Assign keyword tags – Keywords are tagged to a group of test cases and these keywords can be used to categorize test case groups.
- Custom fields – Users can extend the most important objects (Test cases, Test suites, Test plans) in TestLink with customized fields.
- Support for Chinese, Japanese & Korean languages.
- Version Test Case – There is an option to version every test case and the user can visually identify changes between Test Case versions using TestLink.
CollabNet TestLink Integrations to Other Tools
The Test Case execution is done manually, wherein users, after testing a case, will manually update PASS/FAIL in each test case. There is an option for users to use Selenium to run all test cases and update the TestLink with PASS/FAIL. Selenium integration is available for functional testing and there is a JAVA API to update the test results in TestLink.
Similarly, the Build artifact created in TestLink is a placeholder and the user has to create the build artifact manually to assign Test Plan to a build. Using TestLink Jenkins plug-in, the build job in Jenkins will create build artifact automatically in TestLink when a build is successful. In addition, few test management reports from TestLink will be displayed in TeamForge project page.
Reports & Dashboard
The CollabNet TestLink integration provides a wide array of out-of-the-box reports that can be used to generate the following for all permissioned users in the TeamForge system
- Test Report on build
- Test Case Assignment Overview
- Failed, Blocked, Not run Test Cases
- Test Case not assigned to Test Plan
- Test result Matrix
The CollabNet TestLink integration also provides charts that show execution results by Test Suite and overall test metrics.
Additional dashboards indicate progress by project:
Customer Migration Path
The CollabNet customers have two migration paths. The first one is an existing TeamForge customer who is interested in using TestLink. In this case, a migration script has to be run to synchronize TeamForge users to TestLink. Once the users are migrated to TestLink, the permission to access TestLink for migrated users has to be manually created in TeamForge. The following illustration shows the migration path.
The second migration path (casually called as glue) is for customers who are using TeamForge and TestLink as two separate systems that are not integrated. In this case there will be bi-directional migration, in which the TeamForge users are migrated to TestLink and the permission for the migrated users to access TestLink are created manually in TeamForge. The migration also moves selected project and its users, roles and permissions from TestLink to TeamForge. In addition, it will also auto-migrate requirements from TestLink to TeamForge and then associate the migrated requirements to Test Cases.
The CollabNet TeamForge-TestLink integration builds on top of TeamForge’ s capability to support the software development life cycle from plan, code, build and test to deploy. With TestLink integration, TeamForge now has the extended capability to create test case trackers and associate them with requirements and provide traceability and test management right from requirements through release.
The initial user feedback has been encouraging. Register to attend my webinar on November 18th 2014 to learn more. I’d look forward to hearing your feedback and comments about what features you would like to see added in the future.